Liquid Foundational Models (LFMs)

Liquid Foundation Models (LFMs) were recently introduced by Liquid AI , a startup spun off from MIT. The company, founded by researchers like Ramin Hasani and Mathias Lechner, specialises in developing AI systems that diverge from?popular transformer-based architectures. LFMs build on earlier work with liquid neural networks, inspired by brain-like dynamic systems capable of adapting over time.

Confused? Let's break it down.

Why are they called "liquid"?

The models are called "liquid" because they have a model architecture that can adapt and adjust when given new information, much like how a liquid changes shape depending on its container. These models are inspired by the brain’s ability to stay flexible and learn even after they’re trained, which makes them different from traditional models that remain the same once trained. This is different from the commonly known models like GPT and Gemini that use the transformer architecture.

Hold up! What is model architecture?

A model's architecture is the structure or design of how it processes data. Just like a building has an architecture that guides its shape and function, an AI model's architecture defines how it works with data, what kinds of tasks it can handle, and how fast or efficiently it can do them.

... and "transformers"?

Transformers are a popular type of model architecture that has been very successful in AI tasks like language translation and chatbots. Models like ChatGPT use transformers because they are good at looking at large amounts of data all at once and finding patterns.

I mean, ChatGPT is pretty successful. Why make anything different?

Transformers, however, use a lot of memory and computing power. Liquid models don’t use transformers. Instead, they use a different design based on math and systems theory, which makes them more memory-efficient, especially when dealing with long pieces of data or when running on devices with less memory.

What does "memory-efficient" mean in this context?

A memory-efficient model can do its job without needing a lot of computer memory (storage used by programs when they’re running). This is important because it means the model can run on smaller devices like phones or laptops, and it uses less power. Liquid models are designed to handle large amounts of data without using too much memory, which makes them different from models like ChatGPT, which need a lot of memory to run smoothly.

So... is this all just a theoretical novelty or are there any practical improvements using liquid models?

Liquid models make it possible to do tasks that require handling large amounts of data (like analyzing long documents or videos) on devices with limited memory or computing power, such as phones or tablets (commonly called edge devices as they are usually at the edges of networks). They also allow for more efficient use of resources in larger AI systems, meaning that powerful AI can be used in real-time applications like chatbots or document processing without needing as much hardware as before.

For everyday users who use AI to generate text, look up information, or have short conversations, liquid models may not feel very different from models like ChatGPT. However, liquid models can handle more complex tasks, like understanding longer conversations or reading big documents faster, with less strain on the computer. So while basic tasks won’t change much, these models could make AI feel faster and more responsive in more complicated tasks.

I still can't tell if this is a paradigm shift or a minor improvement...

When in doubt, Ask the Audience!

Public reaction to Liquid Foundation Models (LFMs) has been largely positive, with many seeing them as a potential breakthrough in AI. The key innovation is their memory efficiency, which allows them to process longer sequences of data without the massive memory requirements of transformer models like ChatGPT. For instance, the LFM-3B model can handle tasks with long-context input (like lengthy documents) while using significantly less memory than models such as Microsoft’s Phi-3.5 or Meta’s Llama series.

LFMs are also being praised for outperforming similarly sized models across benchmarks. For example, the LFM-1B has set new standards in the 1 billion parameter category, competing with larger models while using fewer resources. However, while they’re impressive on paper, some experts are cautious, noting that the real test will come with broader adoption and real-world use

Sounds good, can I try it?

You can! Go to the playground here and try your usual prompts across various models. You can't do multimodal stuff yet in the playground, but keep an eye out!