The AI Plateau: Myth or Reality?

The AI "bubble" is a hot topic. Many claim it's about to burst, that AI progress has plateaued. I disagree.

This article will explore why I believe this is a misconception. The truth is, we've barely scratched the surface of AI's potential.

Beyond the Hype

Gartner's Hype Cycle perfectly illustrates the current sentiment. The initial excitement over generative AI led to quick adoption, revealing limitations as we built real-world applications.

But does this mean AI is useless? Absolutely not.

We're on the brink of a truly exciting phase: deploying AI in production-ready automation and software that delivers tangible value.

My recent LinkedIn post (August 7th, 2024 as of writing of this article) outlines some challenges in building practical AI solutions. None are deal-breakers, but as with any new technology, we need time to learn, experiment, and refine.

Improvements that Matter

Three key developments stand out this year: small models, larger context windows, and multi-modality. Each holds significant potential.

• Larger Context Windows: Led by Google and Anthropic, increased context windows enhance "in-context learning" – feeding LLMs relevant information within the prompt for better understanding. Imagine being able to give an LLM a whole glossary of technical terms before asking it to summarize a complex engineering report; the results would be far more accurate and insightful. The challenge lies in the "needle in a haystack" problem, where LLMs tend to lose track of information as the context grows. Google seems to be addressing this, making larger context windows more practical, although it's worth noting that using larger context windows can impact performance and cost due to increased token processing.
• Multi-modality: While large language models have revolutionized text-based tasks, the integration of multi-modality is expanding AI capabilities far beyond words. Gemini, OpenAI, and Anthropic are at the forefront of this shift, enabling AI to analyze audio and video, not just text. This opens doors to understanding tone and emotion, generating real-time video edits from simple prompts, and even reducing latency in conversations by processing speech and text simultaneously. Think of a virtual assistant that can not only understand your words but also your tone of voice, providing more empathetic and contextually relevant responses.
• Small Models: Efficiency and accessibility are key to making AI practical for everyday use. That's where small models come in. Models 1/100th the size of GPT 3.5 Turbo are now fast, reliable, and surprisingly capable. Open-source options like Llama 3.1 rival industry leaders, while smaller models like Gemma 2 can even run on your phone, offering enhanced privacy and opening up new possibilities for deploying AI within company networks without relying on cloud infrastructure. The increased speed and reduced latency of smaller models unlock new applications and significantly reduce costs, especially when combined with larger models for specific tasks.

The Road Ahead

Current LLMs have limitations in reasoning and can "hallucinate" facts. However, the emergence of video generation models that learn real-world physics is a game-changer. These models can simulate fluid dynamics, gravitational forces, and other natural phenomena, leading to more accurate simulations and a better understanding of how the world works.

This opens doors to improved automation, robotics, and even virtual environments for training and research. I also anticipate LLMs evolving to plan ahead and generate better answers, potentially leading to more useful autonomous agents capable of complex decision-making.

Conclusion

There's immense value to be gained from AI's current state, and even more potential lies ahead. Building applications now means reaping immediate benefits while laying the groundwork for future improvements.

AI is only going to get better. The next hype cycle might be just around the corner. I encourage you to start your AI journey today – it's a real opportunity for a head start.