Is AI Slowing Down?

Beyond Scaling: How AI World Models, Inference Clouds and Autonomous Agents Are Shaping the Future

It seems we're on the brink of meeting OpenAI’s new creation, an agent charmingly christened “Operator”, slated for an early debut in January. This couldn’t come soon enough, considering Google’s rather public grappling with what we might as well dub the “AI existential crisis” of last week.

The noise, of course, is partly a byproduct of OpenAI's latest clickbait showdown with The Information, the big tech gossip column that seems to revel in pouring petrol on every spark it finds.

So here’s the tea: OpenAI’s latest wunderkind, the Orion model—or as we mere mortals call it, GPT-5—is reportedly not hitting the high notes we all expected. It’s like watching the sequel to your favourite film, only to realise the scriptwriters got complacent.

Between GPT-3 and GPT-4, we saw a leap akin to Concorde first taking flight: unexpected, exhilarating, and undeniably ahead of its time. But now?

Now it feels more like upgrading from a Volvo to a slightly shinier Volvo. Reliable, sure. But no one’s posting about it on Instagram.

Faced with this rather uninspiring levelling off, OpenAI has decided to double down—not by tearing up the blueprint but by sharpening their pencils and adding a touch more finesse.

Instead of the brute force push for sheer scale, they’re leaning into the subtler, more sophisticated arts of reasoning and fine-tuning, like a master sommelier coaxing the best notes from a tricky vintage. Whether this new recipe will finally hit that sweet, elusive flavour of breakthrough, well, we’re all standing by with bated breath.

A Long-Standing Debate on AI Progress

This isn’t the first time we’ve seen this old chestnut roasted. The debate over scale versus sophistication is practically the pub argument of AI development—endless, impassioned, and entirely unresolved.

On one side, you've got the scale evangelists, steadfast in their belief that more compute and more data is the golden ticket. Scaling Laws, they say, as if it's the AI equivalent of Moore's Law: just keep doubling the chips, and exponential performance is bound to follow.

But we’ve reached the point where simply throwing silicon and terabytes at the problem feels like strapping wings to a bulldozer—impressive, perhaps, but not necessarily the smartest path forward.

On the other hand, some argue that there is no path to AGI using the current architectures. This is why someone like Yann LeCun can confidently say:

"AI is dumber than a cat!" - Meta’s chief AI scientist and Turing Award winner

and the implications of this view are significant. But if you know me, you know I always carry an emergency pack of hope.

The Value of AI Today, Even Without Further Advancement

In LeCun’s view, concerns about existential threats from AI are greatly exaggerated. However, no one is arguing that AI isn’t extremely valuable even in its current state.

One of the things I often say is that:

Even if AI stopped developing right now, it would still take years for the world to fully adopt and adapt to the processes and capabilities that the current technology already enables.

I see this with my customers all the time.

OpenAI’s Orion Model: Expectations vs. Reality

Back in May, OpenAI CEO Sam Altman told his staff that he expected their latest frontier model, called "Orion" internally (not GPT-5), to be significantly better than last year’s flagship model.

At that time, only 20% of the training process for Orion had been completed, yet it was already performing on par with GPT-4.

However, after several more months of development, things appear different.

Insiders at The Information tell us that early testers of the Orion model admit it’s better—just not mind-blowingly better. Unlike the monumental leap from GPT-3 to GPT-4, this upgrade feels more like a half-step forward.

Sure, it handles language tasks with aplomb, but when it comes to coding—one of its star acts—it’s not necessarily outshining the old guard. A bit like debuting a shiny new ebook-reader from a certain forest named tech giant.

From the Age of Scaling to Discovery

Reassessing Scaling Laws: Shifts in Industry Strategy

This has led to a new approach in the industry regarding scaling. The information suggests that in response to the challenges posed by the slowing improvements in AI, the industry is shifting its efforts towards enhancing models post-training.

This represents a different kind of scaling law. OpenAI has reportedly established a "Foundations Team" to explore how the company can improve its models, especially given the increasingly scarce supply of novel training data.

Strategies include training Orion on synthetic data generated by other AI models and focusing on improving the model’s reasoning abilities during the post-training phase.

System Design Can Save AI Not Scaling

Former OpenAI Chief Scientist Ilya Sutskever commented on this issue, noting that the 2010s were the "age of scaling." Now, we are entering a new era of wonder and discovery, with everyone searching for the next big breakthrough.

For me, scaling the right aspects of AI is now more important than ever, echoing Yann LeCun’s point in the Wall Street Journal article:

The problem with today’s AI systems lies in their design, not their scale.

I’ve always held a strong view on this, as do the AI experts I respect most in this field. Regardless of how many GPUs are used, today’s AI architectures won’t lead to AGI.

Big tech is throwing metal and power at it and not enough finesse at system design. Similarly, clients are implementing GenAI in use cases that would benefit more from traditional tech at much lower cost.

The Data Problem: Exhausting the Internet’s Knowledge

A significant challenge at this point is that current frontier models have been trained on almost all of the internet’s data, therefore the impetus to shift in focus, away from pre-training, towards post-training implications.

The idea that OpenAI’s O1 model, an advanced reasoning model, is a distinct evolutionary path for LLMs. When Sam Altman announced it, he made it clear that this reasoning model represented a separate track from GPT-4.

Initially, it was assumed that OpenAI would pursue these paths in parallel, but it seems they are now leaning into the reasoning pathway. Confirming that traditional LLMs have reached a ceiling at least from a returns POV in the pre-training.

There are also questions about whether training on synthetic data could introduce new issues, although this remains unclear. Nonetheless, OpenAI seems committed to its focus on reasoning.

Google’s Industry Confirmation: A Wall of Diminishing Returns

The general chatter in AI circles is that this trend of diminishing returns is happening across companies working on leading large models. Very revealing reports from Google that an unexpected wall of diminishing returns despite increasing training data and duration.

Professor Pedro Domingos pointed out that scaling laws resemble S-curves, not exponentials, suggesting we might be approaching the mature stage of the curve.

Many researchers believed that models would continue to improve as long as they processed more data while using more specialised AI chips, but those two factors are clearly not enough.

Google’s earlier Gemini models saw impressive gains simply by feeding them more data and cranking up the compute—a textbook case of scaling laws in action. But now?

It seems that gospel is losing its shine. The notion that throwing in more data and silicon would yield endless improvements is being tested, scrutinised, and, frankly, found wanting.

It’s a relief to see this myth finally challenged; we might actually get some real progress now that the industry is forced to rethink its magic formula.

Inference Scaling: A New Law - The Second Scaling Law

As I have said in my OpenAI O1 article, this shift towards reasoning-based improvement requires significantly more computational power for inference.

I believe the industry is moving from massive pre-training clusters towards inference clouds—distributed, cloud-based servers optimised for inference processing.

• NVIDIA CEO Jensen Huang referred to this as the "second scaling law," focusing on inference rather than pre-training.
• Noam Brown, an OpenAI researcher, stated that having a model "think" for just 20 seconds during a hand of poker, provided the same performance boost as scaling up the model by 100,000 times and training it longer.
• The Information raised questions about the implications for data centres. They wondered if staying at the forefront of chip development will matter as much going forward.
• In other words, it’s looking like we won’t just keep stacking GPUs in massive data centres like it’s a Silicon arms race. Instead, the focus is shifting towards boosting the reasoning capabilities in real-time, right when the user types in a prompt. It’s a tacit admission: the era of squeezing magic purely from pre-training has hit its ceiling. Now it’s all about what the model can think on its feet (Inference time).
• A relatively small investments in inference-time computation can yield disproportionately large benefits. Although inference time reasoning will still require significant compute to serve answers, the focus shift will provide the scale needed.

Shifting Narratives: From Pre-Training to Inference Scaling

• For me the perceived roadblock in pre-training is a milestone, not an end. Pre-training was never meant to be the ultimate solution for neural network optimisation. The AI slowdown is a non-story. Click-baiting.
• The biggest reason if AI is slowing down in pre-training, is that there’s nowhere else to go. If you begin to saturate on benchmarks, nothing is left to do—100 out of 100 is the highest score you can get.
• Efficient scaling is now the focus, with researchers exploring inference scaling as a promising direction. OpenAI’s O1 model illustrates this approach, using enhanced inference for complex, layered decision-making tasks

Meta’s Alternative Approach for AI Progress: Yann LeCun Predicted This All Along

• LeCun has been predicting these diminishing returns from model scaling for years. This week, he posted on Threads:

• He was having a pot shot at former OpenAI chief scientist Ilya Sutskever and his fervent advocacy for the idea that you could just add more compute and data to continue scaling AI.
• LeCun and his team at Meta—Fundamental AI Research (FAIR) — are pursuing new architectures as a path toward AGI.
• Their current focus is on world models, which aim to train AI on how objects and environments interact, rather than just focusing on the connections between words.

In essence, LeCun's world model concept aims to create AI that doesn't just process data, but truly understands and can reason about the world in ways similar to human cognition. While it's an exciting prospect, it remains a long-term goal in the field of AI research.        

• This is a crucial step towards the next milestone of AI progress. We are fortunate to have the minds like LeCun's in this adventure.

OpenAI’s Big Bet: Autonomous Agents

• Additionally to all this, OpenAI is reportedly planning to release an autonomous agent next year. The agent, codenamed "Operator," can control a computer to complete tasks independently, including coding, shopping, and booking flights.
• According to Bloomberg sources, staff were informed in a meeting that the tool would be released as a research preview in January.
• We could see competing agents from Anthropic, Google, and OpenAI.

This is another strategy to make some gains or reduce diminishing returns of AI in general. However, this is filled with problems to overcome as we have seen with Anthropic’s Computer Use – it is wildly expensive to run.

The solution—it is not in pre-training now is it.

The Key Takeaway: LLMs Are Evolving, Not Failing

The main takeaway is clear: LLMs are not failing; they are evolving. Despite the sensationalist headlines, this is a natural part of product development.

The suggestion of slowed AI progress doesn’t align with what people inside the labs are saying. The Information clarified that researchers are finding new ways to enhance LLMs, focusing on test-time compute and inference. LeCun at Meta will no doubt break a ceiling with world models which could be ground breaking.

The AI industry is entering a phase of introspection and adaptation, moving away from a singular focus on scaling toward new methods of achieving breakthroughs. The coming years will likely see a blend of targeted innovation, strategic policy proposals, and shifts in business models as major players jockey for dominance in this evolving landscape.

The headline from The Information paints a bleak picture, but it’s a far cry from the mood inside the big labs. Technically accurate, sure—but the casual takeaway that AI progress has hit a wall is the complete opposite of what I’m hearing.

In fact, the rumblings were so off the mark that The Information had to backpedal and issue a bit of a clarification. Funny how the reality on the ground and the media spin can end up worlds apart, isn’t it?

For me the effective performance of LLM systems will depend on complementary innovations that extend its usefulness and perhaps many years of individuals and organisations finding the right use cases and configuring their workflows to leverage them.

Big Tech still seems fixated on the brute-force path—more chips, more data—while largely ignoring the elegance of smart system design and leveraging the tech we already have. And that, I reckon, leaves a golden niche wide open for the rest of us. 

So, here’s the question: with your system design know-how, what can you fuse with AI today to unlock real, immediate value? It’s not about throwing more horsepower at the problem; it’s about driving it with finesse and making it real for the customer.

#AI #SystemDesign #InferenceClouds #OpenAI #TechInnovation