Reflections on the Shifting AI Landscape

Now that the dust has settled from last week’s Deepseek announcements, I wanted to take a moment to share some of my thoughts on the changing AI landscape.

The recent announcements have significantly reshaped the AI industry—not just because of Nvidia’s stock drop or the implications for US AI infrastructure but because they highlight a shift in how companies approach the future of AI.

1. The Previous Landscape

For a while, many in the industry believed the AI landscape was fairly settled for the next few years:

• LLM Companies: Many major players in the LLM space have raised and spent billions of dollars, making it seem like no one could catch up. These companies have focused on integrating additional functionalities into their solutions, leaving little room for wrapper technologies or new products. This was exemplified by offerings like Operator, Canvas, Projects, and the highly publicized O1 model.
• Big 3 cloud providers: These tech giants have been vertically integrating everything into their technology stacks, leveraging tight relationships with LLM providers, making it difficult for smaller companies to compete.

2. The New Challenge for LLM Companies

LLM companies now face a new challenge: their previously unassailable positions no longer seem so secure. All the billions invested in training must now contend with “free” open models hosted by various providers. More importantly, enterprises are now more hesitant to adopt wrapper technologies from LLM providers, as they don't want to be locked into a single provider and lose flexibility in switching LLMs in the future.

3. A New Era for Frontier AI

Before last weekend, if you wanted access to frontier AI, you had to rely on hyper-scalers. Closed-source LLMs were only available through large companies with corporate agreements in place with LLM providers. Hosting your own LLM was an option, but access to frontier models wasn’t.

This has now changed. You can host frontier LLMs locally or work with companies like Groq and SambaNova to set up dedicated cloud instances, democratizing access to cutting-edge AI.

4. The Rise of Smaller Distilled Models

Additionally, smaller distilled models are becoming more accessible. We now can take a frontier model and create a distilled smaller model. These models can achieve around 98% of the accuracy of their larger counterparts, but at a significantly lower hosting cost. With closed models, we had little visibility into what the distilled models were, but now we can explicitly make a choice between using the larger model or the distilled model.

5. Impact on Cost and Hosting

The net result is a significant decrease in the cost of frontier LLMs, making it possible to host these models locally or in private clouds, further disrupting the AI ecosystem.

6. How This Affects Kavia

This shift is a huge boon for companies like Kavia. Our focus on agentic LLM usage for all aspects of software development requires a lot heavier LLM usage than many others who focus mainly on recommendations. We also rely heavily on frontier LLMs to achieve our results. These recent developments enable us to cater to enterprises that prefer privately hosted LLMs. More importantly, it makes the cost-benefit equation much more favorable, allowing us to make a stronger case for switching their workflows to our platform.

These developments also validate our approach of leveraging frontier LLMs for software development. While others have taken in $100+ millions of dollars in investments and invested heavily in building and training custom LLMs, those efforts now seem like a questionable strategy. With cheaper frontier models available that surpass their custom LLMs— at a fraction of the cost—they’ll need to rethink their approach to building a price competitive solution.

I am really encouraged by the progress made by open source LLMs and am looking forward to all the new use cases that this enables.