The Cruel Supply Chain Economics of AI

The current bloodbath of foundation model startups in the AI space was inevitable, with inflection.ai and stability.ai being the latest victims. The fact that Microsoft and open-source players like Meta seem to be the big winners is a history lesson for those who didn’t experience the last battle of the titans when Microsoft, Google, and Facebook fought for Internet domination.?

Microsoft was initially the big winner last time around when it defeated Netscape, but then open-source won the long game with Firefox, Chromium, and WebKit, ultimately winning the internet browser wars and Mark Zuckerberg learning his lesson as he ended up not owning any of the major distribution platforms (search, browser or mobile).?

Satya Nadella might be all around OpenAI, but open-source is all around him. Having learned his lesson, Zuckerberg is a major sponsor of the open-source ecosystem. However, in the long run, I don’t think many moats will be based on technology IP in this round of the big tech battle; it will all be about the supply-chain economics of AI, and while open-source can be used to deny a win to the competitor it will be hard to use alone as a winning strategy.?

This is also why the bloodbath was inevitable. The economics of foundation model startups simply don’t make sense. The technology's current limitations make solving the supply chain and distribution problems of huge AI models impossible within the limited scope of funding and scale of most of these startups. They are trying to solve a supply chain and market problem, not a technology one, and unfortunately, supply chains can stay rational a lot longer than (VC-funded) companies can stay solvent .?

The dilemma is quite simple, AI is an order of magnitude more expensive to deliver than previous generations of knowledge technology, and the buyers (consumers and businesses) either think it is too expensive for what it does (consumers) or does too little for what they need (business).?

This is why chatbots with retention problems across the board and a proliferation of copilots and RPA 2.0 start-ups are the only use cases that have made any sense so far. In the last battle of the Internet, web companies could ride the economies of scale of Moore’s law as chips got cheaper to deliver significantly cheaper products to end consumers or more specialized applications to businesses (SaaS). Unfortunately, I think that will be a lot harder this time around.?

If we consider the core business components of any AI business, it consists of

• People => IP
• Compute => Energy cost
• Data
• Market => Distribution?

A company must have an edge in one or several areas to be competitive.?

IP, while important, is hard to retain an edge in since the academic and open-source community is pushing breakthroughs as fast as they are available. Thanks to Zuckerberg, there will be no lock-in on IP in the foundation model layer. There are no moats, and building a business around foundation model IP will be hard.?

Compute is interesting as this combined with Moore’s Law to give us the cloud revolution a decade and half ago. Falling cost of chips combined with broadband enabled new business models, but beyond enabling cloud compute has not massively changed our application features. The reason for this is quite simple: compute scaled with the number of users. There was no reason to add exponential amounts of computing server-side as the number of web apps and online use cases grew. Your software did not become twice as bright with twice the compute unless someone wrote some code to utilize the extra compute and, more importantly, monetized it.?

With LLMs, this is different. More compute equals better models, both training and inference, leading to better applications and more use-cases, making it a distinct competitive edge. The more compute, the better experience, independent of any other parameters. This means that compute will quickly translate to the underlying constraining resource of energy cost as that will be the only bottleneck.?

This is why Sam Altman, our new techno-messiah overlord, predicts compute and energy will be our future currency. There is only one problem with that theory: Unless you own a startup making fusion plants, it won’t matter for most people trying to build a business in this space. Secondly, even if you do own a fusion plant startup (and Sam does), energy is a long game with many systemic bottlenecks, so not really relevant for most companies trying to compete now.?

The right market or distribution can solve most technology problems. From a consumer perspective, ChatGPT is famously the fastest application to reach 100 million users. The only problem is that growth is slowing. A year later, they have 180 million users, while they should be closer to 360 million users following classic exponential growth like TikTok, Facebook, and others.?

Without distribution, there is just not enough consumer demand for AI products. And worse is that the places where this demand could arise (an actual intelligent agent on your phone or an AI-controlled browser) are all controlled by incumbents such as Apple (1bn plus users), Google (4.9bn users), and to some degree Microsoft (1.2bn users). My guess is that if OpenAI fails to gain significant market share, all other foundation model startups are worse off with less hype and consumer interest.?

For those who can’t make it as a consumer app, there is always enterprise, and surely enough, many of the foundation model companies have now pivoted to B2B. Even OpenAI strangely announced that they would make an expense app at some point (I'm not sure how that will pay for their server bills anytime soon). The question for most foundation model startups is what they will offer companies they can’t get elsewhere.?

Azure, Google Cloud, and AWS will deliver very capable open-source models directly to companies at a meager cost (Zuckerberg's investments ensure it will be a race towards commoditization that keeps going). Even worse, while initially interested, most companies have met copilots and current versions of bolted-on chatbots with a resounding “meh.” Companies I talk to agree they would pay more for AI that can do more, but the current offerings seem like overpriced gimmicks.???

That leaves me the last and most exciting parameter, data. We already know that the only significant variance in the LLMs right now is the data on which they have been trained. Since they have all used the same public data, they are mostly the same, which is also why most consumers won’t really get the difference between them.?

From a supply-chain perspective acquiring data is also the most expensive component. Effectively there is a limited amount of public data available, and it has taken decades and cost trillions of dollars of infrastructure and human labor to create. Given the size of these models and the amount of parameter growth between each generation, this will be a scarce resource for any LLM.?

To generate enough varied data to scale the parameterization to anything close to AGI we are talking about at least 10-50x the amount of data, and since we have already used 90% of the public data in the models, that will be almost impossible to achieve by scaling existing transformer architectures. This will be even worse in the enterprise space, and the large foundation models will lose their most significant edge as data becomes proprietary.

The economically better alternative is to use synthetic data, which costs a fraction of real data to generate, but to do that, you will need clear success criteria. AlphaGo Zero beat AlphaGo using synthetic data precisely because the game of Go has clear rules for winning so an outcome can easily be validated. In other words, to effectively create a competitive edge in enterprise AI, defining clear winning criteria and having easy validation is worth far more than a bit more publicly acquired data as it will allow you to create comparatively larger models work from smaller amounts of data – more end-user utility at a much lower cost.?

The other major frontier to conquer when it comes to data is time. Right now, all LLMs are primarily trained on a snapshot in time, but if we add time series as a dimension to the models we train on, and if there is enough variance in our data, we can quickly scale the number of parameters available. It further has the advantage that you can start creating models that don’t just predict the next token or word but the next event or action. Time brings down the cost of data and increases the value of the models for the customers significantly.?

This leads me to my conclusion: most foundation model startups are screwed.?

The consumer market depends on distribution, and right now, there is not enough differentiation to pick Inflection or OpenAI over others, which means the big players will win by default. Pivoting to enterprise requires a whole different infrastructure and features that are not model specific but the result of excellent software engineering and product design, not just a chatbot.?

And here the field is wide open…