AI is Killing Innovation. That Has to Change.

Part 3 of 4

From Bare Metal to Black Box

In my view, we need to democratize the AI tech stack – from silicon to servers, data centers to virtualization and the cloud, all the way to AI development platforms. We already see several excellent companies doing this work. Groq, for example, is innovating at the silicon level. Lambda is making tremendous strides at the cloud level. But no one gets all the way down to the root of the problem: compute.

Bluntly, the system for buying compute…DOES NOT COMPUTE.?

• Lack of market transparency DOES NOT COMPUTE: The reality most people don’t see is that compute buyers are flying blind. There’s no real market price – access to the same GPU service can range from $1.58 to >$8.50 per hour. When you do secure access, you have no way of knowing if it’s a good deal or if it will work.

• Vendor lock-in DOES NOT COMPUTE: Long-term contract lock-ins force you to bet on your future needs, often leading to paying premiums or being stuck with unused capacity.?
• Managing multiple vendors DOES NOT COMPUTE: Different providers, different contracts, different tech stacks…it’s like trying to run your workloads across a dozen incompatible ecosystems.

• Unused capacity DOES NOT COMPUTE: Pessimists believe that 10% of compute is actually effectively utilized. I'm not that negative; but I’d say 50%. Even if we were at 99% utilization in the most optimistic scenario, we're still talking about waste on the order of trillions of dollars! To me, that's unacceptable.

• Heterogeneity DOES NOT COMPUTE: One of the biggest reasons we have such under- and poorly utilized capacity is because of the heterogeneity of compute. An H100 chip is an H100 chip until the moment you put it inside someone’s proprietary cloud. The trouble is, as you scale and look to migrate vendors to meet your needs, you’re going to spend a ton of money moving from cloud to cloud. And, eventually, you’re going to get stuck in some vendor’s cloud where they can (and probably will) jack up the prices on you.?

The compute providers themselves aren’t faring much better. They’ve had to build expensive orchestration systems to compete – without upfront guarantees of revenue. When demand fluctuates, GPUs sit idle, burning capital and eroding profitability. Overbuilding to anticipate demand, only to face market downturns, creates massive inefficiencies that inflate costs for everyone and lock smaller players out of the market entirely.

Here’s where commoditization could be transformative for access to raw computing power. In fact, the innate properties of compute naturally lend themselves to standardization and liquid markets:?

• Compute exhibits uniformity as AI’s foundational infrastructure, analogous to power transmission in electricity
• Compute represents an essential raw material for model development
• Compute commands a massive market with capital expenditures in the trillions already
• Compute demonstrates minimal differentiation in core capabilities across providers (core meaning the chip itself, put inside of a DC, with a bare-bones software orchestration system)

What would I propose? We need neutrality. We need a system that injects transparency and liquidity into the process. In sum, it’s time for a market that lets buyers access compute when and where they need it, while enabling providers to monetize unused capacity efficiently.

Final Part 4 lands later this week, or take a look at the full piece here: https://compute.exchange/manifesto/.