Profit Dollars per GPU Dollar

“AWS’ AI business is a multibillion-dollar revenue run rate business that continues to grow at a triple-digit year-over-year percentage and is growing more than 3x faster at this stage of its evolution as AWS itself grew, and we felt like AWS grew pretty quickly.”

“Our AI business is on track to surpass an annual revenue run rate of $10 billion next quarter, which will make it the fastest business in our history to reach this milestone.”

Those quotes from Amazon & Microsoft last week underscore the dramatic transformation in cloud growth rates.

Across the 3 major clouds, the growth rates have increased between 27% and 58% from their nadir about a year ago. But the businesses are 60% bigger today than they were the last time they touched those growth rates.

Plus the operating margins of these companies is massive at around 40% for the top two. GCP’s is the lowest, but accelerating rapidly. It was 3.1% last year.

Microsoft & others have said their growth is limited by GPUs which will continue until late next year. Amazon & Google are developing their own chips :

“As customers approach higher scale in their implementations, they realize quickly that AI can get costly. It’s why we’ve invested in our own custom silicon in Trainium for training and Inferentia for inference. The second version of Trainium, Trainium2, is starting to ramp up in the next few weeks and will be very compelling for customers on price performance.”

And internally, the impacts are real. Google said 25% of new code written is AI generated. AWS quantified it further :

“The team has added all sorts of capabilities in the last few months, but the very practical use case recently shared where Q Transform saved Amazon’s teams $260 million and 4,500 developer years in migrating over 30,000 applications to new versions of the Java JDK.”

All of these advances are expensive:

“We expect to spend approximately $75 billion in CapEx in 2024. The majority of the spend is to support the growing need for technology infrastructure.”

In total, these hyperscalers invested about $52b last quarter in data centers & GPUs.

But the chips are now valuable for longer than they were (again from AWS). “We made the change in 2024 to extend the useful life of our servers. This added about 200 basis points of margin year-over-year.”

The most important metric for these businesses will be profit dollars per GPU dollar cost.

Which chip design will produce the best profits : Google’s TPUs, Amazon’s Inferentia/Tranium, or Microsoft’s Maia and Cobalt?

It’s hard to calculate exactly this figure because the public data isn’t granular enough to compare across the three. But over time we should be able to infer major differences.