The GPU Report - Your AI GPU News of the Week

9 September 2024

Nvidia’s Blackwell GPU & Nvidia Hopper architecture impress in latest MLPerf Inference results

Nvidia’s new Blackwell GPU platform impressed with up to 4 times greater performance than the H100 on Llama 2 70B, and the H200 GPU also performed well across every data centre category.? See results from this latest round of benchmark testing (MLPerf Inference: Datacenter Benchmark Suite Results).

Cloud GPU rental services cheaper in China than in the US

Smaller cloud providers in China are offering Nvidia GPU rental services cheaper than what they cost in the US.? The Financial Times in a recent article cited that there are four small cloud providers in China offering configurations of eight A100 chips for $6/hr, where in the US it costs $10/hr.? The larger cloud providers in China are more comparable to Amazon Web Services which charges $15 to $32/hr the article also cited.

It’s estimated that there are approximately 100,000 H100 chips currently in China, and that smugglers are finding ways to fill demand from China’s AI ecosystem in the face of sanctions placed on China by Washington that forbid Nvidia and other chip makers to sell into China directly. ? ?

AI infrastructure demand not slowing down

Elon Musk’s xAI has brought online its AI training super computer Colossus in Tennessee powered by 100,000 H100 GPUs, making it the largest cluster today, and has plans to expand to 200,000 GPUs (including 50,000 H200 GPUs) over the next few months.

Mark Zuckerberg announced Meta’s plans to have it’s own AI training cluster in place powered by 350,000 H100 GPUs by years end.

xAI and Meta are clearly on offence, with a vertical integration strategy that recognises the importance of the data centre as a means to get out ahead of any competition.

OpenAI is going to develop it’s own AI chips alongside Apple with TSMCs new Angstrom A16 process as it looks to reduce reliance on external technology providers.? It has secured production allocation from TSMC with the A16 node to enter mass production in 2026.? OpenAI may also collaborate with Broadcom and Marvel as part of this partnership with TSMC to develop custom ASIC (application specific integrated circuit) chips.

Nvidia and other investors back Applied Digital with $160 million in funding.? Much of which will be used to buy more Nvidia GPUs.? Applied Digital operates data centers and recently started a cloud business targeted at AI workloads fuelled by Nvidia GPUs.

Nvidia is also investing in Sakana AI a Japanese based AI research company as part of a $100 million investment round led by Khosla Ventures and Lux Capital.? This supports the argument that there isn’t likely other be one global AGI, but rather a number regional AIs that coexist.

Results

Nvidia loses $406 billion in market cap this past week. Yawn…

Nvidia released its financial results for Q2 fiscal 2025 on August 28.

I'll GPU all next week. Buckle up!

Article by Rhys Selby