Game On: Opening the Fight for AI Chip Supremacy

AI is dangerously concentrated.?

While we see new and improved models from startups and in the open-source space, only a handful of players provide the engines that drive much of the AI we use on the front end.?

In the cloud market, just three companies control most of the artificial intelligence hardware—as discussed in depth in our PTP Report focused on the cloud market’s concentration. In that space, we see regulatory pressures working to end practices like egress fees, though the heavy demands for new, powerful data centers make it unlikely that competitors will break through in a significant way anytime soon.?

Even with big tech monopolies in the national spotlight—final arguments are being made in the US vs Google now, and American regulators suing Apple, Amazon, and Meta over monopolistic practices—AI’s concentration continues to grow.

Still, of all of these, no aspect of AI, and maybe tech as a whole, is as narrowly controlled as AI hardware, and in particular, chips.?

With over 70% of the market share, Nvidia— whose chips (and software CUDA) are involved in most everything AI, from self-driving cars to social media—relies on TSMC, the Taiwan semiconductor maker, for most of their manufacturing. And with AI solutions today built specific to the chips that run them, it’s almost impossible for developers to swap between alternatives.

2024’s seen little change in these AI chip market trends: despite having their worst day on Wall Street in four years, Nvidia roared right back at the end of April, adding another $290 billion in value in the company’s best week since May, 2023.?

Nvidia, by all accounts, makes a great product, and has skillfully cornered their market, but in this week's The PTP Report, we look at what this means for the industry, and the efforts of Meta, Google, AMD, Intel, Microsoft, Amazon, and others to open the door for alternatives, desperate to inject more competition into the field.

Nvidia On Fire?

There are only four companies in the world worth more than $2 trillion: Apple, Microsoft, Saudi Aramco, and now, Nvidia. And for most of its life, Nvidia was known mostly for 3D graphics in gaming.?

In 2019, Nvidia's market value hovered around $100 billion (vs Intel’s $200 billion), with its GPU (graphics processing unit) enabling impressive 3D visuals in the surging world of gaming (which itself hit $188 billion in revenue in 2023).?

They made a failed bid to break into mobile chips, but it was their software, the CUDA programming language released in 2006, that enabled Nvidia graphics cards to be more widely used.?

The surprise was that, with CUDA, their GPUs could outperform CPUs at their own game, in areas like multitasking, machine learning, and even crypto mining.?

This enabled Nvidia to bypass chips rival Intel, who also makes GPUs, and which, in 2019 was worth twice Nvidia. Today, Nvidia’s chips are used in gaming, 3D design, data centers (as for cloud computing), and self-driving tech in cars. They were among the first to successfully tackle the challenges in AI chip design

Nvidia is a part of AI inference use, but it’s in AI training where they truly excel. In 2023 Nvidia’s data center revenue soared 400% above the prior year, as the AI race took off.?

We all use Nvidia products on a daily basis, even if we don’t know it (and even aside from 3D graphics), in things like social media applications (TikTok, Instagram, X/Twitter, Pinterest, Slack, and more), streaming services like Netflix, office tools, and self-driving car features.

Their GPU (graphics processing unit) has been the preferred technology for AI, though new variations are coming, under names like NPU, TPU, DPU, SPU, and more.?

TSMC?

Even for those aware of Nvidia, fewer consumers may recognize the name of their core manufacturer: Taiwan Semiconductor Manufacturing Company (TSMC).

TSMC is the world’s biggest contract chip maker, making chips for Nvidia on their designs, as well as providing chips used in smartphones (including the iPhone) and more. But despite a lower demand for many of their other products, their value raised 36% this year alone on the strength of AI chip demand.

TSMC’s initial caution on the continued strength of AI demand limited the availability of AI chips by nearly a year at one point for Nvidia’s popular H100, but now the company is investing heavily to eliminate potential bottlenecks (the delay is now down to four months, as reported by The Wall Street Journal ).?

But with almost all of the production needed for these Nvidia chips happening in Taiwan, who’s sovereignty neighboring China disputes, the US government is working to influence semiconductor industry trends, driving for far greater capacity in the United States.?

This includes Samsung investments in Texas, and a multibillion dollar CHIPS act grant for Intel. (While TSMC does work with CHIPS-funded Amkor Tech in Arizona, these are for Apple chips, not the technology needed for Nvidia’s AI chips.)

The Challengers?

Nvidia’s success stems from several factors, including: seizing on the future of AI chip development from early on, building a large community of AI developers around their products (more than 4 million developers use the CUDA platform for AI), continual innovation in machine learning hardware, forging strong partnerships with other big tech companies, and investing in AI startups.?

Their quality is lauded, and they’ve not shied away from competing with their own customers, who have no choice but to use them anyway.?

But pricing is increasingly a problem. With their newest chips running as much as $40,000 per chip, competitors are faced with a choice: develop better competition or spend an increasing amount of their resources on Nvidia products.

Gil Luria, analyst at DA Davidson Companies tells Vox that Microsoft: “went from spending less than 10 percent of their capital expenditure on Nvidia to spending nearly 40 percent. That’s not sustainable.”

The open-source UXL Foundation, powered by a coalition of rivals including Qualcomm, Google, Samsung, and Intel, are working on a software alternative to CUDA that can work with chips from varying manufacturers. Their goal is to improve the portability of AI technology overall, with plans to bring leading cloud providers Amazon and Microsoft into the mix.

Google's director and chief technologist of high-performance computing, Bill Magro, told Reuters : “It's about specifically—in the context of machine learning frameworks—how do we create an open ecosystem, and promote productivity and choice in hardware."?

Amazon’s $4 billion deal with AI company Anthropic also requires it to use chips Amazon has designed, such as the AWS Trainium, and Google has invested billions in its own chips, which reportedly cost them a fraction of the price, though at varying levels of performance.

And additional options are being funded, with an additional $4 billion being directed into 93 separate efforts to uncouple AI development from the specific chip used.

Concerns over Nvidia’s control of the industry extend beyond their clients and competitors, too, with the EU launching an investigation into the AI chip market late last year, and the FTC blocking Nvidia’s attempt to buy Arm Limited.

[For more on the emerging chips alternatives and an overview of AI developments, check out our bi-monthly Emerging AI Roundups for January and February , and March and April ]

Conclusion?

Nvidia’s story (as its product) is a great one, but the concentration of an essential technology among such few companies is certain to be problematic. And even with the billions being invested in this fight for AI chip supremacy, it’s unlikely Nvidia will be unseated any time soon.?

History indicates that such dominance can’t continue forever, and greater competition will offer more efficiency in AI chips, better pricing, more redundancy and protection in the supply chain, more innovation through greater portability between systems, and increased potential for smaller startups to get underway.?

References?

Strongest U.S. Challenge to Big Tech’s Power Nears Climax in Google Trial, New York Times

How Nvidia beat everyone else in the AI race, Vox

3 takeaways on the state of the global games market (hint: it’s growing), LA Times

EU Begins Early-Stage Probe Into AI Chip Market Abuses that Nvidia Dominates, Bloomberg

How Nvidia Built a Competitive Moat Around A.I. Chips, New York Times

- Doug McCord

(Staff Writer)

Check out other articles from PTP on AI ?

Get the latest updates on recruiting trends, job market, and IT, and expert advice on hiring and job seeking at The PTP Report ?