Microsoft rumoured to be working with AMD on AI chip 'Athena' to reduce dependence on Nvidia

Author: Yuwei Liu, Associate Principal Analyst, Electronic Engineering Album

Microsoft is working with AMD to develop its own artificial intelligence processor, Athena, in which it is providing AMD with financial support and hundreds of employees, according to sources familiar with the matter. But Microsoft spokesman Frank Shaw denied that AMD is a participant in the Athena project: "AMD is a great partner, but not involved in Athena."

As ChatGPT fires out of the ring, the demand for big AI models and applications of all kinds is starting to explode. And in order to support machine learning training for these applications, a lot of AI processors are needed, an area that is currently the domain of NVIDIA's graphics processors (GPUs).

Microsoft is working with AMD to develop its own AI processor, Athena, in which it is providing AMD with financial support and hundreds of employees, people familiar with the matter said. This is part of Microsoft's multi-pronged strategy to secure the supply of AI processor chips as an alternative to Nvidia's solutions.

Microsoft denies AMD's involvement in Project Athena

Microsoft is both a top provider of cloud computing services and an enabler of AI applications. They have now injected tens of billions of dollars into ChatGP parent company OpenAI and integrated it into the Bing search engine and Microsoft Edge web browser, and Microsoft has said it wants to add the feature to all its software products.

The in-house AI chip is a key part of Microsoft's efforts to strengthen its related business, and according to The Information, the initial generation of the chip will be manufactured on TSMC's 5nm process and will be used internally and by OpenAI starting as early as next year for education on Large Language Models (also known as LLM) and any data acquired during and after LLM training.

While there is previous precedent for Microsoft working with AMD and Qualcomm to develop custom Arm architecture chips for its Surface Laptop and Surface Pro X devices, Microsoft spokesman Frank Shaw denied that AMD is a participant in the Athena initiative: "AMD is a great partner, however not involved in Athena."

Developing its own chips does not mean it can break away from Nvidia

However, this does not signal a parting of the ways with Nvidia, and Microsoft intends to continue the close collaboration between the two companies. Nvidia chips are still essential for training and running LLM, and are the chip supplier of choice for many providers of generative AI tools, including Amazon AWS and Google Cloud, and Tesla CEO Elon Musk has stockpiled thousands of Nvidia processors for his own AI business.

The oversupply of NVIDIA GPUs in recent times also highlights the urgent supply shortage faced by Microsoft and other companies conducting AI business.

Because of Microsoft's relationship with OpenAI, and its own new AI services, such as a new version of Bing that chats and AI-enhanced Office tools, require more computing power than was originally expected when the chips were ordered and data centres built. Microsoft is also updating older products, such as GitHub's code generation tools. All of these AI programs run in the Microsoft Azure cloud, requiring Nvidia's expensive and powerful processors.

But this is also an area of strategic focus for AMD. We are very excited about our opportunities in AI, which is our highest strategic priority," said AMD CEO ZF Su on the earnings call on Tuesday (May 2). We are in the very early stages of the AI computing era, and AI is growing in popularity and at a faster rate than any other technology in recent history."

Suzy also highlighted that AMD will have the opportunity to make some of its custom chips for large customers for their AI data centres.

Microsoft's involvement in the chip industry is gradually deepening. Over the past few years, under former Intel executive Rani Borkar, Microsoft is building a chip division that now employs nearly 1,000 people. And in addition to developing chips for servers and Surface computers, back in 2019 the team began secretly working on an AI chip codenamed "Athena" internally, which now has more than 300 people and over $2 billion invested in it.

The sources cited by he Information say that Microsoft has made a small prototype of the chip available for use internally and by OpenAI employees to test performance on the latest big language models such as GPT-4. But even if the project progresses well on schedule, the first version will only be a starting point. It takes years to design and produce a powerful chip, and NVIDIA has a big lead.

In addition to the chips themselves, NVIDIA also offers a full suite of supporting hardware and software around these chips, including chips, programming languages, networking equipment and servers, so that customers can quickly build their own capabilities.

NV GPUs are expensive and in short supply, the main reason why the big players are developing their own

Dylan Patel, an analyst at SemiAnalysis, said it could cost around $100 million a year to develop a chip similar to Athena, and ChatGPT costs around $700,000 a day to operate, with most of the cost coming from expensive server processors. It is estimated that OpenAI would need more than 30,000 of Nvidia's A100 GPUs to commercialise ChatGPT, with Nvidia's latest H100 GPUs currently selling for more than $40,000 on eBay. If the Athena chip is as competitive as Nvidia's offering, the cost per chip could be reduced by a third.

NVIDIA's latest H100 NVL GPU will be available later this year and is ideally suited for large-scale deployments of large LLMs such as ChatGPT. the new H100 NVL with 94GB of memory and Transformer Engine acceleration offers up to 12 times the inference performance on GPT-3 compared to the previous generation A100 at data centre scale.

For ChatGPT's $700,000 per day operating costing methodology, a specific report released by SemiAnalysis on February 9 this year shows that in 2022 Google's revenue in the search business will be $162.45 billion, with an average revenue per query of 1.61 cents and an operating margin of 34.15% for Google's Services business unit, which works out to a cost per Google query of The cost of each Google query is 1.06 cents. Using the 175 billion parameter GPT-3 Big Language model used by OpenAI as a calculation, the daily operating cost of OpenAI is $694,444,000, with 13 million active users per day, each user responds 15 times, so the estimated cost per query for ChatGPT is 0.36 cents.?

It is also worth noting that the world has suffered from Nvidia for a long time, with several companies already developing or launching products to benchmark their chips, and tech giants such as Amazon, Google and Facebook are now making their own chips for AI.