AMD Delivers Generative AI Vision
AMD's new Instinct MI300X is a GPU-based accelerator specifically designed for generative AI workloads.

AMD Delivers Generative AI Vision

One of the most indisputable benefactors of the generative AI phenomenon has been the GPU, a chip that made its initial mark on the world as a graphics accelerator for gaming. As it turns out, GPUs have proven to be extremely adept at enabling and improving the process of training large foundation models and running AI inferencing workloads as well.

Up until now, the big winner in the generative AI GPU game has been Nvidia, thanks to a combination of strong hardware and a large installed base of CUDA software tools. At an event in San Francisco this week, however, AMD came out with both new GPU and CPU hardware and important new software partnerships and updates. Taken together, AMD believes these announcements will help it take a bigger chunk of a datacenter AI accelerator market it predicted will reach $150 billion dollars by 2027.

The new Instinct MI300X chip is what AMD referred to as a dedicated generative AI accelerator. Leveraging the same basic chiplet-based design as the previously announced Instinct MI300A (which the company also announced was now sampling), the MI300X replaces the 24 Zen4 CPU cores in the MI300A with additional CDNA3 GPU cores and High Bandwidth Memory (HBM). In fact, the new chip—which includes a total of 153 billion transistors—has 192 GB of HBM and offers 5.2 TB/second of memory bandwidth. These represent a 2.4x increase in memory amount and 1.6x improvement in throughput versus Nvidia’s current H100 accelerator. While those numbers are almost hard to fathom for most any other application, large language models (LLMs) run most efficiently in memory, so these should translate to solid real-world performance when the chip starts sampling in the third quarter of this year.

In addition to hardware, AMD also made several important announcements on the software side. First, the company detailed the latest iteration of its ROCm platform for AI software development. ROCm 5 consists of low-level libraries, compilers, development tools and a runtime that allows AI-related workloads to run natively on AMD’s Instinct line of GPU accelerators. It also sits as the base upon which AI development frameworks such as PyTorch, TensorFlow and ONNX operate. Speaking of which, one of the two big bits of software news from AMD’s event was a new relationship with the PyTorch Foundation. Starting with PyTorch 2.0, any AI models or applications built with PyTorch will run natively on AMD Instinct accelerators that have been upgraded to support ROCm 5.4.2.

This is a huge deal because a large number of AI models are being built with PyTorch and, until this deal was announced, most could only run on Nvidia’s GPUs. Now model and application developers, as well as major cloud computing providers, will have the flexibility to either use AMD Instinct accelerators directly or even swap out Nvidia accelerators with AMD ones.

The other big software announcement was with Hugging Face, which has quickly become the defacto location for open-source AI models. As part of the new partnership, Hugging Face will work with AMD to ensure that thousands of existing and all new open-source models posted on its site will be made compatible with Instinct accelerators. Longer term, the companies plan to also work on compatibility across other AMD processors, including Epyc and Ryzen CPUs, Radeon GPUs, Alveo DPUs and Versal FPGAs (or adaptive processors, as AMD calls them). Once again, this is extremely important and should help position AMD as a much more viable alternative to Nvidia in a number of AI datacenter environments.

On the datacenter CPU front, AMD also announced their new “Bergamo” and “Genoa X” versions of their fourth generation Epyc processors. They also hinted at yet another version called “Sienna” that they said would be announced later this year. Bergamo is optimized for cloud computing workloads and uses a new smaller Zen4c core and squeezes many more of these smaller cores onto the chip (up to 128). The refined architecture allows it to do things such as run more containers simultaneously, resulting in impressive benchmarks that partners including AWS, Azure and Meta were all happy to discuss in person as part of the presentation. Genoa X pairs the company’s 3D V-Cache technology that they first introduced with their 3rd generation “Milan” series with the 4th generation Genoa CPU design. It’s optimized for technical and high-performance computing (HPC) workloads that need access to more and faster on-die cache memories.

What’s interesting about all these CPU developments (as well as the variations on the Instinct MI300 accelerator side) is that they reflect AMD’s growing diversity of designs optimized for specific types of applications. The nice thing is they can all leverage a number of core AMD technologies, including their chiplet-based design and the Infinity Fabric interconnect technology that they created as part of their first chiplet efforts. It’s a great example of how forward-looking designs can have a very large and long-lasting impact on overall strategies.

One last bit of AI hardware that AMD unveiled at their event was the Instinct Platform. Like a conceptually similar offering from Nvidia, the Instinct Platform combines eight of AMD’s GPU-based accelerators (MI300Xs) into a single, compact hardware design.

To be clear, Nvidia still has an enormous lead on virtually anyone else when it comes to generative AI training and a strong position in inferencing as well. Once these new accelerators and software partnerships start to make their presence known, however, they should make a meaningful impact for AMD. As much as companies like what Nvidia has enabled for generative AI, the truth is, no one likes to have a market dominated by a single player. As a result, many companies will likely be eager to see AMD develop into a strong alternative in this space.

On the datacenter CPU front, AMD has clearly developed into a strong alternative to Intel, so it’s not hard to imagine the company starting to develop a similar profile in datacenter AI accelerators. It won’t be easy, but it’s certainly going to make things more interesting.

Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on LinkedIn at Bob O’Donnell or on Twitter @bobodtech.

要查看或添加评论,请登录

Bob O'Donnell的更多文章

  • Nvidia Positions Itself as AI Infrastructure Provider

    Nvidia Positions Itself as AI Infrastructure Provider

    One of the biggest challenges in analyzing a rapidly growing company like Nvidia is trying to make sense of all the…

  • Enterprise AI Will Go Nowhere Without Training

    Enterprise AI Will Go Nowhere Without Training

    One of the biggest questions currently facing the tech industry is how fast and how wide adoption of Generative…

    4 条评论
  • The Rapid Rise of On-Device AI

    The Rapid Rise of On-Device AI

    While everything related to generative AI (GenAI) seems to be evolving at breakneck speed, there’s one area that’s…

    4 条评论
  • Adobe Reimagines Generative Video with Latest Firefly

    Adobe Reimagines Generative Video with Latest Firefly

    One of the most impressive applications of Generative AI (GenAI) is the ability to create videos from nothing more than…

  • Samsung Cracks the AI Puzzle with Galaxy S25

    Samsung Cracks the AI Puzzle with Galaxy S25

    Finally. After years of half-filled promises and underwhelming realities, it looks like Samsung has finally succeeded…

    4 条评论
  • Nvidia Brings GenAI to the Physical World with Cosmos

    Nvidia Brings GenAI to the Physical World with Cosmos

    In what was undoubtedly the most eagerly anticipated, most closely watched and highly attended CES keynotes of all…

    6 条评论
  • Amazon Refreshes AI Story with New Chips, Models and Platform Tools

    Amazon Refreshes AI Story with New Chips, Models and Platform Tools

    One thing that’s become very clear when it comes to Generative AI (GenAI) is that we’re still in the early days of the…

    3 条评论
  • Microsoft Brings AI Agents to the Mainstream

    Microsoft Brings AI Agents to the Mainstream

    As exciting and impressive as the Generative AI (GenAI)-powered, prompt-driven, chatbot experience may be, the…

  • How Hybrid AI Will Change Everything

    How Hybrid AI Will Change Everything

    When it comes to Generative AI, the common thinking is that the critical tools necessary for businesses to build their…

    4 条评论
  • Cisco Expands AI Infrastructure Offerings

    Cisco Expands AI Infrastructure Offerings

    While most people think about Cisco as a company that can link infrastructure elements together in data centers and the…

    4 条评论

社区洞察

其他会员也浏览了