Hot Chips for AI: Day 3 Wrap-up

The third and final day of Hot Chips is very much like the second day, with presentations on AI, networking, and high-performance processors. Day 3 also features IEEE TCMM Awards, but I will not be covering those in this review because the highlight in my mind was a presentation by AMD President Victor Peng who retired after decades of experience at AMD and Xilinx.

Mr. Peng discussed the road to AI pervasiveness as a series of waves or “supercycles” associated with the adoption of AI by different market segments. The first segment is the cloud, which is where most of the focus is today for training, especially around generative AI. The second wave is endpoints of user devices through digital assistants, digital agents, and applications like games. The third way is the world of machines. Mr. Peng refers to it by its traditional term “embedded,” but it has more recently been referred to as Industrial IoT or IIoT. Mr. Peng points out that there may be lulls between some of the waves, but the adoption of AI will continue until we reach a point of pervasiveness throughout society.

Challenges remain around security, privacy, safety, power consumption, and the value proposition for AI. While the latter may take years and new business models to address, the other issues are being addressed through advanced software, particularly quantization, and advanced hardware techniques like heterogeneous processing of AI workloads and advanced multi-chip packaging. All views are shared by TIRIAS Research.

AMD also had two other presentations on the Versal AI Edge Series Gen 2 for vision and automotive applications, and its Zen 5 series CPU architecture that will be featured across the company’s Ryzen AI PC SoCs and Epyc server processors. Other presentations featured:

-????????? Stanford University’s Onyx programmable accelerator for sparse tensor algebra

-????????? Meta’s new generation MTIA inference accelerator

-????????? Tesla’s Transport Protocol over Ethernet (TTPoE) for the Dojo supercomputer

-????????? Enfabrica’s ACF-S SuperNIC for high-performance computing

-????????? Intel’s co-packaged optical interconnect

-????????? Cerebras’ wafer-scale AI processor architecture

-????????? The XiangShan open-source high-performance RISC-V processors sponsored by several Chinese institutions

-????????? Ampere’s AmpereOne Arm-compatible CPU architecture for AI & cloud native workloads

-????????? Microsoft’s first-generation Maia AI accelerator

-????????? Preferred Network’s second-generation MN-core processor architecture

?

I should note that Hot Chips presentations are a combination of research efforts and solutions that are or will soon be available. Many of these technologies, like co-packed optics, have been under development for decades. While everyone agrees that there is a growing need for optical interconnects for everything from chip-to-chip interconnects to full networks, challenges in cost and manufacturability remain. But with three companies presenting on the topic, it’s clear that there is industry momentum to overcome these challenges.

Key Takeaways

As usual, Hot Chips featured a wealth of information on processor architectures, networking solutions, and related systems technologies. While AI was the central theme of this year’s event, an underlying message is the need to develop solutions at the system level, even future chips. This requires more upfront analysis and simulation to develop “balanced” solutions. This includes the processors, accelerators, networking, memory, and infrastructure, especially in terms of data centers.

Another underlying theme at the conference was the importance of the EDA tools used to design chips and systems. These tools are rapidly integrating AI while new agents are being developed to use them in place of traditional human designers. To that point, it was a bit surprising that we did not see a greater presence from the major players—Cadence, Siemens, and Synopsys. Only Synopsys had a presentation at the event. None of the solutions being presented would be possible without these EDA tools.

TIRIAS Research has always maintained that the is no single solution for AI because not two AI workloads are the same. As a result, we continue to see innovation in AI at all levels of the value chain. China, for one, will play a key role in the development of the open-source RISC-V architecture as an alternative to the traditional processing architectures due to recent politics and limits on technology transfer. And as Mr. Peng indicated, we are still in the early days of moving toward AI pervasiveness. The race to achieve a dominant market position is far from over.

A video recap of the event featuring Kevin Krewell and myself is also available on EETimes.com at https://www.eetimes.com/1417264-2/.