Hot Chips for AI: Day 1

Two interesting topics were chosen for this year’s Hot Chips Sunday tutorials - AI-assisted hardware design and the cooling of hot chips. The morning session focused on AI-assisted hardware design of chips. This is an interesting topic considering the industry that enabled the AI revolution has been relatively slow to leverage AI for future chip designs. As Synopsys highlighted, chip design is “a near-infinite problem.”? The design geometries continue to get smaller as the chips become more complex. A single die may have hundreds of billions of transistors, and a single chip may have trillions of transistors in a single package. The design complexity is increasing exponentially with each generation. Yet AI is not pervasive throughout the chip design process. Many of the tools and processes have been developed over decades, and there are still concerns about the reliability of AI, especially generative AI.?

While there is some debate on where and how AI should be used in the chip development cycle, there is agreement that chip development has reached a point where AI is both necessary and increasingly common through tools and agents that perform specific processes. In some cases, the AI agents even use the standard tools. As an example, Nvidia began using AI for design analysis in 2019 but only recently began leveraging AI for design assistance. One thing that the session presenters agreed upon is that we have just reached the first wave of AI-assisted solutions, but more data and development are required before we reach a point of more automated AI design solutions. As in other areas, AI is part of the solution, and it is far from being able to eliminate humans in the design process.

The afternoon session focused on the cooling of hot chips. From edge applications to the cloud, AI workloads and increasing design complexity are pushing the thermal boundaries of chips and systems. As a result, various techniques and advanced cooling systems need to be employed. Just as with chip design, the challenges of data centers continue to grow exponentially as the data sets, models, and use of AI continue to grow. Liquid cooling addresses several key data center issues - power, costs, and thermal management. According to Supermicro, liquid cooling can reduce server cooling power by up to 92% and the entire data center power consumption by up to 40%. The result is a savings of millions of dollars and the ability to increase the density and efficiency of data centers.?

But even with the liquid cooling solutions that are available today, there is room for future innovation in liquid cooling design, such as immersion cooling, where individual components or entire systems are submerged in a liquid dielectric. In addition, efforts are underway to reuse the heat captured in the cooling process for other applications like agriculture in colder climates.

Nvidia highlighted some of its research and development on future data center cooling solutions. One called OMNICOOL is aimed at a self-contained platform that can fit into a shipping container. While this is not a new concept, the design and efficiency are much higher than any solutions to date.?

For edge applications, Qualcomm discussed the thermal challenges faced by edge devices, while Frore Systems and Phononic discussed innovative chip-level cooling solutions. Frore Systems has developed an innovative solid-state cooling solution as an alternative to both passive and active cooling solutions for slim form factor devices like smartphones, tablets, and laptops. The AirJet Mini is essentially a solid-state heat pump that uses an ultra-thin vibrating membrane to create airflow. Phononic has developed a solid-state heat exchanger that uses both passive and active cooling. These products and innovations demonstrate the growing concern of thermal management and the requirement for more advanced solutions at the system level.

Please note that this is just a high-level recap of the first day of presentations at Hot Chips. There is much more detail available, and my descriptions and static images may not do justice to some of the topics or products. Please check out the presentations if you have access or contact the respective companies for more details.

Day two of Hot Chips will feature product and technology presentations on high-performance, specialized, and AI processors. Look for that recap here on LinkedIn. After the conclusion of Hot Chips, my colleague Kevin Krewell will join me in a video recap of Hot Chips that will be available on EETimes.com