An EPYC Moment in AI ??

Throwback Thursday: Let’s take you back to this infographic we created for our weekly newsletter, Belamy, a couple of weeks ago. (Subscribe to it here , if you haven’t already!)

The RGB spectrum in the infographic—Red (AMD), Green (NVIDIA), and Blue (Intel)—is finally converging in high-performance computing and AI, creating a whole new spectrum of possibilities. Once fierce rivals, these tech giants are now collaborating to push the boundaries of AI workloads. AMD’s EPYC CPUs are emerging as critical components, powering NVIDIA’s GPUs and delivering significant improvements in both training and inference performance. This synergy optimises large-scale AI tasks, with AMD’s high-core processors boosting NVIDIA’s H100 GPUs, while Intel and AMD’s partnership breathes new life into the x86 ecosystem.?

The result: An ‘EPYC Moment’ in AI, where collaboration among these industry leaders is unlocking new levels of performance, scalability, and innovation, setting the stage for the future of computing.

Making NVIDIA GPUs EPYC

Even as competition soars between the leading chip manufacturers, AMD and NVIDIA, a new partnership is quietly emerging between the two rivals.?

Case in point: AI and the computing world.?

Few realise that AMD’s EPYC CPUs are critical in powering NVIDIA’s GPUs for large-scale AI workloads. As AI models grow in complexity, the demand for both GPU and CPU performance sees an uptick, with AMD CPUs showing significant potential to enhance NVIDIA’s GPUs.??

“We’ve shown a 20% improvement in training and 15% improvement in inference when connecting EPYC CPUs to NVIDIA’s H100 GPUs,” said Ravi Kuppuswamy , senior vice president & general manager at AMD. At the Advancing AI 2024 event, Kuppuswamy cited the Llama 3.1 inference model with 8 H100 GPUs, where the CPU provided significant value in large-scale GPU clusters.?

He added that this is a collaboration between AMD and NVIDIA, where they have identified the best EPYC CPUs to optimise the configuration between CPU and GPU.?

AIM noted this interesting synergy as the chip manufacturers partnered to integrate AMD’s EPYC CPUs into NVIDIA’s HGX and MGX GPU systems. It optimises AI and data centre performance by leveraging AMD’s high-core processors alongside NVIDIA’s parallel computing GPUs, while promoting open standards for greater flexibility and scalability.

“We don’t want to force choices on our customers… We will continue to push open standards and interoperate with vendors across the industry,” said Madhu Rangarajan , corporate VP at AMD, EPYC Products. He emphasised AMD’s open approach, supporting diverse customer needs.

AMD’s 5th-generation EPYC processors , with NVIDIA’s HGX and MGX GPU clusters, are likely to optimise the performance of data centres and enterprise tasks to the next level. “Even the fiercest of rivals can come together when it benefits their customers,” said AMD, underscoring the importance of this partnership in advancing AI and high-performance computing.

Previously, AMD claimed its EPYC processors deliver twice the performance of NVIDIA’s Grace Hopper Superchip across multiple data centre workloads, showcasing significant advantages in general-purpose computing and energy efficiency. “Our EPYC processors provide a lower total cost of ownership due to their performance, energy efficiency, and extensive x86-64 software compatibility.”?

It highlighted how NVIDIA’s Arm-based CPUs lag in non-AI workloads compared to AMD’s Zen 4 EPYC processors. “This is good news, as NVIDIA is currently using the vastly inferior Intel Xeon in its systems,” a Reddit user posted, suggesting that AMD should leverage its strengths to capture a larger share of the HPC CPU.?

AMD x Intel In an unlikely turn of events, chip rivals AMD and Intel have also come together to form the x86 Ecosystem Advisory Group. The group will focus on expanding the x86 ecosystem by simplifying software development and improving platform interoperability.?

Intel and AMD will provide developers with tools to create scalable solutions and identify architectural enhancements that can meet the demands of modern computing, such as AI workloads, custom chiplets, and advancements in 3D packaging.        

“We are at the cusp of one of the most significant shifts in the x86 architecture and ecosystem in decades – with new levels of customisation, compatibility and scalability needed to meet current and future customer needs,” said Intel chief Pat Gelsinger.?

“We proudly stand together with AMD and the founding members of this advisory group, as we ignite the future of compute, and we deeply appreciate the support of so many industry leaders.”

“Establishing the x86 Ecosystem Advisory Group will ensure that the x86 architecture continues evolving as the compute platform of choice for developers and customers,” said AMD chief Lisa Su.?

AMD and Intel believe that x86 is still relevant in the era of AI. In an exclusive revelation at the Advancing AI 2024 event, an AMD executive told AIM that the company was the first to bring neural processors to the x86 environment.?        

“In the x86 world, we introduced those first neural processors in 2023 with a product we call ‘Phoenix Point ’ delivering 10 TOPS of neural processing performance and enabling several workloads, such as Windows Studio effects and many other third-party ISVs that were supporting those early chatbots and assistants on the device,” shared the executive.

‘Phoenix Point’ is the world’s first fully accelerated AI inference engine on x86 processor silicon with the new XDNA architecture.

DisARM-ing Qualcomm?

This move by Intel to partner with AMD comes at a time when the company is fighting for survival.?

According to a recent report, Qualcomm is likely to wait until after the US presidential election in November before deciding whether to pursue an offer to buy Intel. With Intel’s acquisition, Qualcomm is possibly trying to strengthen its hold in the PC market and add Intel’s Lunar Lake with x86 architecture to its portfolio.?

Intel appears to be not ready to move away from x86 architecture anytime soon, while its competitors are shifting to ARM architecture. To advance the x86 advisory group, AMD and Intel garnered support from major PC manufacturers, including HP, Microsoft, Dell, and Lenovo.?

During the unveiling of Lunar Lake , Gelsinger ended the ARM vs x86 debate, saying, “The final nail in the coffin of this discussion is that some claim x86 can’t win on power efficiency. Lunar Lake busts this myth. This radical new SoC architecture and design delivers unprecedented power efficiency—up to 40% lower power consumption than Meteor Lake, which was already very good.”

According to Statista, in the third quarter of 2024, Intel processors accounted for 63 percent of x86 computer processor tests, while AMD processors represented 33 percent.?

When focusing solely on laptop CPUs, Intel is the clear winner, capturing 71 percent of laptop CPU benchmark results in the second quarter of 2024. AMD processors made up 21 percent of the laptop CPUs tested.        

Meanwhile, Qualcomm is challenging Intel with its ARM-based processors. The company recently launched Snapdragon X Elite, its latest ARM-based processor designed for Windows laptops, to compete with Apple’s M-series and Intel’s x86 processors.

Microsoft recently introduced a new category called ‘Copilot + PC,’ which can run generative AI models directly on the device without relying on cloud support. Interestingly, during the announcement Microsoft appeared to favour Qualcomm processors over Intel and AMD for its AI capabilities.

Qualcomm’s Snapdragon X Elite and X Plus are set to launch in new Windows Surface PCs, along with offerings from Dell, Lenovo, HP, Asus, Acer, and other major OEMs in the coming months. These processors feature NPUs capable of 45 TOPS (tera operations per second), slightly exceeding Microsoft’s minimum requirement.

On the other hand, Intel claims that Lunar Lake processors are 30% faster than AMD chips and 68% faster than Qualcomm’s offerings, although these claims have yet to be validated through real-world testing. Intel’s Lunar Lake features 40 NPU TOPS and over 60 GPU TOPS, resulting in more than 100 platform TOPS.        

“It does seem like Lunar Lake is the kiss of death for Snapdragon X Elite. Similar battery life, but with broad app compatibility of x86, and an actually usable GPU. However, long term though, I hope this isn’t the death of Windows-on-ARM. It’s always good to have more silicon vendors, and hence more competition,” a user posted on Reddit.

Similarly, AMD recently launched the new Ryzen AI PRO 300 Series, based on the AMD ‘Zen 5’ architecture, which is AMD’s implementation of the x86-64 instruction set. The processor offers over 50 NPU TOPS of AI processing power, exceeding Microsoft’s Copilot + AI PC requirements.

On the other hand, much like Qualcomm, Apple, which previously used x86, has also moved away to ARM architecture. Last year, Apple signed a new deal with ARM for chip technology extending beyond 2040. Apple is already using ARM’s V9 architecture for its latest M4 MacBook chips, which it announced in May.

Enjoy the full story here .?

‘Indian IT Services Companies Are Not Product Companies’

Indian IT giants like TCS, Infosys, Wipro, and HCLTech are shying away from fully embracing generative AI. Barely investing in R&D, they are busy focusing on maintaining existing projects rather than driving innovation.?

“Indian IT services companies are not product companies,” said Mohandas Pai, former CFO of Infosys, adding large AI models require significant resources, which India currently lacks. Despite having the funds, companies like Infosys and TCS remain focused on vertical solutions rather than broad AI innovations like ChatGPT. Pai emphasised that startups should drive R&D, though investment remains a hurdle. Read on.?

AI Bytes?

• Mistral AI launched its new Ministral 3B and 8B models , designed for edge computing with enhanced capabilities in knowledge reasoning and function-calling, offering up to 128k context length and efficient inference.
• NVIDIA chief Jensen Huang is expected to join Reliance Industries chairman Mukesh Ambani at the NVIDIA AI Summit in Mumbai to discuss India’s AI future.?
• Nearly 84% of consumers in the fashion and beauty categories made purchases based on AI-driven recommendations and influencer promotions during the 2024 festive season.
• Zerodha has announced a $1 million annual FLOSS Fund to support free/libre and open source software projects worldwide, offering grants between $10,000 and $100,000 to address the financial sustainability challenges faced by open-source developers.
• Lenovo unveiled the ThinkSystem N1380 Neptune, a next-gen water-cooling system built to support NVIDIA’s Blackwell platform and AI applications, enabling 100KW+ server racks without specialised air conditioning.?
• Google.org recently announced a $15 million funding initiative to train the US government workforce in AI.?
• BharatGen, India’s government-backed AI initiative, has launched e-vikrAI , a vision language model designed for Indic e-commerce to auto-generate product titles, descriptions, and pricing from images.?
• SurveyMonkey has established a GCC in Bengaluru to tap into India’s diverse talent pool while expanding its global presence and boosting development efficiency across time zones.