NVIDIA GTC 2024 Highlights & Insights

Date:?March 18 to 21, 2024 Location:?San Jose, CA

KEY TAKEAWAYS:

• GTC is no longer a gaming conference. Welcome to GTC 2024, the AI infrastructure show!
• Blackwell is about “GPU as a Platform” more than it is about the GPU itself.
• Generative AI has graduated from LLMs to Mixture of Experts bringing about a new architecture for AI supercomputing.
• The next phase in NVIDIA’s evolution as a company is autonomous cyber-physical systems platform.?
• Despite the excitement and fanfare, generative AI is in its early phase and has much to prove.
• Security for generative AI applications and systems is lacking and depends largely on isolation and compartmentalization. Is that scalable??

NOTICE:

The full version of the report which includes the neXt Curve analysis section is available on the neXt Curve research portal at www.next-curve.com/insights/.

While we make our research available to the public, we will be taking measures in the future to minimize our contributions and public exposure to full reports and articles here on LinkedIn.

We would appreciate your support of neXt Curve research by engaging directly with our content channels such as our media center and YouTube (https://bit.ly/3Hq53ps) and Buzzsprout (https://bit.ly/43mr2Hm) podcasts. You can discover all of neXt Curve's content at www.next-curve.com.

As independent analysts, our insights are our livelihood. Support us so that we can continue to serve you.

EVENT SUMMARY:

After five years as a virtual event, NVIDIA’s GPU Technology Conference was live for its 2024 showing. What a difference five years has made. The last time Jensen Huang , CEO of NVIDIA, hit the stage at the company’s marquee developer conference which last drew gaming developers, bitcoin miners, and CGI designers taking in the latest in NVIDIA GPUs and CUDA.

Since then, we have cycled through several hyped technology trends including crypto, metaverse and Web3, autonomous vehicles (A.K.A. the search for SAE 4+), and machine learning (A.K.A. “AI”). In each of these hype cycles, NVIDIA has somehow found itself at the center of it all and also at the tail end of the disappointments. During an industry analyst Q&A session, Jensen responded to an analyst’s question about the future of crypto and the company’s role by simply stating, “Not very much.” Next question.

Now, NVIDIA finds itself in the frontier of the latest hype which took off last year, generative AI or GenAI. As Jensen put it, this frontier is a lonely place as his company dominates the space that is GenAI supercomputing for the moment.

We landed in San Jose on the first official day of the event which took place at the San Jose Convention Center in downtown San Jose. Unfortunately, this year of all years, we didn’t make the exclusive but limited roster of individuals and firms who were part of the analyst program. On the other hand, that allowed us to get some stellar seats at the SAP Center for Jensen’s keynote?(link)?which most of the analysts had to watch remotely or up in the nosebleed section.

Needless to say, the venue was packed, and the 2-hour presentation was as theatrical and flashy as ever. It has always helped that NVIDIA’s technology has been behind a lot of the visual flashiness in their presentations. It has served them well. It certainly served ?Jensen well as he took an audience of around 16,000 on a dense and fantastic journey outlining his vision for accelerated computing and introducing Blackwell, the star of the show.?

Blackwell is NVIDIA’s new GenAI supercomputing GPU architecture which is basically two GPU dies stitched together using likely a TSMC 2.5D (though rumored Samsung) chip-to-chip interconnect that provides 10 TB/sec of bandwidth (link). As Jensen put it, AI needs big GPUs. Blackwell is a very big GPU.

Blackwell is an interesting counterpoint to what we are seeing with Cerebras and their third generation Wafer-Scale Engine (WSE-3) AI processor (link) with 4 trillion transistors versus Blackwell’s 208 billion. Indeed, bigger GPUs result in a dramatically streamlined and performant system generation over generation for NVIDIA, but there was much more that factored into the order of magnitude densification and scaling of NVIDIA’s new AI HPC systems.

While GenAI supercomputing sucked a lot of oxygen out of the conference, the other aspects of NVIDIA’s GPU empire were present from automotive, robotics to XR. neXt Curve had an expansive agenda for GTC 2024 to cover most of it which included:

• Developments in trustworthy and safe generative AI frameworks and tools
• NVIDIA’s AI HPC systems and AI supercomputing?
• Automotive update and SDV enablement
• Omniverse and the current state of The Metaverse
• AI-native RAN NVIDIA style
• Edge AI and GPUs at the edge
• AI Foundry/Factory and new developments
• NVIDIA’s AI PC vision and approach

GTC 2024 Highlights

The Jensen keynote aside, which took up the better part of the first day of the conference, our three days at GTC came with a number of planned and unplanned highlights.?

At an analyst dinner hosted by Chetan Kapoor , Director of EC2 Product Management at AWS, I was fortunate enough to have vacated the head of the table to have Bill Dally , SVP and Chief Scientist at NVIDIA, accidentally join our table. He was apparently at the wrong analyst dinner, but happened to be early and was unanimously welcomed to stay once we realized who he was.?

Shruti Koparkar , GTM Leader of AI/ML Acceleration at AWS, and I ended up having a personal conversations with Bill about the innovations that went into Blackwell and the new line of H200, B100, and B200-based GenAI supercomputing systems.?In particular, we had a back-of-the-napkin lesson from Bill on NVIDIA’s 2nd generation Transformer Engine which supports dynamic precision tuning down to FP4 without losing accuracy for inference workloads. Bill cited it as one of the key innovations that contributed significantly to the performance improvements of the new Blackwell-based systems announced at GTC 2024.

On Day 2, we had the opportunity to take part in the industry analyst Q&A session with Jensen Huang attended by some of the leading independent analysts and research firms covering the semiconductor industry. I found Jensen’s views on NVIDIA’s opportunities in the telecommunications industry intriguing as the company had announced the AI-RAN Alliance with ten other founding partners at MWC 2024 (link). Jensen went surprisingly long and passionately on the topic.?

NVIDIA’s foray into the RAN is a long coming as it first announced EGX for telco five years ago at MWC Los Angeles. Jensen and team are clearly banking on influencing 6G and the future of telco infrastructure that is proposed to be “AI-native”.?The company announced the launch of its 6G Research Cloud (link) to provide researchers with the compute and simulation resources and services to invent 6G on top of NVIDIA’s technology stack.

Later in the evening, I had the chance to chat with Praveen Vaidyanathan , VP & GM of Compute Products Group at Micron Technology to talk about the significant role of high bandwidth memory on the densification and scaling of NVIDIA’s new GenAI supercomputing platforms and the company’s latest HBM3e products (link).

Praveen and I were later joined by Micron’s VP of Communications, Pete Lancia , to talk about Micron’s fab build out in the U.S. and the implications on the balance of sourcing of an essential element of massive GenAI computing.?

Our Day 3 of GTC 2024 was largely dedicated to the exhibition hall with access limited due to doors opening at noon. Many attendees thought was too late. I agree.

While waiting for the doors to open, I had a chat with Linda Yao , COO and Head of Strategy of Lenovo’s Solutions & Services Group (SSG) about the group’s AI services under the banner of “Lenovo AI for all” that the company announced at their Tech Day 2023 event in October of last year. Their strategy and offerings seem to have evolved quite a bit with a welcome emphasis on security as the first tenet of their approach in delivering safe and responsible AI solutions to their customers.?

According?to Linda, Lenovo’s AI services will provide customers with advisory support from discovery, through design/build, to ongoing maintenance of AI (not just GenAI) solutions. They will do this by channeling the company’s?Pocket-to-Cloud footprint as well as their edge AI and AI supercomputing expertise (link).?

Our final briefing of the day was with AWS, a key NVIDIA cloud infrastructure partner. We spent time with the AWS team to get the hyperscaler’s perspective on the generative AI opportunity and how the cloud leader sees its role in the future of GenAI computing given their many coopetition plays in “AI” silicon, models, and tooling.

For AWS, it boils down to customer choice. If the customer wants NVIDIA’s stack, that’s what they will get, especially if that customer is NVIDIA. Back in November of last year at re:Invent, AWS and NVIDIA announced Project Cieba to build the largest GPU-based AI supercomputer for NVIDIA’s internal AI development work (link).?

Martin Yip , Head of EC2 Product Marketing and Betsy Chernoff , Principal of GenAI Product Marketing at AWS met with a small group of analysts to share the news that AWS would host a 20,736 chip GB200-based AI supercomputer capable of 414 exaflops (link).

On the XR and Metaverse front, Apple stole the headlines with NVIDIA’s announcement that Omniverse will stream and integrate with the Vision Pro thus lending credence to the enterprise and industrial play for the headset. However, the more interesting XR news from our perspective came from NVIDIA’s research and Lenovo.?

Starting with Lenovo, it was good to see the XR team led by Vishal Shah and the workstation team led by Rob Herman fusing their industrial XR and edge?computing chops together into what they call an end-to-end reference architecture for VR collaboration.

The solution is centered around a ThinkStation PX Spatial Computing Appliance (workstation) that is able to support up to four ThinkReality VRX headsets in same scene engagement for a wide range of industrial and enterprise use cases such as collaborative design, service & support, and operations & management.

Seems like it could be pretty cool for gaming too.???

NVIDIA’s research also showcased AI-assisted volumetric video generation (sorry, not GenAI based) on device. This beautiful video was captured on a laptop fitted with a stereoscopic camera. The video is then processed in NVIDIA’s Maxine using LP3D (link) and a EG3D GAN.?

The synthetically generated volumetric content can then be viewed on a smartphone or any other device that supports 3D displays enabled by companies such as Leia and Orbbec who were present at GTC 2024.?

We were particularly excited as these technologies and developments validate the tech horizons study we conducted for Ofcom over five years ago. These technologies have great potential over the next five years to bringing about synthetic volumetric content for immersive reality applications, content, and communications to the consumer and enterprise mainstream.

Needless to say, GTC 2024 was a special event with so much to unpack. This summary is just a fraction of what neXt Curve picked up in our three days at NVIDIA’s premiere developers conference.?

If you are a vendor or enterprise looking for advisory services and deeper access to neXt Curve research insights and consulting services, contact us by direct messaging Leonard Lee here on LinkedIn.??

KEY ANNOUNCEMENTS:

NVIDIA

• PRESS RELEASE “NVIDIA Blackwell Platform Arrives to Power a New Era of Computing” (link)
• PRESS RELEASE “NVIDIA Launches Blackwell-Powered DGX SuperPOD for Generative AI Supercomputing at Trillion-Parameter Scale”, March 18, 2024 (link)
• PRESS RELEASE “NVIDIA Launches Generative AI Microservices for Developers to Create and Deploy Generative AI Copilots Across NVIDIA CUDA GPU Installed Base“, March 18, 2024 (link)
• PRESS RELEASE “NVIDIA Announces Omniverse Cloud APIs to Power Wave of Industrial Digital Twin Software Tools“, March 18, 2024 (link)
• PRESS RELEASE “NVIDIA DRIVE Powers Next Generation of Transportation — From Cars and Trucks to Robotaxis and Autonomous Delivery Vehicles“, March 18, 2024 (link)
• PRESS RELEASE “NVIDIA Announces Project GR00T Foundation Model for Humanoid Robots and Major Isaac Robotics Platform Update“, March 18, 2024 (link)
• PRESS RELEASE “NVIDIA Unveils 6G Research Cloud Platform to Advance Wireless Communications With AI“, March 18, 2024 (link)

AWS

• PRESS RELEASE “AWS and NVIDIA Extend Collaboration to Advance Generative AI Innovation”, March 18, 2024 (link)

Lenovo

• PRESS RELEASE “Smarter AI for All: Lenovo Unveils Hybrid AI Solutions that Deliver the Power of Tailored Generative AI to Every Enterprise and Cloud in Collaboration with NVIDIA”, March 18, 2024 (link)
• PRESS RELEASE “Accelerating the Opportunities of Generative AI with Lenovo AI Workstations and NVIDIA Accelerated Computing”, March 18, 2024 (link)

Micron Technologies

• PRESS RELEASE “Micron Commences Volume Production of Industry-Leading HBM3E Solution to Accelerate the Growth of AI”, February 26, 2024 (link)

Dell Technologies

• PRESS RELEASE “Dell Offers Complete NVIDIA-Powered AI Factory Solutions to Help Global Enterprises Accelerate AI Adoption” (link)

neXt Curve ANALYSIS:

NOTE: This section is available only on the neXt Curve research portal at www.next-curve.com/insights/.

• TAKEAWAY 1?–?GTC is no longer a gaming conference. Welcome to GTC 2024, the AI infrastructure show!
• TAKEAWAY 2?–?Blackwell is about “GPU as a Platform” more than it is about the GPU itself.
• TAKEAWAY 3?–?Generative AI?has graduated from LLMs to Mixture-of-Experts (MoE) bringing about a new architecture for AI supercomputing.
• TAKEAWAY 4 –?NVIDIA’s claim that they are the first AI PC company lends to the confusion.
• TAKEAWAY 5?–?The next phase in NVIDIA’s evolution as a company is autonomous?cyber-physical?systems platform.?
• TAKEAWAY 6?– Security for generative AI applications and systems is lacking and depends largely on isolation and compartmentalization.
• TAKEAWAY 7?–?Despite the excitement and fanfare, generative AI is in its early phase and has much to prove.??

RELATED MEDIA & PRESS RELEASES

• NVIDIA GTC 2024 Event site (link)
• LinkedIn: neXt Curve’s GTC 2024 Research Agenda (link)
• LinkedIn: NVIDIA GTC 2024 Day 1 Take (link)
• LinkedIn: NVIDIA GTC 2024 Day 2 Take (link)
• LinkedIn: NVIDIA GTC 2024 Day 3 Take (link)
• neXt Curve reThink Podcast: Recap of NVIDIA GTC 2024 (link)
• neXt Curve reThink Podcast: Recap of NVIDIA GTC 2024 (audio) (link)
• IoT Coffee Talk: GPU’s Are Eating the World? (link)

COMPANIES ENGAGED:?

Thanks to the many companies that engaged with neXt Curve at GTC 2024.

• 英伟达
• 联想
• Amazon Web Services (AWS)
• 美光科技
• ZEDEDA
• Google Cloud
• Dell Technologies
• Supermicro
• Leia Inc.
• Cloudflare
• Digital Realty
• CoreWeave
• Mavenir
• Zetaris
• Samsung Semiconductor
• SK hynix
• Magic Leap
• Edge Impulse
• Vapor IO
• KT
• PureWeb
• Astera Labs
• Scan AI
• QCT
• Perplexity
• 惠普企业服务
• 华硕
• Apptronik

Subscribe to the neXt Curve Insights monthly newsletter to be notified when the next newsletter is published. Go to?www.next-curve.com?to be added to our mailing list. You will also be notified when we publish new research notes and media content.?

Follow us on?LinkedIn,?Twitter, and?YouTube.

This material may not be copied, reproduced, or modified in whole or in part for any purpose except with express written permission or license from an authorized representative of?neXt Curve. In addition to such written permission or license to copy, reproduce, or modify this document in whole or part, an acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced.

??2024 neXt Curve. All Rights Reserved