Intel Vision 2024 Highlights & Insights

Date:?April 8 to 9, 2024 Location:?Phoenix, AZ

KEY TAKEAWAYS:

• Is accelerated computing about GPU??Gaudi 3 says otherwise.
• The GenAI supercomputing race is driving Intel to become a systems company.
• Intel is leaning into “Enterprise AI” with RAG (Retrieval Augmentation Generation).
• Lunar Lake is going to heat up the AI PC race in H2 2024 with an aggregate 100 TOPS of AI compute.
• The “brownfield” could be an advantage for Intel as the new AI hype settles down across enterprise cloud to the edge.
• Tiber is a good start toward providing an identity for the software side of Intel.
• GenAI security is all about confidential computing. That’s not enough.?

NOTICE:

The full version of the report which includes the neXt Curve analysis section is available on the neXt Curve research portal at www.next-curve.com/insights/.

While we make our research available to the public, we will be taking measures in the future to minimize our contributions and public exposure to full reports and articles here on LinkedIn.

We would appreciate your support of neXt Curve research by engaging directly with our content channels such as our media center and YouTube (https://bit.ly/3Hq53ps) and Buzzsprout (https://bit.ly/43mr2Hm) podcasts. You can discover all of neXt Curve's content at www.next-curve.com.

As independent analysts, our insights are our livelihood. Support us so that we can continue to serve you.

EVENT SUMMARY:

For analyst, Intel Vision is a busy event. The analyst relations team does a tremendous job of making sure that almost every minute of our waking hours are planned and packed full of 1-on-1s, roundtables, and briefings with Intel executives and industry luminaries. It is Intel Vision, which the company describes as their “premier event for business and technology executive leaders to come together and learn about the latest industry trends and solutions in advancements from client, to edge, to data center and cloud.”?

This year, Intel is bringing to Vision their vision for “bringing AI everywhere”, which is not surprising given all the energy and excitement channeled into this increasingly dubious thing what is “AI”. More often than not, folks are talking about generative AI or “GenAI”.

neXt Curve’s agenda for Intel Vision didn’t preclude AI in its many shapes and forms by any means. We just had broader interests which we brought to the event that include:

? AI PC momentum and emerging value cases

? Core Ultra at the edge and Edge AI

? Deep dive into Intel’s Edge Platform

? Gaudi and Intel’s future in Gen AI HPC

? Gen AI security, privacy & confidentiality

? DCAI and updates on Granite Rapids, Sierra Forest and beyond

? Telco and data center networking

Intel Vision – Day 1?

Day 1 of the event was largely NDA limiting what we could share. This is typical with Intel’s Vision events and isn’t surprising given that analysts have a full day packed with privileged roundtable discussions with Intel executives.?

Our day started early in the morning with an intense discussion with Jeni Barovian Panhorst , VP & GM of Data Center AI, Saurabh Kulkarni , VP of Data Center AI, and Reza Jazayeri , Program Director of AI Products Group (who stood in for Eitan Medina ) to unpack the to-be announced Gaudi 3 AI accelerator.

We asked a lot of questions and got some great insights from the DCAI executives who shared some impressive benchmarks versus NVIDIA’s H100. In simple terms, Intel claims that Gaudi 3 delivers an average speedup of 1.5x versus NVIDIA’s H100 outperforming its peer in most (not all) benchmarks of inference workloads for Llama-7B Llama-70B, and Falcon-180B models.

The most notable aspect of the new Gaudi 3 is that it is two compute dies integrated with 2.5D high bandwidth interconnect similar to the approach NVIDIA took with their recently announced Blackwell GPU to realize a “bigger GPU”.

Gaudi 3 is based on TSMC’s 5nm process in contrast to 7nm for Gaudi 2.?For?reference, NVIDIA’s Blackwell will be manufactured with TSMC’s 4NP process.?Gaudi 3 also comes with 128GB of high bandwidth memory (HBM) thanks to eight HBM3e chips bringing it closer to the memory capacity of the NVIDIA H200 placing it in the midrange of GenAI accelerators to come in the second half of the year.

You can find details about Gaudi 3?here?on the Intel site.

Next, was Christoph Schell ’s keynote which was heavily laced with AI, more specifically generative AI. One thing that we noticed throughout the course of the event, various Intel teams used AI to mean generative AI. We felt that Intel missed the opportunity to remind customers of?Intel’s rich non-GenAI and HPC legacy. Going all in with GenAI is the overriding marketing playbook at the moment.

We managed to take a break from a packed agenda of keynotes and analyst roundtable sessions to witness the once-in-twenty-years solar eclipse. Intel was kind enough to provide protective sunshades so that we could experience the eclipse without blinding ourselves. Thanks, Intel team!

The highlight of our afternoon was an analyst roundtable with?Intel CEO, Pat Gelsinger . He had great responses to questions I had about RAG and the role of the CPU in the era of accelerated computing. It seems that Pat is banking on RAG to make Enterprise AI happen. I continue to reserve broad concern about RAG frameworks given what I have observed in my research.?

I share some of my insights on this topic in the neXt Curve Analysis section of this report.

Another insight I drew from our time with Pat was the unlikelihood that organizations will migrate all of their data and?knowledge repositories to a new “GenAI First” platform and architecture. For this reason, the CPU may continue to secure a comfortable home in the brownfields of the accelerated computing era.?

The other highlight of the afternoon was a briefing that we had on security for AI, and AI for security” with Greg Lavender , CTO of Intel and Dhinesh Manoharan , VP of Security for AI. Greg and Dhinesh talked a great deal about the new Trust Domain Extensions (TDX) supported by their data center processors from Sapphire Rapids and beyond.?The duo also mentioned Intel’s intent to expand their confidential computing features and services to edge infrastructure for industries such as healthcare and automotive for critical workloads and data.

As we had seen during the course of the last few years, confidential computing is the prevailing focus of Intel and its peers. I feel the discussion about GenAI security has to be holistic with all players in the game contributing to address significant gaps that will challenge safe and responsible adoption of GenAI tools by organizations and consumers.?

Again,?I share some of my insights on?the?topic of GenAI security in the neXt Curve Analysis section of this report.?

Intel Vision – Day 2

Day 2 of Vision 2024 started very early in the morning barely giving the analysts time to digest and contemplate the implications of the NCAA Men’s Basketball championship blowout on the future of Intel in the era of accelerated computing. It was worth it because a small group of analysts had the privilege of having an exclusive briefing with Greg Lavender, CTO of Intel on Intel’s software technologies and strategy for tackling the next round of the AI opportunity that is GenAI.?

Bottomline, Greg and team are betting on abstraction by bridging silicon to AI development frameworks with open software technologies to effectively neuter CUDA especially across the edge. Both Pat and Greg seem pretty confident that the open approach will also work for GenAI HPC. I think that Intel has a lot to prove.

I share my insights in the neXt Curve Analysis section of this report.

Next up, Pat’s keynote. As is typically the case, there weren’t too many surprises. Obviously, the details and official announcement of Gaudi 3 is great media fodder. The announcement of Lunar Lake was expected but the aggressive launch timeframe of Q3 of 2024 was a bit of a surprise that is a testimony to the urgent tempo of execution by the Client Computing Group to lay claim to and fortify Intel’s prospects in the AI PC race.

The awkward topic of essentiality that emerged from Pat’s talk threaded through most of the GenAI technical keynotes and presentations was that of data types, in particular FP4. NVIDIA’s announcement at GTC 2024 that its upcoming Blackwell will support FP4 data types through their 2nd gen Transformer Engine will set a new bar for GenAI HPC system performance for both model training and inference.?Intel will have to match NVIDIA in this regard for GenAI HPC as well as the edge.

Another big topic at Intel Vision 2024 was what Pat and team called enterprise AI.?Justin Hotard, EVP of Intel’s DCAI group hosted a panel on taking GenAI to the enterprise with Edward Calvesbert of IBM , Charles Xie of Zilliz , Steven Huels of 红帽 , and?Rohit Tandon of 德勤 . The operative imperative for enterprise AI is operationalizing AI in particular GenAI.?

There was a great deal of talk about narrowing the application of GenAI for enterprise value and leveraging RAG architectures to make responses more enterprise relevant. This makes a lot of sense as enterprises?will?require utility specific to their business as well as?security controls to ensure confidentiality and privacy.

Charles Xie made an interesting comment that?quickly?grounded the panel discussion. AI is expensive. I would add that it is also difficult to realize value. Rohit Tandon and Edward Calvesbert alluded to the need to think through how GenAI will be operationalized in the organization.?However, the cost of testing and continuous validation of AI was not talked about. These are important missing items that I hope will become priority talking points going forward as Intel and its partners evolve enterprise AI toward a full lifecycle discussion.?

One of my favorite keynotes of the day featured Naveen Rao , VP of Generative AI at Databricks who gave a mesmerizing master class in generative AI that framed how we have arrived where we are today.

According to Naveen, over the last few years we have ridden a fast-accelerating curve of compute and economic scaling that continues to make massive GenAI HPC cheaper and technically able to push the frontier of model size, architectures, and capabilities. He?dubs?this new curve Mosaic’s Law that states that “the amount of learning per $ will quadruple every year”.?

Could Mosaic’s Law be the new Moore’s Law or subject to it. It is apparent that many of the breakthroughs in AI HPC scaling, and economics are due to system level innovations that are ultimately tied to the same?physics?that challenge Moore’s?Law?today.?

Regardless, thought-provoking stuff. I’m looking forward to revisiting Mosaic’s Law next year and the year after. ?

My final session for Intel Vision 2024 was an analyst roundtable discussion with Sachin Katti , SVP & GM of Intel’s NEX (Network & Edge) Group. The group is the tip of the spear for Intel’s vertical industry enterprise AI push. Sachin talked about how Intel is increasingly taking a systems mindset and approach with partners to accelerate the path to value of AI, not just generative AI.?

Sachin hinted at both soft and hard reference systems designs that incorporate software and platform services to help scale the deployment and management of and orchestration of AI workloads and data across edge infrastructures and fleets of endpoints. This is the essence of what Intel announced at MWC 2024 now renamed Intel’s Tiber Edge Platform.?

There was a ton of talk about chips and GenAI throughout Vison 2024. It was refreshing to hear mention of Intel’s IPU (Infrastructure Processing Unit). Sachin mentioned that an “AI-optimized” version of an IPU would be coming as well as new UEC-compliant Ethernet adapters to foster AI fabrics across data centers and the edge.?

Summary Intel Vision 2024 was an important set up for the company’s products across DCAI, CCG, and NEX portfolios at a time when NVIDIA is sucking all the oxygen out of the room and commanding the data center narrative while Qualcomm and Arm present challenges on the AI PC and IoT fronts respectively.

The AI (really GenAI) focus was a little overwhelming, but Intel ended up making a pretty decent case for its inference opportunity and positioning from data center to edge. The big question remaining is how the GenAI hype cycle will play out as Intel, its peers, and ecosystem push to make enterprise GenAI express sustainable value to enterprises and consumers.?

Either way, Intel is slated to make a big push in 2024 toward proliferating energy-efficient AI compute across the markets and installed bases it serves. That will be a boon for AI overall and the prospect of greater AI augmentation across industries.?

KEY ANNOUNCEMENTS:

• PRESS RELEASE: “Intel Unleashes Enterprise AI with Gaudi 3, AI Open Systems Strategy and New Customer Wins” (link)
• PRESS RELEASE: Intel Gaudi 3 (link)
• PRODUCT PAGE: Tiber Software Portfolio (link)
• PRODUCT PAGE: Tiber Edge Platform (link)

neXt Curve ANALYSIS:

NOTE: This section is available only on the neXt Curve research portal at www.next-curve.com/insights/.

• TAKEAWAY 1 –?Is accelerated computing about GPU??Gaudi 3 says otherwise. (link)
• TAKEAWAY 2 –?The GenAI supercomputing race is driving Intel to become a systems company. (link)
• TAKEAWAY 3 –?Intel is leaning into “Enterprise AI” with RAG (Retrieval Augmentation Generation). (link)
• TAKEAWAY 4 –?Lunar Lake is going to heat up the AI PC race in H2 2024 with an aggregate 100 TOPS of AI compute. (link)
• TAKEAWAY 5 –?The “brownfield” could be an advantage for Intel as the new AI hype settles down across enterprise cloud to the edge. (link)
• TAKEAWAY 6 –?Tiber is a good start toward providing an identity for the software side of Intel. (link)
• TAKEAWAY 7 –?GenAI security is all about confidential computing. That’s not enough. (link)

RELATED MEDIA & PRESS RELEASES

• Intel Vison 2024 Event site (link)
• LinkedIn: neXt Curve’s Day 1 at Intel Vision 2024 (link)
• LinkedIn: neXt Curve’s Day 2 at Intel Vision 2024 (link)?
• neXt Curve reThink Podcast - Intel Vision: Intel's AI Show (with Karl Freund ) (link)

COMPANIES ENGAGED:?

• 英特尔 , 联想 , Dell Technologies , IBM

Subscribe to the neXt Curve Insights monthly newsletter to be notified when the next newsletter is published. Go to?www.next-curve.com?to be added to our mailing list. You will also be notified when we publish new research notes and media content.?

Follow us on?LinkedIn,?Twitter, and?YouTube.

This material may not be copied, reproduced, or modified in whole or in part for any purpose except with express written permission or license from an authorized representative of?neXt Curve. In addition to such written permission or license to copy, reproduce, or modify this document in whole or part, an acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced.

??2024 neXt Curve. All Rights Reserved