AI Chip Vendors’ Strengths and Positioning by System Type

In our previous research report, we highlighted the 5 year outlook for the percentage of AI workloads to be run by the type of system they would run on (see “AI Workloads by System Type - A 5 Year View”, J.Gold Associates, Technology Insights, June 19, 2024, also available on our newsletter on LinkedIn at https://www.dhirubhai.net/pulse/percentage-ai-workloads-system-type-5-year-projection-jack-gold-bkz2e?). In that report we indicated that the current dominance of AI workloads running on large data centers or hyperscaler cloud instances would shift dramatically, primarily to the edge. In this report we will look at the types of processors and the vendors that will be powering the various AI system types, and how their strengths in particular segments might affect their positions in the market, including the strengths and weaknesses of the incumbent processor suppliers.

Figure 1 shows our segmentation of the various chip makers’ market strength by AI system type. Below we’ll further analyze each system type and the vendors that impact each segment.

Figure 1: AI Chip Vendors’ Market Strength by System Type

Cloud/Hyperscaler

Nvidia remains a clear leader in the high end of the market powering a large number of hyperscaler cloud instances for AI training. However, the hyperscalers are also building their own AI chips (e.g., AWS Trainium, Google Trillium TPU, Microsoft Maia) that are customized for their unique infrastructure, as well as for cost leverage. While they may not be as powerful as the Nvidia AI superchips (although that may not be the case for all applications), the hyperscalers can price services running on their processors more affordability by not having to pay the high price for purchasing the Nvidia chips at up to $40K each. There is also a significant class of AI solutions that will run happily on a high performance CPU with built in accelerator (e.g., Intel Xeon, AMD EPYC) or competitive GPUs (e.g., AMD MI class, Intel Arc). We expect the hyperscaler market to offer a wide array of processors to meet the growing diversity of AI training needs, as well as some higher end inference based workloads. While in the short term, Nvidia’s dominance in this segment is likely protected, longer term (2+ years) we expect a significant dilution of its market share.

Data Center

This is a broad market, encompassing the largest scale systems (e.g., tens of thousands of Nvidia high end processors), to more modest systems operating on enhanced standard CPUs that have added AI internal acceleration. This is similar to what processors have done in the past adding internal graphics acceleration capability. Traditional providers in this space are Intel and AMD, with a smaller contingent of ARM-based solutions (e.g., Ampere). We expect the move towards more traditional data center servers running AI workloads to accelerate as we move to inference based workloads, as well as fine tuning and RAG optimizing of existing models. It’s also likely that as newer, more efficient modeling methods are developed, they will increasingly be run on traditional servers, both from a cost/performance advantage perspective as well as for greater compute availability. This will benefit the traditional players who have a well established data center business (e.g., Intel, AMD). While very large scale processors from Nvidia will be present in some data centers, it will be far more limited than in the higher end hyperscaler environment, although Nvidia’s lower end GPUs will have a role to play in this space, albeit with competition from AMD and Intel GPUs and specialized chips (e.g., Gaudi).

Edge

We expect the vast majority of AI workloads to migrate to edge based systems over the next 2-3 years. Edge encompasses a wide range of systems and processing capabilities – from small internal processing in sensor arrays, robotics and similar machinery, to more advanced capabilities needed in autonomous vehicles, medical diagnostics, smart infrastructure, etc.? One critical factor for success in this space is the ability to provide compatibility across a continuum of scale and scope of processing, enabling companies to target the workload to the needed compute requirements. Open source platforms and development environments will have a key role to play in this space, and processor vendors that don’t adequately support open platforms (as opposed to proprietary solutions like CUDA) will be at a major disadvantage. This means open and compatible ecosystems like Arm and x86 from AMD and Intel will have significant advantages as they create compatibility from small to large computing needs. They allow up scaling or down scaling as the processing requires as well as ease of porting solutions and reuse. This easy access and wide variety will be critical to the success of Edge and the vendors who supply processing solutions. We expect Arm based solutions to dominate the low end, while more robust solutions will be focused on the traditional dominance of x86 based compute (e.g., Intel, AMD). And while Nvidia is attempting to downscale its AI solutions with products focused on this space at the mid to high end of the edge market, we expect it to achieve a limited presence in this vast solutions area as it struggles against the incumbents.

IoT

There is overlap between the IoT space and edge computing. Many IoT solutions have been thought of as small processors embedded inside cameras, or powering localized industrial equipment. But in fact IoT is expanding greatly to include higher end autonomous systems like cars, transportation equipment, large machines, smart buildings and cities, etc. Much like edge, there is a need for an open ecosystem to provide scalable solutions. Even though most processing is embedded into IoT, increasingly we’re seeing IoT connecting to Edge processing to compliment what’s embedded. The current IoT processing world is dominated by Arm processors, and with Arm’s inclusion of AI functionality into its available IP, we expect this domination to continue. That is advantageous for suppliers in this space like Qualcomm, Marvel, MediaTek, Broadcom, NXP, and a large range of others, as well as custom designed solutions.? But increasingly we are seeing a focus from players like Intel, AMD, Nvidia and others who rightly understand that the new AI powered needs will require more than the relatively simple processors currently employed. Further, as Arm licensees like Qualcomm, MediaTek, Marvel, etc. embed NPU capability into their devices, we expect to see an increase in the use of localized on-device AI workloads. We do expect Arm to remain the dominant architecture for this environment for the foreseeable future, although others like RISC-V will acquire a significant share, especially in the lower end, over the next few years, while others like x86 will also share in the growth of computing requirements in the more computationally enhanced solutions.

PC

We expect that within 2-3 years, 85+% of new PCs will include built-in AI capabilities. The move to add features by Microsoft in the OS and in its productivity apps, as well as the built-in advantages that the PC manufacturers can deploy uses AI to enhance security, privacy, management, performance improvements and user experience will propel AI to become a must-have. The incorporation of AI accelerators into the PC processors will make the enablement of AI simpler and more effective enhancing a variety of productivity apps that will appeal to enterprise users in particular and making AI accelerated PCs a must-have in the next replacement cycle. Just as when the GPU was first integrated into the base processor and obsoleted stand alone GPUs for most users, we expect the same trajectory for AI capabilities that are fully integrated into the processor. Intel and AMD dominate the current PC processor market. But Qualcomm has made a concerted effort to become a player in the PC space given its potentially better battery life and sleekness of products using the Snapdragon Elite chips. This is a high growth area for Qualcomm, but we don’t expect it to capture more than 10-15% of the PC market in the next 1-2 years, with most of that share coming in the consumer and SMB space. Still, given the size of the market, that’s a significant market share and revenue opportunity. Nvidia will still maintain its share of the discrete GPU market (as will AMD) for high performance needs, including in workstations enhanced with AI, but the general PC market will utilize an integrated SoC processor from industry leaders Intel and to a lesser extent AMD, with a minority share for Qualcomm and eventually other Arm licensees.

Mobile

As more AI applications make their way to mobile devices, the need for increased processing will accelerate the inclusion of AI enabled processors. With its overwhelming share of the current mobile market, there is no question that Arm architecture will remain the majority of processing options powering smartphones and tablets. While there are several viable mobile processor vendors, Qualcomm remains the leader by a large margin, although MediaTek in particular has lately offered some increased competition in the mid to high range of the market and is moving away from it traditional lower end solutions. We do expect the continuation of custom mobile processor solutions, primarily Apple created, but that is still based on the Arm architecture. The Chinese government has made this area a priority for expansion, but so far has not made major inroads into this market, at least not at the mid to high end where the majority of AI workloads run and profits are attractive. We do expect AI workloads to migrate to lower end devices over the next couple of years, but much of the compelling AI features will remain in the higher end and premium devices, an area currently dominated by Qualcomm. We expect Qualcomm to maintain its leadership in this area for the next several years, while competitor MediaTek gains more traction.

New Processor Entrants

There has been a significant number of new players entering the AI processor market in the past 1-2 years, with more expected to arrive in the next 1-2 years. But since they are relatively new players with a lack of established market presence and proven capability, it’s difficult to position them effectively as they stake out their particular niches. We do expect a few of the new entrants to ultimately be successful, while many of the others will either fade away or be acquired in the next 2-3 years. Of note in the new entrants is Cerebras, with its wafer scale technology that is positioned at the high end of the market and challenging the dominance of Nvidia. It recently signed a deal with Dell that will give it distribution beyond its own internal sales efforts and importantly add Dell’s endorsement of its technology. This is a company worth watching if it can scale out its products and live up to its promise of faster and especially significantly lower power massive scale AI training systems. Other newcomers that have received significant funding include SambaNova, Groq and Etched, but are still early in their journey. There are a number of new entrants internationally as well, with China placing particular emphasis on this space. Indeed, Chinese companies, armed with government sponsored initiatives, are a key area to watch over the next couple of years.

Bottom Line: The processor market for AI enabled systems is vast and highly varied. No current vendor will be able to capture it all, and specialized vendors will be coming on line in the next 1-2 years to further diversify the available solutions. We expect the winning vendors to expand beyond the few now in a leadership role. We also expect that the shifting needs and high growth for AI enabled systems will offer major opportunities to a variety of vendors that will concentrate on specific AI system types and processing areas. One thing is clear. Virtually every type of system in the future will be AI enabled and that is ultimately good for all vendors and users.

Copyright 2024? J.Gold Associates, LLC.

J.Gold Associates provides advisory services, syndicated research, strategic consulting and in context analysis to help its clients make important technology choices and to enable improved product deployment decisions and go to market strategies. Join our mailing list to receive updates on our research and overviews of our reports. Email us at:? info (at) jgoldassociates (dot) com