AI PC Buyers Beware: New Standards Could Render Your Device Obsolete

You’ve heard of AI PCs, right? If you recently purchased one of the latest AI PCs announced at CES 2024, I’ve got bad news for you--Microsoft says they don't qualify as 'Copilot+ PCs' (its branding for 'AI PCs').

?

At its May 20 event, Microsoft made clear that Copilot+ PCs must have a "processor or System on Chip (SoC)" on its approved list, and have an NPU capable of 40+ Trillion Operations per Second (TOPS).

?

So, just six months into 2024, anyone who's got into the AI PC game early has come away disappointed to learn that their new toys aren't 'AI PCs'.?

?

How early AI PCs ran into obsolescence in mere months

?

Many of the computers introduced at CES 2024 run on Intel Meteor Lake chips, which the company revealed in December 2023. The biggest architectural shift for Intel in four decades, these chips landed in time for the biggest PC brands to unleash their new computers that are marketed to accelerate AI workflows. They also achieve about 11.5 TOPS, which is quite a bit shy of Microsoft's 40+ TOPS minimum requirement.

?

Meanwhile, if you're following Intel's roadmap, they're not sitting still. Intel's working furiously to put the finishing touches to Lunar Lake, a chip capable of 48 TOPS. Computers running these chips are due in the third quarter of 2024.

?

AMD, however, has its Ryzen AI chips--capable of 50 TOPS performance--ready in laptops announced at Computex 2024. So, PC makers like Asus released similar-sounding laptops, like the Zephyrus ROG G16, not once but twice this year--the Intel Meteor Lake version at CES 2024 (remember, 11.5 TOPS) and the AMD Ryzen AI version in June, which Asus says is capable of "up to 81 TOPS". The former has only been on sale in Singapore in recent months, while the latter will arrive in the coming weeks. If you were a regular consumer and not someone who constantly follows industry news, you wouldn't have known this.

Every major PC brand is replicating this strategy, promoting and selling 'AI PCs' in January that would be obsolete by mid-year. It's not a head scratcher--companies are selling early adopters stopgap hardware. Gotta meet those quarterly earnings after all! Intel, AMD, Microsoft, and PC makers are all complicit as every single one of them promoted AI PCs at the start of 2024.

?If it's a matter of TOPS performance, any computer running a powerful GPU, like Nvidia's RTX 40-series card, launched in late 2022, is pretty much able to crunch the math for on-device processing. In fact, Nvidia has called BS on Microsoft's NPU requirement, as its GPUs are capable of achieving anything between 100 to 1,300 TOPS (see article: Nvidia criticizes AI PCs, says Microsoft's 45 TOPS requirement is only good enough for 'basic' AI tasks).

Of course, these GPUs need a lot of power—utilizing between 80W to 150W compared to, say, 20W on Apple's M4-series chip. Since Apple introduced its own high-performance and uber power-efficient silicon, PC vendors and chipmakers have been trying to come up with their own version of this new type of computing device. After several tries, it appears that Microsoft has done it through its partnership with Qualcomm.

The new Surface computers feature the latter’s Arm-based chips that not only deliver longer battery life but boast, what do you know, 45 TOPS performance. Arm-based computers have traditionally seen less successful adoption due to incompatibility with most x86-based applications, but the Redmond company seems to have crested this obstacle with its Prism emulator.

Everyone’s bringing their BDE (big dick energy) to the table, not that it matters, because Microsoft's requirements are specifically targeted at its list of 'approved' NPUs. That means all computers running Nvidia's latest RTX GPUs don't qualify as Copilot+-ready. Otherwise, how's Microsoft and its fleet of vendors going to sell more stuff to you?

You can’t just pay for a service like Midjourney or ChatGPT and call it a day. If you can process all that intelligence and enjoy last-mile performance in a nanosecond while running a device that lasts all day, why won't you?

Everyone's gotta make money off the AI money train.

Meanwhile, in Apple universe…

PC makers aren’t the only ones guilty of drawing seemingly arbitrary and artificial lines to get you to buy new hardware. Mac users are in a bit of a conundrum too. In March, Apple announced its M3-powered MacBook Air, an ‘AI computer’, only to launch iPad Pros with the new M4 chip in May. In case you're curious, the M4 only manages 38 TOPS. To be clear, the M3 was introduced just last October. See the parallels with the PC/Windows world?

At WWDC 2024, Apple made clear that the entire M-series line of chips will be ready for all the AI innovations being announced. More deft at messaging, Apple never mentioned TOPS once at the annual event. Still, Apple doesn't get a pass for not talking feats and speeds, because while many of the new AI-related features will run on M-series chips (iPads and Macs), these chips are only available in more expensive hardware.

Does TOPS really matter?

At the end of the day, does TOPS really matter in real-world AI performance? This Embedded article has a robust discussion on this very topic and the TL;DR of it is: not really. TOPS numbers don't quite equate to real-world workloads. It’s more useful to consider the kinds of compute power required by different types of AI. GenAI, for instance, is more processor-intensive, while machine learning can run on even low-cost processors. So technically, whether you rely on a GPU or NPU depends on the task at hand.

It's obvious that this year, the conversation is not just about AI infrastructure but the worthiness of one's ability to run AI on-device. The problem is chipmakers aren't making it easier for consumers to understand what to buy or if they should buy anything at all.

As brands try to spin about how AI empowers human beings, it’s become clearer that this technology that we’ve been talking about for decades is going to widen the digital divide, not just between those who use AI and those who don't, but also those who pay for better AI tools and those who don’t or can’t. AI could have been the great equalizer. Instead, PC makers (including Apple) are making it harder for us to understand why we need the new hardware to run AI tasks. And the transition period is filled with hardware that leave buyers with a bad taste in their mouths.

As a consumer, ask yourself if you really need an AI PC for now. The kinds of workloads today don't quite demand new hardware just yet. I mean, my older computers ran various AI tasks just fine last year. Still does. To everyone who just bought a 'new' AI PC in recent weeks and months, you may be irritated, but you'll be fine too. You may not qualify for the latest Microsoft Windows Copilot+ features, but do a couple of seconds matter?

For the rest of you, if you haven’t fallen for the AI computer trap this year, perhaps you could fall for it next year. I’m willing to bet it'll top any TOPS you'll find in computers this year.

?

# # #