WWDC24 Event Recap

Apple has been prodded and poked for not having an "AI strategy". Frankly, I don't think most of those folks who prod and poke rightly know what an AI strategy would look like other than selling massively expensive GPUs and supercomputing appliances to hyperscalers.

The reality is Apple has had an "AI strategy" for a while. The company has been pioneering consumer AI since they introduced Siri to the world back in 2011. Since then, Apple has driven AI features across their devices and services and was first to put a neural engine or NPU (Neural Processing Unit) into a smartphone, tablet, and even a smartwatch.

What could Apple possibly do to upstage what we have witnessed from Google, Microsoft, Nvidia, and Open AI? This is what makes WWDC 2024 curious and highly anticipated as expectations of an earth-shattering Cupertino "AI strategy" hits a fever pitch in the era of generative AI.

Since ChatGPT broke out onto the world stage last year, Apple has continually mentioned their long legacy in machine learning (ML) and intent to take a thoughtful approach to adopting generative AI. All of this seems to have fallen upon deaf ears that could only tune into "AI" not aware that Tim Cook's early mentions of transformers, which is the neural network architecture innovation that make generative AI models possible, have been deployed in some of the earliest GenAI applications to land on a smartphone such as Apple's AutoCorrect.

Did Apple revolutionize or reinvent AI? Did they finally catch up if they were ever behind?

Here's the neXt Curve take on Apple's WWDC announcements.

Key Announcements

It's obvious that Apple intended to save the "just one more thing" for last. For this reason, everything up to that point in the keynote seemed pedestrian bordering on trivial, but there were some cool nuggets in each platform category the team covered in the 1-hour and 45-minute presentation that often highlighted Apple's AI chops.

VisionOS 2

Apple kick things off with visionOS 2. As some of you might know, I have a Vision Pro (and haven't returned it) and use it daily. I personally looked forward to what Apple had lined up in the keynote for the next major version of visionOS.

Spatial Video and Spatial Photos are the primary media types that I consume on my Vision Pro. The ability to convert regular photos into Spatial Photos is something that will enhance the appeal of the Photo app. There are several third party applications available that convert regular photos into a 3D rendition, but I find their outputs less than stellar. Hopefully Apple has cracked the code of quality immersive conversion.

A year ago, I wrote about how the Apple would bring Spatial Video editing support to their Final Cut Pro (FCP) media editing software. Later this year, Apple announced that it would bring Spatial Video editing to Final Cut Pro. This should be a significant boon for consumer and prosumer creators who currently deal with tedious workarounds and manual workflows to stitch together Spatial Video in tools such as the popular DaVinci Resolve.

Apple also announced that it is partnering with Vimeo for immersive media hosting and distribution. Vimeo will launch a Vision Pro app for sharing and viewing Spatial Videos. This is a welcome development given the paucity of native Spatial Video content. Public content is nonexistent. No word on YouTube for Spatial Video support. To date, YouTube does not have a native visionOS app.

I also prognosticated a year ago that Apple would lean into live broadcast and media. Apple is doing exactly that announcing a partnership with Blackmagic Design combining the forces of Blackmagic 8K 180-degree cameras, DaVinci Resolve Studio, and Apple's Compressor software to bring studio-quality production technologies to Immersive Video.

Vision Pro owners will undoubtedly welcome immersive content that is on par with Apple's own productions such as "Prehistoric Planet". Most of the VR 180 and VR 360 content out there is of less than compelling quality for the Vision Pro.

visionOS 2 will bring SharePlay which allows Vision Pro users to experience photo sharing with fellow Vision Pro users. The feature uses what Apple calls Spatial Personas to teleport friends and family members into your space. This is about as close to "metaverse" as we have seen from Apple but a private flavor. Call it a “siloverse”.

As I have posited in the past, games are not the XR killer app. It is turning out that immersive content is. As we are witnessing with Spatial Personas and SharePlay, immersive communications delivering shared experiences in a Spatial Computing context is starting to take shape.

iOS 18

The iOS 18 presentation was a bit of a snoozer. Some of the new features that were announced seemed trivial. For example, the Home Screen and the added abilities to place app icons and widgets anywhere on the screen and tint icons were perfect fodder for the Android camp to brag and claim, "We have had that for years!"

The most notable feature announcements were related to the Photos app which turned out to be a reminder of Apple’s ML (Machine Learning) legacy and expertise.

We have already seen generative content features in Photos for a while. A great example is Memories which creates personalize collections of photos and videos set to music. Late last year, Apple introduced the Journal app which recommends Reflections tagged throughout your week that you might want to write a journal entry about.

iPadOS 18

The standout feature of the upcoming iPadOS is the new Calculator app. Initially, this announcement seemed like a bad joke. How can a calculator app be noteworthy?

iPad Calculator app will introduce a new feature called Math Notes. Math Notes intelligently solves handwritten math expression using the Apple Pencil Pro and is able to handle complex math expressions involving variables and functions.

Math Notes will automatically update the results of an equation based on a change in a value, notation, or the formula itself. Math Notes will also generate graphs that will parametrically update themselves.

Seemingly trivial, Math Notes stood out like the Open AI GPT-4o math problem demo. Unlike GPT-4o, Math Notes is a good old AI app that leverages a clever integration of several classical ML models and capabilities. Math Notes proves that you can build compelling "AI" apps that matter for users without GenAI.

MacOS Sequoia

With all the excitement of AI PC and Microsoft's Copilot+ PC, it seemed that the next MacOS version, Sequoia, needed to impress with substantial AI features. In short, the impression was subtle but compelling with a heavy dose of privacy and security through what appeared to be feature trimming (GenAI risk reduction).

Let’s start off with the stuff that the Google community will declare, “We had that!” but are still a boon for Mac users. Lenovo and Samsung introduced smartphone mirroring at least a year ago. Then there are the pioneer attempts such as Dell’s Mobile Connect of yesteryear.

While iPhone Mirroring might become a target for Android/Windows OEM bragging rights, did Apple users ever really need it given Continuity and the tight integration between MacOS and iOS? The answer to this question is why the Apples user experience continues to be the envy of the industry. Regardless, I'm sure Apple users won't complain about this nice-to-have but really cool enhancement to Continuity.

Apple took the opportunity to use their macOS presentation to introduce the new Password App that will come with Sequoia as well as iOS 18, iPadOS 18, and visionOS 2. This will be a welcome evolution of the KeyChain app which has helped users get a better handle on the management of security across their devices as well as the applications and websites that they visit and use.

Consumers need all the help they can get when it comes to security. Making it easier is a big deal. From the looks of it, the Password App will provide users with improved visibility and control of their security settings across connections and networks. In addition to passwords and credentials, the Password App will centralize the management of Wi-Fi passwords, verifications codes, and provide a dashboard for managing security alerts.

Safari will be getting an AI make over with Siri Highlights which uses ML to identify important things on a webpage such as directions, and "intelligently" present quick links to summaries on people, music, movies, and TV shows. Siri Highlights will also provide a summary of the page you are visiting.

The new Reader mode in Safari will also feature Summarize which likely uses an LLM to create a table of contents and a brief abstract of an article that you are contemplating to read. This feature could be useful in helping you determine if the article is worth reading provided the AI-generated summary does the composition justice.

The AI features of Sequoia and its platform peers seem modest up to this point in the keynote. Modest but cautious and practical.

Of course, there was one more thing.

Apple Intelligence

The big reveal of WWDC 2024 was Apple Intelligence. More than a product, it is an "AI framework" not unlike what we have seen from Microsoft Copilot and Google Duet. Yes, a revamped chatbot framework with a vast library of models and interfaces that surface intelligent features across Apple devices and experiences.

The big revelation of Apple Intelligence - Apple developed its own LLM (Large Language Model) and diffusion model to drive the many generative AI-augmented features highlighted throughout the keynote.

Craig Federighi, SVP of Software Engineering at Apple, described Apple Intelligence as “personal intelligence.” Apple Intelligence is about generative AI features that use your data on device to contextualize and theoretically "personalize" your user experiences across Apple devices. Ideally, this is done with privacy and security maintained.

Apple Intelligence boils down to three simple applications; Rewrite, Summarize and Image Playground. These applications are enabled by two foundation models that underlie Apple Intelligence.

• Rewrite provides compositional support that you might expect from a ChatGPT-like chatbot, offering tone-based draft recommendations for things like email replies, proofreading, and hopefully a better spellchecker. Sorry, no translation. English only upon launch in the Fall.
• Summarize does exactly that. It will summarize an email, article, or webpage.
• Image Playground generates an image in four "deepfake-resistant" styles based on a graphical or image input that can be further refined by the user with text prompts.

Apple asserts that these features will run locally on device based on their foundation LLM and diffusion models to keep things more private and trustworthy by keeping applications grounded on your data. Apple claims to do this with its Semantic Index which sounds like a vector and/or graph database that securely stores embeddings of your personal data/content to enable Apple Intelligence personalization. This mechanism is likely similar to Microsoft’s own Semantic Index for Copilot.

Devices with the A17 Pro processor (with a whopping 35 TOPS of NPU compute) and M Series processors from M1 to M4 will get Apple Intelligence when it launches in the Fall. Interestingly, the 11 TOPS of the M1 appears to be the minimum NPU compute requirement to run Apple Intelligence. This contrasts with the 40 TOP minimum threshold chipmakers and Windows OEMs need to meet to qualify for Microsoft's CoPilot+PC designation.

Another big revelation coming out of WWDC 2024 was the nature of OpenAI ’s ChatGPT “integration” with Apple Intelligence. Many media outlets had speculated prior to the WWDC event that Apple had to resort to using a third party model, namely OpenAI’s ChatGPT, because they were “behind in AI”.

Apple Intelligence is based on Apple’s own foundation models, NOT OpenAI’s models despite wide speculation to the contrary.

How is Apple Intelligence different from the likes of the widely known ChatGPT which OpenAI famously launched to the public last year? Apple dubs models such as GPT-4 that underlies ChatGPT as “world models” which are trained on much larger sets of content with the controversial ambition of becoming “AGI” or Artificial General Intelligence.

Apple has “integrated” ChatGPT at the UI level with privacy safeguards implemented to ensure that users have full awareness and consent when their requests are offloaded/transferred to ChatGPT.

Apple is clearly not OpenAI exclusive citing that it is open to “integrating” with other world model providers in the future. This agnostic approach is smart and allows Apple to better ease Apple Intelligence into markets like China and the EU where the concerns about world models and AI in general are growing along with regulatory scrutiny.

Apple also introduced Private Cloud Compute, which amounts to a hybrid AI cloud infrastructure. Apple described features of Private Cloud Compute that look a lot like confidential computing more commonly associated with cloud computing for banks, healthcare and other industries that require trusted environments for critical and highly private applications and transactions.

While the mechanism for when and how Apple Intelligence requests are orchestrated between device and cloud/server remains a mystery, Apple claims that server-side requests and processing do not store the user’s personal data. This suggests a serverless implementation of Apple Intelligence services delivered on Apple’s own data center software stack running on Apple Silicon-based data center infrastructure.

According to Apple, the policy for Apple Intelligence is on device first, Private Cloud Compute second if the request requires the assistance of a larger Apple Intelligence LLM, and finally, a recommendation to offload to a third party world model such as ChatGPT for requests that Apple Intelligence is not well suited to handle.

Curiously, it does not seem that Private Cloud Compute will support the Apple Intelligence Image Model. This makes sense as Apple is likely looking to minimize the much higher cost of providing vision, video, and multimodal GenAI services from their Apple Intelligence cloud.

It's also interesting to note that Apple has taken an approach with Apple Intelligence that alleviates Apple from having to pay third parties, including OpenAI, for tokens or GPUs. Apple is reported to have rented TPU instances through Google to train Apple Intelligence foundation models. That's all OPEX and what I think is a smart avoidance of massive capital investment in data centers given the speculative present and future of GenAI supercomputing.

While not entirely new, Apple has found a fresh role for App Intents which is essentially Shortcuts which Apple acquired back in 2018. This Apple’s answer to the emerging talk of agents from Microsoft, Google, and Nvidia. It turns out Apple has had a framework for agents for several years now that will now be driven by a rejuvenated Siri.

Siri infused with Apple Intelligence could put Apple in an interesting position of leadership in “AI”. While panned for years for being a bit of a lame duck among virtual assistants, a more contextually aware and conversant Siri could change the game especially with the introduction of Type to Siri effectively making Siri Apple’s equivalent of Microsoft's Copilot.

One esoteric Apple Intelligence announcement that stood out for me was Adapters. Thanks to the many technical briefings with Qualcomm on edge AI, I surmised Adapters referred to LoRA or "Lowest Rank Adaptation", a parameter-efficient method of fine tuning that involves adding parameters to an AI model rather than the costly exercise of changing parameters through full fine tuning.

Adapters allow a foundation model to be specialized for different domains of use and expertise precluding the need for multiple expert models. This will be important if Apple want’s to functionally scale Apple Intelligence while keeping its on device AI footprint lightweight but powerful and extensible.

Speaking of lightweight and tiny, Apple Intelligence on device will support 4-bit precision (INT4). Yes, INT4 is what Nvidia’s Blackwell BH200 NVL72 modular supercomputers will support dramatically accelerating certain (not all) GenAI inference workloads in the cloud.

Conclusion

neXt Curve has long argued that generative AI will find bright spots in narrow applications that are dependable enough to be valuable to consumers and enterprises.

It seems that Apple, in their thoughtfulness, has arrived at the same conclusion.

With the announcement of Apple Intelligence, Apple has demonstrated that they are indeed not “behind in AI”. They are taking an angle with generative AI that Wall Street and the industry didn’t anticipate and struggles to comprehend.

Apple Intelligence is flipping the script on "AI". According to Apple, AI (including generative AI) will need to run on device first and reach out to the cloud only as necessary with privacy first.

The value for Apple users? Trust.

The most astonishing revelation coming out of WWDC 2024 was how quickly Apple was able to mobilize its platform assets to assemble what is arguably the most compelling and comprehensive chatbot/agent framework that will deliver generative AI-infused features across most if not all Apple device categories.

IF Apple Intelligence turns out to be as good as marketed, Apple will have a very powerful platform to take the Apple experience to the next level. However, WWDC 2024 was notably devoid of live demos that have frequently embarrassed Apple’s peers to the tune of billions.

Unsurprisingly, Apple Intelligence will be rolled out in the Fall as a beta.

For a more in-depth neXt Curve analysis, go to www.next-curve.com.

Related Content:

• LinkedIn: "WWDC 2024 & Apple Intelligence Live Takes" (link )
• LinkedIn: "Immersive Video" (link )
• Linkedin: "Math Notes" (link )
• neXt Curve reThink Podcast: "Data Dive 911! Microsoft Recall & Apple Intelligence Privacy (with Debbie Reynolds )" (link )
• The neXt TechCheck Podcast (with Mark Vena ): "Hot Takes from Computex and WWDC" (link )
• Tantra's Mantra Podcast (with Prakash Sangam ): "Computex 2024 and Apple WWDC Event Recap" (link )

Subscribe to the neXt Curve Insights monthly newsletter to be notified when the next newsletter is published. Go to www.next-curve.com to be added to our mailing list. You will also be notified when we publish new research notes and media content.

Follow us on LinkedIn , Twitter , and YouTube .

This material may not be copied, reproduced, or modified in whole or in part for any purpose except with express written permission or license from an authorized representative of?neXt Curve. In addition to such written permission or license to copy, reproduce, or modify this document in whole or part, an acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced.

??2024 neXt Curve. All Rights Reserved