Unlocking the Untapped Potential of Apple's MR: A Game-Changer for the Killer Apps Tech Ecosystem

Title: "Revolutionizing Realities: Apple's MR - The Overlooked Powerhouse in Tech Innovation"

Article Translation:

This article offers a comprehensive analysis of an underappreciated core capability of Apple's Mixed Reality (MR), a feature that could pave the way for groundbreaking applications, and it also identifies key players in this emerging field.

#Apple MR #Content Application Ecosystem

The enhancements of Apple's MR hardware over current VR technologies are multi-dimensional:

First, the display resolution has been elevated from dual 4K to 8K, substantially enriching the visual experience. Second, the shift in interaction modes from traditional controllers to eye movement and gesture-based interfaces presents a more streamlined and intuitive user experience with lower learning and adaptation costs, all while boasting superior Visual Simultaneous Tracking (VST) capabilities.

Third, the computational power: The M2+R1 split architecture allows the M2 chip to support dual 8K rendering, with the R1 acting as a dedicated processor to enhance interaction precision and speed (low latency).

Currently, the most significant expectation for Apple MR’s content scenarios, from both Apple and the industry, centers around spatial video. Spatial video capitalizes on Apple MR's exceptional display capabilities with innovations in the player (a 6DOF 3D player) and leverages the camera configuration of the Apple Pro series phones, thus creating a complete cycle of mobile UGC creation and MR playback display. This not only forms a robust content loop but also maximizes the photographic and MR display capabilities of Apple smartphones.

However, spatial video has not fully exploited the performance of VP, merely utilizing its display capability. Yet, the most fundamental capability of Vision Pro - MR capability - has largely gone unnoticed and unexplored in the market.

The MR functionality, present in devices like Quest Pro, Quest 3, and Pico 4, has not been optimally implemented, primarily due to:

High latency: Commonly, VST MTP (Motion to Photo) exceeds 20 milliseconds, sometimes even surpassing 50 milliseconds, causing dizziness. The human physiological threshold for MTP latency is generally considered to be around 20 milliseconds (like in Pico 4), with anything beyond that causing discomfort and nausea.

Low image quality: The direct view of the external world through VST suffers from low resolution and pixelation (a common issue with 4K), poor color reproduction (e.g., black and white pixels in Quest Pro), and noticeable distortion at the edges where images from dual VST cameras merge.

Weak depth perception: Insufficient sensors lead to a lack or inaccuracy of depth information. For instance, Pico 4’s VST has almost no depth information, Quest Pro lacks TOF, and Quest 3 is somewhat better in this aspect.

Apple's Vision Pro essentially addresses these three major pain points.

Firstly, VST latency: Apple's R1 chip reduces it to 12 milliseconds, effectively mitigating physiological discomfort caused by MTP latency.

Secondly, Apple's dual 8K resolution enhances VST image clarity (not yet at retina level), with Apple's refined algorithms minimizing perceptible distortions in the VST image.

Additionally, Apple's rich array of sensors, including binocular vision, dToF, and structured light, combined with R1 chip-based fusion algorithms, significantly improves spatial scanning and positioning accuracy, offering far superior spatial perception capabilities compared to existing MR products.

Thirdly, influencers who have experienced Vision Pro’s VST and MR capabilities outperform current MR devices. If Quest 3's MR functionality is considered usable, then Vision Pro's MR functionality is truly effective.

With the enhanced MR functionality, several breakthroughs are possible:

1. From purely virtual to a blend of virtual and real, becoming a new tool of productivity:

The VST feature allows Apple MR to support not just purely virtual content but also a blend of virtual and real, becoming a tool for productivity. For instance, assisting individuals in repairing and assembling large items like furniture, significantly enhancing efficiency.

2. From indoors to outdoors, expanding usage scenarios:

With improved VST latency and image quality, the product can transition from indoor to certain outdoor uses, such as in public transportation, restaurants, and parks, making it portable (also facilitated by the elimination of controllers and an external power source to improve battery life). By 2024, you will likely see many people wearing Vision Pro in public spaces in major cities.

3. From single-player to multiplayer content, opening new avenues for content applications:

Once the product is used outdoors, it creates connections between people. Leveraging Vision Pro's strong spatial perception capabilities and easy-to-learn interaction methods, light interaction and interactive content become easily playable.

In conclusion, based on the VST functionality of MR, I foresee the following content types as particularly promising:

First, weak interactive content such as Go, chess, and other tabletop games. Alone in a cafe, you could wear Vision Pro, turn on the MR feature, place a virtual chessboard on the table, and play chess (virtual) while drinking coffee or eating (real). If multiple people are involved, they could all chat, eat, and drink while playing chess or virtual tabletop games (based on the same virtual tabletop content for competition or battle). For instance, Mirrorscape, a board game company, has developed a multiplayer AR board game "ARcana," based on the official Dungeons and Dragons fifth edition gameplay. Mirrorscape announced the public beta version of its AR board game platform suitable for smartphones and Apple Vision Pro, expected to debut on Vision Pro. Domestically, companies like Denar Intelligence Valley have also developed several MR board games, and other companies with mature board games can quickly enter the MR board game space, such as CMON listed on the Hong Kong Stock Exchange.

Second, weak interactive social games based on LBS+VST MR, in various settings like bus stations, subway stations, parks, or buildings, can place various virtual information in three-dimensional space. For example, placing a virtual character or treasure chest at a subway entrance, which upon clicking reveals items or contact information, completing tasks for upgrades or rewards. This concept can be developed into games, like Niantic's "Pokémon GO," "Monster Hunter," Tencent's "Let's Hunt Monsters," Shunwang Technology's "City Spirits," etc. It can even evolve into local social networking, similar to the early days of Momo, placing specific social information in certain spaces, like "Wubu Space," or even venturing into edgy and grey areas. This holds the greatest potential and the most likely chance to create killer applications.

Third, practical AR tools, such as AR home decoration, AR try-on for clothes and shoes, etc. Based on Vision Pro's spatial scanning and positioning capabilities, the experience of combining virtual and real objects is enhanced.

BR,

VandaGeek

December 15, 2023