The Matrix Metaverse

I just watched Matrix Resurrections and wanted to share a few interesting XR Matrix Metaverse concepts with the world.

First of all, I was excited to see my friend?John Gaeta?getting some screen time in the new Matrix movie! I worked for John as the CTO of Magic Leap’s IPL (Imagination Propulsion Laboratory) “think tank” but John is best known as an Academy Award winner for inventing?bullet time.

Our conversations frequently cover many real-world Matrix Metaverse talking points listed below including: XR Co-Presence, Digital Twins, XR Portals, Teleportation, XR Embodiment, Avatars, XR AI Oracles, and World Scale. Note: The evolution of the real-world metaverse will lead to the Spatial Web as described in?Gabriel Rene's book?The Spatial Web.

XR Co-Presence

John and I frequently still discuss the importance of XR co-presence in the metaverse. It’s an idea where VR users can teleport to a geo coordinate in the real world and interact with someone physically present at that location via AR — and visa versa. (Note: these discussions inspired most of my work on the XRPA.NET real world metavese project — more info)

The new Matrix Resurrections movie does a great job portraying the XR co-presence idea throughout the movie. You can see Sequoia virtually co-present with Bugs and team on most of their trips inside the Matrix while he remains on the ship. If you pay close enough attention, you will even see hologram dissolving effects on him when he disconnects.?

You can also see the 3D volumetric streams of the programs from the Matrix appearing in the real world such as Morpheus and the digital botanist.?

Checkout the work underway by my friend and former Magic Leap colleague David Doepp as well as my friends at StreamSoar in the 3D volumetric streaming space.

Digital Twins

There are many references to digital twins in the new movie. They are portrayed as city models appearing as green matrix code. The best example was during the modal security alert where Neo walks into a camera overlay of the digital twin. Digital twins are vital in order for the real world metaverse to evolve into the spatial web as explained by Gabriel Rene, friend and founder of Verses.io.?

Here’s an XRPA.NET example of a user teleporting to a dynamic digital twin of Disneyland via WebXR — more info.

XR Portals and Teleportation

Mirrors and doors are used as XR Portals connecting “AR” and “VR” worlds as well as “matrix-to-matrix” destinations i.e. teleporting / holoporting.?

There are many startup companies working in the XR Portal space and many cool brand experiences available like this Verizon Christmas portal.

XR Embodiment

John and I also often discussed the idea of XR Embodiment. This could be the next wave of AR glasses commerce experiences. Imagine if your favorite celebrity wore a pair of Hololens or Magic Leap glasses to an event such as a concert and you could experience the event as they did/do by seeing and hearing everything that they saw, heard, and said. Sign me up!?

Avatars as Residual Self Images (RSI)

We often see ourselves differently than others see us in the real world. Avatars are a way for us to create our residual self images in the metaverse allowing others to see us as we want to be seen. This even includes NFT-based wearables such as clothing, weapons, venues, etc.?

XR AI Oracles

Remember the Oracle who appeared in previous Matrix movies? She was an AI program designed to help or guide people in the Matrix.

When I first joined MagicLeap in 2018 through the acquisition of my decentralized supercomputing company,?Computes, I was blown away to meet?MICA. She is/was an AR-based, life-sized digital human only visible from within MagicLeap’s AR glasses. While she didn’t yet speak verbally, she would walk up to you and stand in front of you, make eye contact with you, and try to keep your attention focused on her eyes. Over time, Magic Leap built several interactive demos where MICA would interact with you in training simulations or interactive games.

You could imagine future versions of MICA sitting down with you and your family at the dinner table helping you plan a family vacation together.

I’m not sure what happened to MICA. I haven’t heard anything about her since the company’s massive layoff a couple of years ago. Checkout my recent article,?Digital Humans: The “Amazon Echos” of the Metaverse?for more ideas and information on XR AI Oracles!

Back to my friend, John Gaeta. He was also the Creative Executive Producer of?The Matrix Awakens, an Epic Games Unreal Engine 5 Experience. Watch his new video clip below and try to convince me that the Matrix is not real. ??

In addition to John’s contributions to the Matrix, he is also credited for creating the foundation of Magic Leap’s metaverse AKA?The Magicverse?along with Magic Leap’s founder,?Rony Abovitz, and?Neal Stephenson, the author of?Snow Crash?and thus the godfather of the metaverse.

As you can see from the visuals below, everything is interconnected — the physical world and the digital twin, the Internet of Things (IoT), artificial intelligence (AI), human and environmental inputs, communications, maps, and transactions. Verses are simply “layers” on top of the metaverse construct. The result is a system of systems of dynamic digital twins, spatial addressing protocols, contextual data/deep learning, transactions with privacy and security, XR interfaces (ML, Hololens, Oculus, Vive, etc), human and world interfaces, an app layer manager, persistence, and a compute manager (AKA?Computes inc.?my decentralized supercomputing platform acquired by Magic Leap in 2018).

Remember The Matrix’s construct? Coincidence?

World Scale

IPL created a few world scale demos including: smart cities, a life-size flying whale, and even a life-size elephant that could interact with you. These experiences were typically too large to experience as intended with a mobile device thus requiring a headset to truly experience things at world scale.

IPL, under John’s leadership, created the whale and elephant exactly as was conceptually presented at the start of Magic Leap as highest fidelity, interactive mixed reality experiences. The whale was based in the exact source materials as the original whale in gym concept and the elephant was cinema caliber as well. Both could be experienced at 1:1 scale. They were impressive next generation proof-of-concepts that validated the promises made by Magic Leap in the beginning. They pointed toward a “metaverse for the real world” as some like to describe it.

My favorite experience developed with John in IPL was Magic Rain! It was basically Matrix code visible through the ML1 as the device scanned the room. It still gives me goosebumps :)

For a number of these experiments, we worked with development partners like Weta, Magnopus, The Mill, Framestore, and ILM. We leveraged Unreal and other super tools. We partnered with groups like The UN, Smithsonian, Dosney, Imagineers and Dubai World Expo Curators amongst others.

Here are a couple of other interesting and interactive Matrix Metaverse experiments to share:?Living Machines?and?Artificial Human with Gun.

Here’s a great new interview with Keanu Reaves and Carrie-Anne Moss discussing the metaverse, cryptocurrency, and AI.

Well that’s just my first take from watching Matrix Resurrections the first time. Like the trilogy, I’m sure I’ll be watching this one many more times and catching many more Easter eggs throughout the movie.

Thanks Matrix! Now I have a bunch of new ideas for?XRPA.NET?and?Metaverses.io! Stay tuned…