Extended Reality - Augmented, Virtual, and Mixed

With the release of the Apple Vision Pro last week, Extended Reality and Spatial Computing are back at the top of mind for technophiles. But what does this mean for everyone? Is Vision Pro going to grow the market? Who uses these platforms? What does development entail?

Let’s find out!

What is Extended Reality (or XR)?

Extended reality is a term that encompasses different aspects of these new interactive platforms so let’s review the terms from Wikipedia:

• Augmented reality is a system that incorporates a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects.
• Virtual reality is a system that simulates experiences and employs pose tracking of the users and 3D near-eye displays to give the user an immersive feel of a virtual world.
• Mixed reality is a term used to describe the merging of a real-world environment and a computer-generated one. Physical and virtual objects may co-exist in mixed-reality environments and interact in real-time.

As you see, Augmented and Mixed Reality have, essentially, the same definition. Augmented reality is used more generally for virtual and real objects in the same space. Microsoft uses the term Mixed Reality to describe its Hololens experience of virtual and real object interaction.?

Augmented or Mixed Reality are really what spatial computing is all about. Users can see their surroundings and interact with the computer technology within the overlay provided by their headsets or phones.

Who uses XR?

Gamers are the original audience most familiar with VR, but that is rapidly changing. Training and simulation are the most natural places for AR and VR use. For example, ?Microsoft’s Hololens and the Magic Leap are focused on education, manufacturing, and medical applications for training and simulating real-world situations. The platforms allow employees to train without being present in the factory, in a school, or performing a procedure on a patient. Headsets and systems for Steam VR namely, the Index and Vive, are on the higher end of the gaming virtual reality rigs and require a dedicated, preset space with sensors mounted to the walls. The Playstation VR and Playstation VR 2 are a little more flexible and simple to set up. It can connect to your TV, does not require sensors, and has a significantly lower price point. Facebook’s Quest platform is the least expensive of the group but has the drawback of requiring a Facebook account with overall lower fidelity.?

Industrial Designers and Architects have integrated more VR and AR to aid in their workflows to share designs and visualize products for approval prior to going through the expense of prototyping or building. The systems targeted at these spaces are the Varjo, with high resolution and more realistic rendering, Microsoft Hololens, and the Magic Leap.

Are there issues with XR?

The first issue is price. With the exception of phone-only focused interactivity, most of the gear needed is expensive and users need a reason to spend that money. The second is there aren't enough apps outside of gaming that would appeal to a broader audience.?

Lastly, and surprisingly, is the Experience Design. The user experience focuses too heavily on familiar paradigms of flat, 2D technology - literal flat screens hanging in space instead of novel interactions. Trying to make the new spatial format “familiar” lessens the dynamic experience and feels less valuable or special than it should be for everyone. Making the experience feel magical and delightful to do everyday tasks will go a long way to strengthening the power of spatial computing.?

Developing for XR

There are two primary development platforms for Extended Reality, Unity and Unreal Engine. Unity is a preferred IDE for Vision Pro while most other VR platforms can be developed with Unreal Engine. Unity is a robust IDE with similar features to Unreal Engine and both have their strong suits. However, Unreal Engine is a very flexible system with a user-friendly, node-based programming interface that has wide adoption due to its low cost. Apple and Epic Games (the developers of Unreal Engine) have a history of legal action which makes it unlikely that the Vision Pro team will jump on that platform.

How Apple Vision Pro will affect the Extended Reality segments?

Apple has a strong growth effect on industries; the company may not be the first in a segment but they drive interest and purchases for the segments they invest in. The Vision Pro platform will put extended reality into the hands of non-VR enthusiasts, getting more people interested in the platforms and spatial computing as a whole.

Extended Reality is not going away anytime soon, even with the current challenges of experience design and pricing, the technology that powers spatial computing is becoming more accessible every day.