Pioneering immersive therapy with Apple Vision Pro: A research summary of Project Aria

Authors: Jillian Ahrens , Chief Product & Design Officer, RealizedCare

Katie Howarth , XR Designer, RealizedCare

This past year, the product team at RealizedCare, in collaboration with Hunter Hoffman, Ph.D. , a pioneer in VR pain management, and Mindway VR , spearheaded the exploration of Apple Vision Pro’s potential in enhancing pain relief, (e.g., distraction), and mindfulness. We called it?Project Aria.?

Our objective was to understand if we could take advantage of the introduction of spatial computing and increased immersiveness of the Apple Vision Pro to create an experience that reduces pain and improves mental well-being. The result was a meticulously crafted immersive experience aimed at pushing the boundaries of what spatial computing can achieve in therapeutic settings.

Continuing the Journey: A new program built on existing research

Virtual Reality (VR) analgesia is a non-drug pain management technique first published by Hoffman et al, 2000.?Hoffman's work on SnowWorld demonstrated how VR could alleviate the agony of pediatric burn patients by using immersive environments to draw patients’ attention away from pain, making medical procedures (e.g., burn wound cleaning) less painful and more fun.?

Aria’s pain management builds on this successful foundation. It is a fully immersive experience, cutting off the real world around the user and pulling their attention beyond the physical world.?Combining real tactile sensations with virtual visual and auditory inputs, participants float down an iridescent, bioluminescent river. They pass by spatially localized waterfalls and immersive sound effects, all while their real hand rests in a bowl of lukewarm water. In Aria, using eye tracking and hand gestures to interact with objects in the virtual world, looking at and clicking glowing orbs transform the world around you, taking advantage of spatial computing to bring relief to those dealing with pain.

Harnessing water’s healing energy for stress relief

Aria Mindfulness, a variation of the program, is?designed to help people learn and practice new psychological skills, like?stress reduction, breathing techniques, that they can use later out in the real world to help reduce persistent pain and persistent stress.??

VR mindfulness is a technique first published by Hoffman and colleagues in 2016. In his recent laboratory study using our new Apple Vision Pro with healthy volunteers, Hoffman’s preliminary results using Aria Mindfulness (with mixed reality tactile input using real water) show a very large, statistically significant increase in participants ratings of how much more mindful they are feeling, after only 10 minutes in Aria Mindfulness.

In Aria Mindfulness, participants hear guided meditation instructions as they float slowly down the iridescent bioluminescent river, again, with their hand in a bowl of real water. Water is central to the Aria experience. It builds on the longstanding tradition of using water for healing, focusing on its therapeutic bene?ts.?

Water and mindfulness have been used for centuries in ancient rituals to help alleviate tension, calm the mind, and induce tranquility, and reduce psychological problems (e.g., depression). By harnessing immersive technology like the Apple Vision Pro, Aria transports users to stunning virtual environments, integrating water's soothing properties into a unique approach to pain relief and mindfulness. This fusion of ancient wisdom and cutting-edge technology offers a fresh approach to modern healing.

What’s next? Exploring other worlds of water

In addition to iridescent bioluminescent rivers, we have also designed but have not yet tested VR worlds with water in other physical states of liquid and vapor, “Evaporation” and “Condensation”. Picture yourself drifting through mesmerizing landscapes shaped by the ?ow of water, the glow of bioluminescence, and the artistry of cutting-edge design.?

In Aria “Evaporation”, interacting with glowing orbs make light beams stretch to the sky, ripples dance across the water, and bioluminescent patterns pulse on the surface, helping the participant focus on their breathing. These interactive elements make each journey through Aria unique.??

While doing small-scale user testing we discovered a few main themes.?

• Users consistently found Aria’s visual experience captivating and calming.?
• More novice VR users were particularly impressed, noting a sense of immediate awe and relaxation.?
• Even experienced users, like Hunter, were surprised by the high-quality visuals and attention to detail of Aria bioluminescent river.?
• Overall, participants reported feeling more at ease and relaxed throughout the experience, with the design effectively creating a peaceful environment.

Key Findings

Through this experimental design, we have made some key ?ndings about the capabilities and potential of the Apple Vision Pro for extended immersive experiences.?

Apple’s Vision Pro enables more immersive environments

The headset’s stunning visual clarity makes environments more engaging, giving users an increased sense of presence. Historically, an increased sense of presence elevates the effects of pain relief in VR. Apple Vision Pro’s novel human-computer interface using hand gestures and eye-tracking instead of controllers makes interactions feel more intuitive and natural, avoiding the clunkiness of traditional controllers.?

Connecting the tangible to virtual reality is effective

Hunter Hoffman's idea of connecting the real world with the virtual world—such as physically placing your hand in water in a water-based experience—proved to be a critical advancement, providing a more enriched and effective therapeutic experience. We encourage users to start Aria with their hand in a physical bowl of lukewarm water to feel more connected to the virtual water around them.?

Apple Vision Pro still needs enhancements

The capabilities of the Apple Vision Pro allows for more progressive approaches to modern healing with its advanced technologies, however, the Apple Vision Pro has challenges that need to be addressed, particularly concerning the capabilities of the ?rst-generation headset. For example, to create a realistic water experience, we had to develop custom graphics because existing solutions for other platforms like Quest were not directly compatible. Additionally, there is a lack of haptics given the lack of controllers which takes away from computer-generated physical sensations in the experience.??

Another challenge is that the Apple Vision Pro headset is calibrated to each participant.? Guests often need to go through an initially interesting but time-consuming calibration of their hands, eye tracking, and interocular distance adjustment before using the Apple Vision Pro.? But for everyone except the owner of the headset, our understanding is that this calibration data is currently lost as soon as the guest takes off the Apple Vision Pro. We would love to see a version of? the Apple Vision Pro that can store the personalized information from guests for future use (e.g., “hey Siri, set profile for Mariana”) so a guest doesn’t have to recalibrate each time they takes off the headset.??

What’s next for Project Aria?

Moving forward, we’re excited about adding custom functionalities to this experience. Plugging in AI-based interactions will increase immersion. Think of reaching out your hand and a creature instinctively landing on it, using AI to increase the environment’s awareness of the participant’s presence. When the participant sees the world reacting to their existence, it helps them feel like they are actually there.??

We are also exploring ideas like shaders that react to touch, having materials (e.g., a new visual texture) appear and travel up the hand/arm by touching them in the virtual world. By increasing reactivity in the world, we can ultimately increase presence and therapeutic effectiveness. We look forward to developing these ideas and exploring all that this headset has to offer.?

Whether you’re a researcher, developer, or healthcare professional, there’s immense potential to create new solutions for pain relief and increased mental health and wellness using the new Apple Vision Pro. Together, we can push the boundaries of coping with pain and foster resilience, healing, and well-being.?