Exploring Accessibility on the Apple Vision Pro (AVP) ?

I was out on the town Saturday night (a rare occurrence since COVID) and ran into an old friend at a show. He and I go way back - we've been arguing about Macs vs PCs for nearly 20 years. But he wanted to hear all about Apple this time because he saw a picture of me with my Vision Pro.

A friend of his has ALS, and he is hopeful that Apple Vision Pro (AVP) could provide a computing world that is conducive to someone with minimal mobility. Up until that moment I hadn't given much consideration to this use case. But as we started to talk, I realized the immense potential that the AVP has for people with disabilities.

From day one I've been stunned by how powerful the eye tracking in the device is. The system's affordances make this interaction paradigm natural and simple (in conjunction with simple hand gestures). However, eye-tracking technologies have been a core game changer for the ALS community, so the potential for the Vision Pro is evident.

This morning I spent some time researching some of the accessibility tools and how one might need to modify the core out-of-the-box experience to meet the needs of someone with limited mobility.

As one small but critical example - most users summon Home in the device by clicking the digital crown which is located across the top of the headset. Unfortunately, this could be problematic or impossible for someone with limited muscle control.

As always, Apple has done a stellar job of providing a myriad of interaction alternatives.

• You can summon the home by looking up to the very top of your view (but this is probably the hardest eye "gesture" that I've come across when using the AVP.)
• You can also access the home if you have Voice Control set up, but with something like ALS even speaking can become difficult.
• One of the most interesting/fun accessibility affordances they offer is called Sound Actions. You can associate a common action with a sound like a clicking of the tongue, or saying the letter T. It took all of 30 seconds to set up, and I don't think I'll use the digital crown to open Home ever again.

So... why does this matter? The AVP has the potential to allow differently-abled folks the ability to be more self-sufficient in a myriad of different ways, which is empowering. A simple example? This technology allows easy access to all of the rich communication tools that Apple already offers. FaceTime, Messages, etc. Sharing photos & videos. Screen sharing. Sometimes people think that VR is a technology that isolates us. I think that is a narrow view.

These notes offer just the tiniest tip of the iceberg about the accessibility features available in AVP. Apple and the teams that worked so hard to bring these thoughtful solutions to life deserve kudos, and I'm excited to track what happens in this space and to learn how the app dev community harnesses the potential of this opportunity.

If you'd like to go down the rabbit hole, nerd out, and learn more, there are plenty of great resources out there. Here are just a few to get you started -

• https://support.apple.com/guide/apple-vision-pro/get-started-with-accessibility-features-tan426c48bdc/visionos
• https://www.youtube.com/watch?v=IW3pa8YQeAs
• https://zmknox.com/2024/02/18/vision-accessibility-on-apple-vision-pro
• https://developer.apple.com/videos/play/wwdc2023/10034/ (for app devs)

Vaguely related... please holler if you've got questions about the AVP or resources that you'd like to share! I'll probably be writing a bit more about it in the coming weeks if people are interested.