Augmented Reality and the Broker

I’ve only just unboxed the HoloLens 2 a few hours ago. Most of the time since, I’ve had the headset on – my eyes are a bit sore. This is my first experience with an AR headset. I always hoped Google Glass would catch, but it was ahead of its time. So now I sit here reflecting on my first 3 hours of time spent in an Augmented world.

First, the unboxing is wonderful. It’s quick and easy to set up assuming you have a Microsoft 365 account already. The initial amazement at the holographic objects wasn’t quite as life-altering as when I experienced Virtual Reality through the Oculus Rift. More on that another time.

The headset itself is intuitively adjustable and as comfortable as a bulky headset is going to get. The awkwardness questions posed by society are rendered largely moot given I work from home and rarely wear trousers now. When it comes to long periods of use with tech, it has to be worth the change in behaviour or it won’t stick. But before I get into the use cases, the exploration of the apps which already exist through the Microsoft Store are somewhat wide-ranging and pretty well developed. I’d heard that Magic Leap had more impressive stuff out of the box and to be prepared for bare bones with HoloLens.

It felt like magic.

There have only been two moments which have blown my mind thus far and one came from the robot shooter game where evil techno-creatures appear to be bursting through my actual wall in my house. The effect is unexpectedly accurate. It isn’t entirely clear how well my large room was mapped out so to see a more distant wall in play was a wonderful little surprise.

The second surprise came from an experience in the Playground app which allows you to see what kind of visual and interactive effects can be achieved. While lightning bolts coming out of my fingers was cool, the simple joy of experiencing an automatic, eye-driven scrolling function on text made the hair on the back of my neck stand on end. Imagine getting to the bottom of the page and the page auto-scrolling down based on your eye movement (or back up the page). It felt like magic. I found this particular ability within the headset to be one of those things that I could find hard to let go of.

And that is the entirely the kind of thing AR/VR needs to accomplish – a feeling that without them something kinda important is missing.

So, what is it good for? I’ve set up my first mixed reality meeting using the Spatial app and found it much more intuitive to use in AR than in VR. It will be worth understanding whether the feeling of a shared whiteboard and more tangible presence becomes one of those things which is just plain better. We’ll test a variety of collaboration contexts to see where it shines and where the hassle beyond a phone call is just not worth it.

I found I memorised the AR environment unusually fast.

Another thing, which may not apply to everyone, I found I memorised the AR environment unusually fast. I don’t have metrics to point to as evidence but as I took the headset off and handed it to my partner for her to try, I found it easy to guide her through every menu and experience I’d had even though I’d only done it once. I knew exactly where I’d left virtual screens pinned around my living room and what was on them. One of the touted use cases for AR/VR is accelerated learning – the opportunity to be more hands-on than a book allows has obvious benefits. Whether that marks out an opportunity in the financial services world for training seems entirely unclear generally speaking but more obvious in specific niches.

The talk-to-type seemed slow. Whereas, I found Oculus to be nailing that at the moment – better than I’ve ever seen it. However, the virtual keyboard in AR was way easier to get on-board with as VR has not figured this kind of interface out yet. Generally speaking, it is easier to see how an AR headset can play a role in a day-to-day simply because it does not cut me off from the real world – it simply adds to it.

But beyond a potentially better way to have certain types of meetings or to collaborate, where could it work for me and my colleagues in financial services, namely specialty commercial insurance?

1. Screen space is a nightmare, I work on a Surface Pro (iPhone, iPad, and Razer for media projects) and love being able to move around throughout the day. I'll move between lounging, or standing or even taking meetings outside. This could give me much needed screen space that moves with me.
2. I don’t know yet if internet research and browsing is doable for long periods of time within the headset, I would guess not. It does depend on the day and the type of project I’m involved in but I could see the benefit of walking around more as I work, read updates, check email, without having my head buried in a phone. Is that enough of a reason to throw a headset on? I still don’t think so.
3. Sharing my video from my Point-of-View live isn’t important to anything I’ve done yet. It could be a fascinating way to do update content for the company and just do recorded interviews with the headset on. I can imagine the people who are shy or just awkward in front of a normal camera doing much better just talking to a person with a HoloLens on. Again, this wouldn’t be all day, but would act as a camera which is used occasionally.
4. TL/DR: I don’t know that I care that much that I could have a 3d version of a graph about cyber attack activity in Wisconsin hovering over my coffee table. Data visualisation is a strange area for me at the moment. I’m in a pessimistic place about data visualisation in that I think it is rarely meaningful. The opportunity to visualise data usually means you either already understand the trends and patterns you are looking for and could simply derive meaningful alerts algorithmically, or you don’t understand the trends and patterns and don’t have a maths background to interrogate the data you have. Either way, if you are turning to data vis you’re probably missing the point.
5. This is not an original idea and the headset wouldn’t be for me to use, but we could embed these headsets with certain types of insurance products which need inspections as part of the underwriting process. The point would be to get a better account of the state of a building or an asset using something not dramatically different to taking a video with an iPhone except now both of the inspector’s hands are free while shooting the video. Seems cool but if they aren’t already doing this kind of video tour with an iPhone as part of the process then I struggle to believe a headset will change behaviours. I'd bet money it’s still where the world will eventually go but we may be a generation holding too tightly to “that's the way we've always done it" for these to get much traction.

It could be that the headset is more than the sum of its parts. It could be that in small ways a headset brings a collection of small improvements which mean we win business because we’re different, we’re innovative, and we attract talent and the next generation of the best and brightest because we do things like augment staff with actual AR (or VR) capabilities. It could be that simple but I’m convinced there need to be more small wins for this to hold water.

I also feel like I lack the necessary imagination, so if anyone has an answer for that…

I’ll keep exploring, it’s only been a few hours and the real trials don’t begin until next week.