Windows Mixed Reality Headset from HP

So this box showed up on my door step a few days ago.

Not that it was a surprise, I've had the thing on order for several months now.

It's this pretty sleek-looking VR headset!

HP's entry into the VR space is something Microsoft is calling the Windows Mixed Reality Headset. Acer has one as well; it looks kind of like a kid's lunch box strapped to your face. While Dell, Asus, and Lenovo are joining in with their own implements later this year, you can get the HP or Acer device right now. I had paid around $360 for the HP plus rush shipping. I haven't tried the Acer yet, but the electronics are supposed to be identical, and in a conversation online, it sounds like the HP is sturdier than the Acer.

Setup was fairly easy. One thin, round cable at the headset that runs the length of your room back to your PC, where it splits into HDMI and USB 3.0 for the last 6 inches. There are no other connections, i.e. no power cable and no external tracking hardware. Even my touchscreen monitor required more connections, but the HP can pull all the power it needs from the USB 3.0 port.

Windows 10 Creator's Edition auto detects and installs everything in seconds. There is a very brief calibration process. Stand in one spot and stare at your monitor for ten seconds (and the name of your love will appear! Erh, ahem, wrong site). Walk around your room. It's somewhat similar to setting up SteamVR, but not as fiddly. There's no "set your controllers down on the ground", for an example. You don't have to make sure the headset has a line of site to anything in particular while walking around the room, for another.

In my case, I did have some minor trouble during setup. My computer is a little old, so the USB 3.0 ports on the motherboard weren't able to supply enough power to the headset. I happened to have an extra USB 3.0 daughter card to install into a PCI-E slot. Incidentally, this is the same card that the Oculus Rift refused to recognize as USB 3.0, but it works fine with Windows MR. Assuming I didn't have the equivalent of a Ford Pinto with a V6 as a computer, the whole setup process would have been a breeze.

That's a bad analogy, the Pinto had a V6 as a factory option until 1980.

Once I got inside, the voice-over assistant instructing me on how to use the software made reference to "hitting the Windows button". This was confusing because there's no button on the device and I was standing in the middle of the room, far away from my keyboard. Given Microsoft's marketing on these things, I had expected it to work like the HoloLens, just with an opaque screen. That means I was expecting to use hand gestures to control the device.

That is not the case. You need a wireless Xbox One game pad. Microsoft specifically says you're on your own with any other brand of game pad. I use the Xbox One S controller, because it uses Bluetooth Low Energy for transmission, which is the only version of Bluetooth anything should ever be, and because proprietary wireless dongles are straight-up malarkey. I have one of the few BLE dongles that works with Windows, so all of my devices now support BLE, which is quite useful for many of my hardware projects in IoT and Wearables.

The central Xbox button opens the Start Menu. Left stick moves you slowly and smoothly forward. Right stick scrolls window views, but does it rather poorly. Bumpers turn you 90 degrees in a blink. The Y button teleports you. The A button selects things. The B button cancels out of things. I don't know what the X button does. The triggers can be used to position things in Z space, but only if you are using the Resize tool, not when you're initially placing them, which is annoying.

This will become a theme throughout my experience with the headset: kind of fiddly, not quite right interactions. To preface, before I get too deep into complaints, this is just one app, the Mixed Reality Portal app that comes standard as the Windows Holographic experience. Most of my complaints are strictly about this app and don't really apply to the headset itself.

The Portal app is somewhat equivalent to Oculus Home on the Rift and Gear VR, or Viveport or the SteamVR system-view on the HTC Vive. Like those apps, it's kind of your main foray into the headset and how you launch other apps with it. Unlike those apps, it's actually useful on its own, there is space in which you can move around, place app windows, and have your layout remembered for later use. On the Rift and Vive, you need a 3rd party app like BigScreen or Virtual Desktop to do this, and they both have strict limitations on what you're able to do. With Windows Holographic, viewing standard 2D applications is a default feature. It's essentially the VR equivalent of your standard computer's "desktop".

You can mostly get by using a mouse and keyboard, but movement is kind of annoying—teleport only. Also, there are some defects with some apps (like Facebook's UWP app) where some UI elements only respond to selections indicated with the gaze cursor and triggered with the game pad. But despite these annoyances, it feels like the system as a whole is trying to give me as many options on how to interact with it as possible, to let me use the system how I want, rather than forcing me into a specific set of interactions. Oculus and Valve have taken more of a hard-line stance on "good VR UX", which I personally think is a mistake, as it can cut down the number of options for people with physical disabilities who may not be able to use things in strictly proscribed ways.

They are supposed to eventually release motion controllers, but I am not aware of how you can acquire them, if they are even available, or on what sort of timetable or price point they will be available. All I've seen is a rumor that they'd be bundled with the Acer headset for an extra $100 late this year. If anyone is privy to the real story, please let me know. No one from Microsoft or the HoloLens team has replied to any of my inquiries.

Wearing the device is very comfortable. It's very similar in setup to the HoloLens, except it doesn't weigh as much as a clay brick balanced on the bridge of your nose. It's probably the lightest headset I've used thus far. Even if it's not literally so, it certainly feels like it, as the center of gravity for the device is far rear or where it ends up on competition devices. The effect is that the display end of the headset has very little momentum of its own when you turn your head back and forth, a distinct problem with the Samsung Gear VR and HTC Vive.

It's also probably the most comfortable headset, as well. The previous most-comfortable HMD fell on my own hacked-together version of a Google Cardboard I built with a head strap salvaged from a jeweler's dual-lens magnifier mask and weather stripping around the face. The HP is significantly less janky, I promise. But the principle is the same: the leverage for holding the display on your face is taken from the crown of the skull, not across the cheekbones and around the orbital bone. With some devices like the Samsung Gear VR, I personally start to feel a dry mouth sensation as I wear the device. I suspect it's pinching something related to my saliva ducts. But the HP headset is much more akin to a snugly fitting hat or helmet, an experience far more amenable to comfort and not looking ridiculous when one removes the device.

Getting into the sweet spot on the lenses is somewhat difficult. It's using a Fresnel-type lens (FYI, it's pronounced frey-NEL, not FREZ-nel) similar to the Vive and Rift, but unlike the Vive or Rift, there are no unsightly lens flares, rays, or banding artifacts. No, the Windows MR headset has its own, unique issue, where viewing through it off-sweet-spot makes the image look smeared.

To get the best view, you will want the band to sit across your forehead, just below the hard part where you can headbutt another person and not be hurt yourself. If you've ever visited Glasgow, Scotland you know what I'm talking about. The dial on the back of the strap (which is the primary differentiating feature from the Acer headset, along with not being a ridiculous toy-color) should sit just about where a bun hair-do would sit. So if you have hair that is more elaborate than "balding, middle-aged, overly verbose guy", then you might have some difficulty—though it should be noted it's probably significantly less difficulty than the competition headsets would give you. I've personally seen their overhead straps cause quite a bit of trouble for people, men and women alike, with long, voluminous hair. You will be able to tell if have the headset positioned correctly by paying close attention to the direction of the smearing of the image. Move the headset up or down in the opposite direction of the smearing. Start with the forehead, then rotate the back into place while turning the dial until it is snug and secure, but still comfortable.

That sounds quite fiddly, but it's less effort than getting an HTC Vive in the optimal position, and more comfortable than even the Oculus Rift. The Vive hugs your face very tightly (especially if you're doing it correctly, you want those lenses as close to your face as possible, which means crushing down some of the weather stripping with your face a bit). The Rift does as well, though not quite as tightly, thanks to the more rigid design. The straps for the HTC Vive and Samsung Gear VR, being nothing more than elastic with some Velcro sewn into the end, lacks the rigidity necessary to be able to leverage the structure of your skull for support; they have to brute-force hug your orbitals, at 90 degrees to gravity, which is the worst possible way to try to fight a force. The Rift can get some grip on your skull because the overhead strap is fairly rigid, but its center of gravity is so far forward that it still needs to stand on your upper cheek bones quite a bit. Also, the Rift's weather stripping is so narrow that, at least for me, it leaves deep impressions on my cheeks and across my forehead. The Windows MR headset rests all of its weight on your forehead with a thick pad, using normal force rather than friction force to fight the machinations of gravity (or Grab-ity, as my little sister used to say, which I think is quite an appropriate moniker). Once the contraption is situated on your head, the display itself cantilevers into place. When I took it off, I had no more crease on my forehead than I ever do after wearing a baseball hat strapped snugly enough to not fly away in wind.

Once you do get it just right, the image is super clear and there is very little "screen-door effect" (SDE). And that cantilevering feature really helps to keep your placement if you ever need to peak outside of the headset for whatever reason. This is nearly impossible without screwing up your setup on the Vive. It's possible, but quite annoying with the Rift, by peaking through the nose hole. It's a main feature with HP.

That's enough on the hardware. On the software side, it's shockingly difficult to find information about and apps specific to this headset.

Shocking, I say!

Most of the links I find when Googling are either press releases about the headset or about apps for HoloLens. There is no separate Windows Store category for holographic apps. The holographic apps do not show up in the desktop view of the Windows Store. And the browser-based Windows Store won't let you install holographic apps because it can't recognize your PC as supporting Windows Holographic, even if you're using Edge as your browser. So you're basically stuck with 2D UWP apps and a few carryovers from the HoloLens that are installed by default. But lacking the AR view, things like the Holograms app just don't have the same impact. I'd really like to get rid of most of the legacy HoloLens stuff, but I haven't yet been able to figure out how to edit the Start Menu. It's not long-press. It should be long-press. Standardize on UI metaphors, people!

The emphasis on the MR Portal also makes it feel very productivity focused, not gaming focused. Not that it couldn't be good for gaming, it is a 90 Hz display, after all. But after using the apps and thinking about how to build software for the MR Portal environment, I'm having flashbacks to Windows 3.1 and how gaming stayed on DOS for so long because the window manager took too much control over the rendering and event handling pipeline away from the developer. This is certainly more of an issue of product positioning than technical limitations, though. DirectX originally came to life because of this, but we now have DirectX and it's all you would need to build something different than the MR Portal.

In the first days of Windows, Microsoft had been pushing GDI as the API for rendering game graphics, much in the same way they are pushing UWP now. Ski Free is perhaps the only recognizable, 3rd party game that managed it. The famous Space Pinball game would essentially be a launch title for DirectX. UWP apps in Windows Holographic feel like trying to play Ski Free and be happy about it when all your friends are playing Wolfenstein 3D and Doom in DOS. VR is not meant to be a 2D environment. I suppose it's kind of nice that I can get a few legacy apps into the space fairly easily (and any non-UWP apps can be collected into a single Desktop view window). But ultimately, I want for a UI that is completely expressed by virtual objects that are literal interpretations of their purpose. To carry into the immersive space the Windows-Icons-Menus-Pointers metaphor that we've had mostly unchanged for more than 35 years is just... well... wimpy.

Take a look:

Tracking is really solid, even better than HoloLens. HoloLens used to have trouble with sitting in front of my monitor. This has no issues. Text is very nice to read in this headset compared to the Vive or Rift. I tested this by placing the "Desktop" window view (which you can see in the center of my short video) in roughly the same place as my monitor in the real world. I did have to scale up text, but not a lot. The vertical resolution is about 16% better, and the field of view is slightly narrower, so perhaps that's it, but it feels like there is something more. There is very little visible edge between each pixel, and that edge is not black, it appears the color of the adjacent pixels bleeds into a little bit. It improves the overall experience dramatically. There are times with the Vive that I find it very hard to ignore the SDE, that I have to get into a clear-minded focus, or actively busy and distracted, to forget that it is there; the default is to notice it. But with the Windows MR display, I have found that I have to expend effort to notice the SDE, that the default is to ignore it. It's there, but it just so happens to be on the opposite side from the Vive or Rift of an edge separating being actively distracting from not being a terrible hindrance. As a result, I think this headset is MVP for live-programmable VR ala Primrose VR and RiftSketch.

As you can see in the video, this isn't "mixed reality" at all. Microsoft goes on at length to explain what they mean by "mixed reality", but I can't make heads or tails of where this device is supposed to fit on their spectrum and what it's supposed to mean, feature-wise. There is no pass-through camera mode. I doubt, given the positioning of the cameras, that having a pass-through mode would be visually appealing. There is no hand gesture capture. There is also no volume capture, so that demo of the live space being represented as a point cloud in-headset that Microsoft showed off 6 months ago is either fake or so janky in real use they can't release it to humans. The device can use the cameras on the headset to figure out its position, but only after you do a calibration walk around the room to setup a tracking volume. It's up to you to make sure you define the boundaries far enough away from stuff that you don't run into anything.

Cortana is garbage, but that's nothing new. I have to yell to get it to respond. It's weird, I seem to remember it working better on the HoloLens. It keeps piping up to remind me that I can talk to it, which is getting quite frustrating. It ends up putting a voice inside your head that doesn't have a body and I haven't figured out how to turn it off yet. I had to flip up the headset (after having to first remember that I could) to "escape" from it at one point, it's just so jarring. I have built software that uses voice commands before, including a HoloLens app, and it can be a great way to interact with things, but in this case it feels like more effort than it's worth.

No motion controllers as of yet is extremely limiting. Standing with a game pad is stupid. Sitting at your keyboard, which will be on your desk so it will not be in your tracking volume, so you will probably be overlapping the boundary chaperone, is also stupid. I kept clipping in and out of electric snow flakes as I sat at my keyboard and tried to type an email. Not fun.

I put on some music in Groove (the soundtrack to Moana, if you must know). In the Portal app, they render the audio as if it is coming from a specific point in space. It works pretty well, so well it gave me the impression I had an invisible stereo in my real house after I flipped the headset up to be able to use my regular monitor for a few things. Spatialized audio as its own device without a headset at all is an interesting idea. Also, it's pretty much the only VR that will work for blind people, so we should probably pay more attention to audio.

I am very curious to try this headset out with a backpack computer. You could conceivably do this in a warehouse, straight out of the box, unlike Rift or Vive. Well, the Vive could eventually do it, once the new version of Lighthouse is released, and you're ready to spring a lot of extra money for each set of base stations you need to scale your tracking volume. The Windows MR headset needs no such thing.

MS says they want to get this working on $500 PCs. I assume that means the $500 PCs of some nebulous tomorrow, not right now. But if they pull it off, that's around $900 all-in for VR with motion controllers. Gear VR ends up around that when you price for the phone, headset, and controller, but it's 3-degrees-of-freedom, not 6-DOF. Assuming the motion controllers work, I would say it would supplant the Vive as the "best" VR system, at that point. I still like the Vive for the motion controllers and library of apps that take advantage of them. While I've used the Rift quite a bit with game pads, I don't have a lot of experience with Touch, as I find the design causes very painful cramping in my hands (despite the insistence of tech-nerds that the design is "more ergonomic").

I know I complained a lot about this headset in this article, which might give you the impression that I don't like it. I actually like it a lot. There are some pros and cons with the device, as with any. This is a development kit, after all. It's kind of interesting to me that this device seems to have about as many flaws as the competition devices, which have called themselves "consumer ready" for over a year now. Microsoft is clearly going for a higher standard on what they consider releasing to the broader market. No motion controllers is definitely the biggest drawback. I had bought it for the cheap price, expecting it to be a only OK in terms of tracking and resolution. Surprisingly, price is not the biggest positive factor. The excellent tracking, the great setup experience, and the clarity of the screen are second to none.

It's unfortunate that there is so little content for the headset, because to me, inside-out tracking is the future. If we daydream up our ideal VR/AR headset, I don't think anyone fantasizes about setting up external tracking hardware (and all the associate power and data requirements to go with it). The Rift and Vive are excellent devices, but their designs are pretty clearly dead-ends. Microsoft and its partners definitely have a head start on a far superior design, one that can grow in obvious ways over time. Without a public utility approach to external tracking hardware, it will always be limited to installations, which means someone, somewhere is probably going to try to monetize access to it. That's not the sort of future that I want for immersive tech.