Validating Features for an Augmented Reality Product

How do you design and validate a feature for a product of a type that no one has used before, built for a platform that very few people currently have? ?That, in a nutshell, is the difficulty of validating product features for AR native products for the Vision Pro.

If like me you come from a background of software product management, you’ll find that the traditional techniques used to validate a feature, don’t quite apply. Typically, one starts off with user interviews to understand their pain points, then you progress to mocks which are shown to potential users to get their feedback. ?You then launch demos to get people to sign-up or ideally even pre-pay for your product. The latter being the ultimate validation. Finally you monitor the usage of features to make sure that they are useful.

For a product running on the VisionPro though, each of these techniques, while necessary, turn out to be less useful than they are in more traditional contexts.

The Challenges of Describing an Augmented Reality Product in User Interviews

First, having conceptual discussion around a 3D spatial computing environment turns out not to be very effective. Most people have trouble visualizing a user interface that exists in 3D. They keep trying to associate your product with something they are familiar with in the 2D space of normal screen based applications.

This turns out to be both an important constraint and a hint on how new spatial computing products need to be designed. ?It is important that for 3D spatial computing products, that they either have easily understood counterparts in 2D space or familiar analogies in the real world. People have no trouble visualizing a Netflix App for the VisionPro, or a virtual monitor for their Mac, for example. ?They have an easily understandable real world analogue for those products: a monitor or a TV on a wall. This makes it easy for them to visualize the benefits of the product.

If your product is more conceptual, like the spatial IDE (integrated development environment) that I’m working on, ?it’s harder to convey the product idea. I found myself having to describe a 3D blow-up of a web-application with the code behind the page being visualizable in 3D, git trees in three-dimensions, connected to class diagrams and code snippets distributed in 3D space. ?It’s harder, but one signal of whether I’m resonating with a user is when they understand what I’m talking about and then start analysing the idea, poking holes in the product concept.

While on the face of it, objections might seem to be an indication that your product or feature is not viable, that isn’t necessarily true. At this stage, what you’re looking for is understanding. You should pay attention to reasons a person would not use your product, but you need to put it in context. Often, negativity about a verbal product description is indicative of scepticism of your ability to build the product as described. Users are used to being on the receiving end of marketing hype and they automatically discount your descriptions of your product. This is normal and you should not take that scepticism to be an indication that your product itself has no value.

If however, you get blank stares, or worse, polite agreement, then you have a real problem. The user almost certainly did not understand your product.

Product Mocks are Harder to Create and May Lead to Overly Optimistic Feedback

Mocks are also surprisingly complex to use as a way of validating a product. The first difficulty of mocking an #AR product is that it’s a lot more involved. Unlike a SaaS or e-commerce product, you cannot mock a #AR product in something like Figma or Adobe XD.? You need to replicate a 3D environment with something like Blender or Maya.? The skill set needed to create even a low-fidelity, mock is much greater. A high-fidelity mock is essentially a small cinematic special effects project, and would be, for the most part infeasible both in terms of money and time.

The second problem is more subtle but much more pernicious.? Typically, 3D mocks and videos of them, are inherently more engaging than 2D photos or mere verbal or written descriptions.? So I would get more wow! reactions when I showed my users a 3D mock. The user in this case is reacting to the novelty of the demo rather than the utility of the functionality that the mock represents. ?A good way that I’ve found to separate a signal from the noise in this case is to try and get the user to describe how they might use the feature. If the responses are vague, it’s a signal that your feature isn’t quite as useful as you might have thought. If, on the other hand, the user starts describing how they might use it and the problems they might run into, then you’ve landed on something useful.

Prototypes Need to be Usable Outside the AR Environment?

Prototypes are the next step in validating features. With VisionPro prototypes, however, I faced a peculiar problem.? There aren’t too many #VisionPro users yet. Adoption is growing very rapidly, but the subset of VisionPro users who are also developers, while not negligible is small. So to get better feedback, I’ve had to actually create two versions of every prototype, one built with WebGL and Web Assembly that allows a user to experience a feature in their browser, and one that actually runs on the Vision Pro. In addition, I create videos of the user interactions and use those to get feedback.

3D prototypes elicit much less scepticism than mocks, because they are functional, the user can kick the tires of the product to see if they would like to use it. ?This makes it easier for them to believe that you have something real. With a prototype, a strong indication of whether you have a winning feature is repeated and lengthy exploration. If the user spends time exploring the prototype, that is a strong sign that you have a useful feature. ?Quick drops offs are indication that the feature isn’t quite resonating for some reason. In the latter case, I try and see if I can get some facetime with the user to get their feedback on why they were not interested in the feature. Failing which and

Web prototypes of an AR product however, have significant limitations. Because the interaction experience is not the same as one would have in the real Vision Pro product, one needs to be careful to design the prototype in way that the UX of web prototype does not get in the way of gathering meaningful feedback. ??If done right, the feedback can be valuable.

Be Aware of Apple App Store Rules when Planning for Pre-Payment

I haven’t yet reached a stage where I could reasonably ask for pre-payment for the product, but given the constraints of the Apple VisionPro AppStore, this may be less easy to do. However, I suspect that features like in-app purchases and free trial period for the app may be good ways to test if the product is likely to get real paying customers.

To summarise, validating a #AR product is significantly more involved than validating a SaaS product or software application, but if traditional techniques are suitably modified you can use them effectively to find product market fit for your #AR product.

#productmanagement #AR #VR #augmentedreality #virtualreality #spatialcomputing #visionpro #userresearch #productmarketfit