How will Augmented & Virtual Reality affect our average daily lives in 5-10 years?

Let's not worry about the escapist aspects of AR/VR (aka the Mixed Reality spectrum) and leave aside the industrial uses for now. Let's focus the question on our daily [mostly mundane] lives, where we spend 95% of our time and money.

Consider: movies and games are about a $90 billion dollar market today, but simply talking on the phone is a trillion dollar one.

I used this kind of thinking to help found the HoloLens project, back in 2010. I later gave a stealth talk on some related ideas, without disclosing the project of course. More recently, my friends started a VC/incubator on the theme of building superpowers. Even Bill and Melinda Gates are talking superpowers these days.

The common thesis boils down to:

Every game changing technology can be seen as a human superpower.

Augmented/Virtual/Mixed Reality is all about making more superpowers.

Try the exercise yourself. Pick any notable technology, like fire or television or even facebook, and figure out what superpower that best maps to.

Fire is easy?—?it’s the power to unleash energy. In fiction, we might expect a mutant “firestarter” character to unleash that energy using the power of her mind, blasting enemies with her eyes.

That's a bit too dramatic, isn't it? Simply discovering how to spark some tinder (the material, not the app) 60,000 years ago was enough to revolutionize human existence and set us on a modern path. Was the discovery of fire not akin to a superpower of the day?

Likewise, the internal combustion engine has done so much to change the world (good and bad) while giving us super speed. Yet we take it for granted.

To be a superpower, the mechanism doesn’t have to be mutagenic or metaphysical.

It simply has to empower us to overcome some physical or mental limit that we couldn’t before. 

So what is facebook’s superpower? It's essentially giving us the power to stay in touch with the friends and family, which is a limited form of omnipresence. But, as it happens, I know quite a few people who might say lurking or invisibility is the main draw for them. And for others, it’s the opposite, a place to share and feel valued. Facebook's key shortcoming is that while it's all about people, it generally only shows the artifacts of people, like shadows dancing on the cave wall. To succeed in AR/VR/MR, facebook needs to re-add people as directly and completely as possible.

Claiming massive potential for AR/VR/MR might not seem very dangerous today, but I started on this path 25 years ago, with my first hapless VR startup. I was literally living in my office due to lack of salary, and yet dreaming up the likes of AR contact lenses, teleportation and practical invisibility cloaks. This stuff was so science fiction back then that Neal Stephenson actually hung out in our office to observe.

Perhaps coincidentally, my next big “aha” moment came in 1999, when I helped turn a 3D "earth" demo into a product that gives people the superpower to fly anywhere ?—?and more transformationally to help them understand the 3D world and add context. That became Google Earth, with billions of unique downloads, thanks to Google.

But I got an even bigger chance to put these ideas into action when I was at Microsoft, late January of 2010. Our Xbox bosses needed something really game-changing for the next 10 year cycle. I had a few ideas.

I’d previously prototyped one of those, as a solution to making mutual eye contact in video conferencing. I'd called it Holographic Telepresence in honor of Star Wars.

Fast forward past a bunch of houseocardian drama, and the incarnation today is the amazing HoloLens, coming soon to a face near you. Of course, that’s mainly due to the hard work and invention by thousands of people over six years, with probably billions of dollars spent. Kudos to everyone who worked on it.

The result of this Telepresence will be the elimination of space and time (and obvious devices) as barriers to communication. For years, we’ve been promised that teleconferencing would remove the need to travel to work or fly around the globe. But it’s never been the same as being there.

Remember that idea I mentioned about making eye contact? Most humans have a natural superpower to read other people’s emotions right off their faces -- especially the eyes. Without that, we’re closer to robots.

So what if virtual communication was finally as good as being there? Enter the era of “beaming in” or “ghosting” as the most ubiquitous of all of our new superpowers.

But telepresence is just one aspect. There were twelve or so superpowers in my 2013 talk, extruded from twelve very well-known game-changers. Here’s a slide:

Spatial Entertainment is already here in primitive form, as “3D projection mapping” on big buildings at night. Extend that idea a bit and we see the natural world transformed anywhere, anytime, into anything we can imagine. We can imagine music as imagery, synaesthetically transforming our world around our auditory activities. The “money” scenario in 2010 was imagining the movie “Avatar” crawling out of your TV screen, transforming your living room into Pandora. It’s all the more magical if our families can each see that world each from their own perspective, and even view it through their own filters.

Of course, we won’t all want or need to see the same things at the same times out in the world. There will be a subscription model to this reality, especially outdoors, and hopefully a parameter denoting how much we trust any of these content providers to replace or augment what we see.

But, for what it’s worth, one dirty little secret of Mixed Reality is that less is more, as in removing distractions is generally more useful than adding them. It’s all about managing information overload and improving our cognitive ability to process and understand what matters. Which leads us to…

Perceptual Intelligence is probably the most useful and misunderstood benefit of MR. “Terminator Vision” was an early entry to the popular zeitgeist, providing us a short list of pithy things to say in any psychopathic situation. But that depicted way too much information. And why did a computer even need to read its own AR screen?

Indeed the more appropriate technology is already being applied towards helping soldiers deal with the chaos of war, where it’s difficult to know friend from foe or see the threat around the corner. We can overlay simpler icons on the world to help soldiers grok this information quickly. This will save lives.

For some of us, that’s only a shade more intense than visiting the average shopping mall. Being able to spot a magic glow around a product we might like or dislike, or sense when a friend is nearby and available, is a new kind of superpower we should all expect to help us improve our experience of the real world. Subtlety is king. Putting labels or key statistics over everyone’s head is arguably insane.

Ground Truth has been with us for a decade or more in the form of Google Earth, Waze and the like. These companies spend big money to capture a reliable copy of the world, or crowd-source it, and then reproduce it in a form suitable for applications to, well, apply. Those apps are a simple form of augmented reality: social, local and mobile. The “mirror world” they require is made possible by the extent to which we can conflate and connect all of that raw data. The superpower here is being to tap into this layer of information for any purpose, and even add to it.

Creative Projection is gaining a lot of traction lately, with an unleashing of 20 years of pent up energy around collaborative AR/VR content creation. I love this space, and it’s not hard to understand the appeal of creating things using our bodies directly, with an interface that virtually disappears. This superpower maximizes the bandwidth from our internal creative factories, where we can easily dream up anything, moving towards a new medium that we can share widely and even print on demand. This behaves much like fire, releasing energy of a more creative sort.

Lifestreaming & Lifesurfing applications are also popping up more and more in the form of Periscope, Instagram and Twitter. But those portray reality in a very narrow sense. AR/VR will do it better. In these scenarios, some attractive portion of the population transmits aspects of their lives for popular attention. Some desiring portion of the population surfs these streams to wittingly provide that attention and perhaps some appreciation.

The combination of living one’s life and interactively consuming the lives of others can be considered a form of bi-location. We can imagine a future in which we inhabit not one body at a time but dozens, even hundreds, popping in and out of awareness and participation. To put that in more concrete terms, imagine your trusted friends or experts showing up throughout your day as needed to help you make decisions, where you might do the same for them.

A more explicit kind of bi-location comes when we can use holographic telepresence again to experience another place while still being present in the here and now. If that sounds impossible for people to do, just watch a seasoned Starcraft player for a few minutes.

Of all of these new superpowers, Zeitgeist is the one that’s hardest to imagine because it’s most different from our current experience. First, imagine what happens when everyone is transmitting some aspect of their lives?—?including at some level their emotions or needs (with suitable permissions)?—?where others can consume those aggregated feeds. Then imagine a kind of x-ray vision for the world, from interpersonal to global scopes, where we can almost literally see where people are having fun right now, or where people are lonely or sad, or in need. This superpower forms the technological equivalent of empathy, potentially on a global scale. It could well be the most transformative of the lot, we shall see.

Short of that, crowd-solving is a much more direct action application of crowds to solving particular tasks. Think: flash mobs with a purpose, coordinated via mixed reality. We see this online with crowd-sourcing already, but need only wait a moment for it to come into the physical world in force. The “on-demand economy” is perhaps the first and easiest layer of this nut to crack, with tasks managed more linearly and for direct remuneration.

So there you have twelve real-world superpowers, coming soon.

But what about really flying, immortality, invisibility, and the like? Aren’t those too hard for AR?

Those are all possible at the more virtual end of the spectrum, at least for now. That doesn’t mean they’re lame. Experiencing flying in a virtual world is a very profound, if not practically locomotive, experience.

Some VR experiences do have real-world effects to boot. Psychologists have already applied VR to treating phobias and anxieties, with some clear success. VR allows us to introduce the perceived threats in measured doses, and to repeat the immersion at a moderately comfortable pace, habituating us to something we previously thought impossible.

Given how many people who survive a suicide attempt after jumping off a bridge say they realized their mistake almost instantly, what would happen if they could jump virtually, but have it feel just as real? Could it have the same transformative experience? I am sure we’ll find out, in a kind of simulated immortality.

So what’s your most desired superpower?

What does it say about how you might presently feel constrained by the natural world or real life?

What changes in your life when we reduce or remove that limitation?