2022- the year technology starts coming together

I don’t think I can recall a year that left so many people so universally weary by the end.?

Of course, there were a lot of reasons for this. But for me, one of the biggest was that working and living through 2021 felt a bit like playing chess in a rally car. Every time you tried to decide your next move, the pieces were scattered indiscriminately by a new rule change, border closure or variant. All this, whilst also subconsciously trying to make up for the (perceived) loss of progress from 2020.

They say necessity is the mother of invention - well 2021 was a mother of a year.

Thankfully, there’s always a silver lining and this time, it’s that 2022 is hopefully going to give us the chance to start (relatively) fresh atop a foundation of hard-won innovation, burnished progress and exciting potential.

What’s ahead for immersive & real time tech?

2022 feels like the start of a new phase; one where my separate strands of interest, Virtual Production, Augmented Reality, Virtual Reality and interconnected spatial experiences, start to find some common ground.

If, say, we’re all building a skyscraper together, and the past 10 years was spent working on the materials needed to build it; now we’re finally heading on site to start putting it all together.

And just like a real building site, once you’re out of the ground, things tend to accelerate.

So my bet is that 2022 will be the year of breaking down silos and working with industries and experts you haven’t before, on products and experiences that transcend existing platforms and formats.

This merging and mixing of technologies and skill sets will have a groundbreaking effect on how we all create and consume content, whether linear video, multiplayer gaming or the vast spectrum of things we experience that now live in between.

This isn’t just a story about our leisure time either. Workplaces are going to be equally impacted by the blurring of content types and realities. If anything they will probably be at the vanguard, because most workplaces have seen the dramatic need for revolution in this space over the past 2 years, and have a radical remix that is hybrid working to use as inspiration for change.

So to mark this moment, I wanted to spend a little time looking at some specific examples of how technology projects and platforms will overlap more this year.?

But before we jump into that; all innovation is equal parts vision, hard work, perseverance, and luck. And those of us working today are all extremely lucky that the foundational pieces needed to create the future were first spun up by industry veterans decades ago. If not for their vision and fortitude developing engines, platforms, capabilities and standards well ahead of their time, we would not have all the exciting tools we have access to today. But more than their products, many have also freely shared their expertise to ensure these spaces move ahead more quickly than would otherwise be possible. So many thanks to those that have worked to create a more welcoming, open industry.

Merging Technologies, Content, Industries

To say that engines have been eating the world is an understatement. Versatile engine devs will be close to the hottest commodity this year and remote hiring will see them The number of industries now using them regularly is exploding and it would be easy to think that it must be because they are becoming more capable and accessible. While that is absolutely true, I think the rise in use is primarily because they are the best communication tool we currently have. There is no better way to communicate than to show the reality of what you are trying to create. And that’s what engines do best. They allow you to create a living, breathing world not constrained by any physical or financial limitations, that you can walk around in, film in, design in, test in, meet others in and collaboratively work in. It’s no coincidence that the real estate and architecture businesses are diving in head first. Showing someone the reality of what you are trying to create is a lot more powerful than telling them about it.

But the effect engines are having goes further than that too. By bringing everything into the one place, and allowing it to come out the other end on a number of different platforms or formats, it creates cost effective content crossover opportunities that will change how we think about content in general. Real time engines are the crucible that will lead to all sorts of new alloys.

Let’s walk our way through a rough example of what’s possible (and even happening in small ways) now.?

Say you have a story to tell - a story that is set in a mostly abandoned, alternate reality Las Vegas. This world is a mirror image of ours with stark visual differences; extensive decay and urban rewilding that all hint at the fact that something went terribly wrong in this version of our past.?

Audiences of the near future won’t see this story in terms of one platform or format. They will see it as a universe that they can and should be able to experience in different ways for different purposes. Marvel has taught us that revisiting a cohesive, interconnected universe is a much more sticky experience than a one-off visit to a far away land. Consumer brands are following this path, too. And streaming platforms like Netflix, Amazon and Apple are tipping their hand to this direction with their recent push into tie-in games. Extending existing IP and providing ever-sticker experiences that are worth returning to again and again is the holy grail of content creation and subscription models today.

So what does that mean for production of our alt-reality Las Vegas??

Well let’s start by dragging a 3D model of the current city into Unreal - a 2cm detail scan from Aerometrex - and visually remixing it to suit your story.? Nanite eats it up and allows you digitally walk the streets of this world in real time, deciding which locations you want to use as the settings for your story. Or you can jump into VR to move around freely, building and modifying elements in collaboration with a developer so that the spaces within this world feel exactly how you imagined. From there, you begin adding life layers - sort of. You throw in digital extras - Metahumans - that look and behave exactly as you want because you recorded the facial expressions and movements you want them to use. Or you could use volumetric capture if variations aren’t necessary, to get a different look and feel - 3D reality captured in full motion.

Now you’re left with a living, breathing world that can become the background environment for your introductory film or TV series, using Virtual Production to place real actors within these virtual worlds. The environments go to set to begin the shoot, using VR to block out the action first, testing lenses and angles digitally before anyone else is even added to the call sheet. An iPad with LiDAR-enabled AR is used to place modelled set pieces within the LED volume you’re going to be shooting against, to ensure everything will fit and mesh perfectly when your limited physical sets are eventually built and cameras begin to roll. The real time environments and tracked camera position, lens, zoom, focus and aperture allow for a creatively free shoot, increasing on set collaboration and those ‘happy accidents’. Most (but not all) of the shoot happens in this one volume, with lighting and the necessary atmospheric conditions dialed in quickly to allow for an efficient production without logistic problems or weather delays. And thankfully during Omicron times, set numbers can be limited, along with the corresponding exposure risk.

From the big screen to every other screen

Now as we said earlier, let’s say people want to explore the world you’ve created in more detail, not just be led around within your film on a flat screen. Your initial Unreal environment, (if considered from the start) could also be used to create a multiplayer experience that extends the initial series narrative, allowing audiences from all around the world to explore this reality freely while creating their own story within it. With some albeit significant effort, VR users could be joined by console, computer and mobile users wandering around this space in real time, discovering elements and completing missions. This literal and figurative metaverse could continue to extend the story and the world you initially mapped out, while also updating season after season to keep players guessing at what might happen next. Some content could even be user generated, scaling your world far beyond what you can create yourself.

But this world doesn’t just need to live on flat screens and in VR headsets either. Again using the base assets created earlier, a geo-located AR experience could also be created that anchors this world over the real Las Vegas - showing players ever changing portals or rifts in our dimension and giving a glimpse at what is happening live in this desperate mirror world. Or by using Niantic’s impressive new Lightship ARDK, this treatment could even be expanded to create a global alternative reality to explore and inhabit. In the future we might even be able to make these different immersive experiences interconnected, so that players and changes made to one could be seen in the other.

So that gives us one story told three different ways, using the same base environment as the foundation for each thanks to the flexibility of real time engines today.

As an aside, I also think the same theory could apply just as effectively to hybrid workplace collaboration in the coming years. Right now we’re all grappling with very structured remote tools that don’t give us a lot of dynamism or nuance in how we communicate with others at work. We can message, email or book in a VC; not a lot of options there.?

But imagine a digital twin of your office that you can visit and explore on any device; that recreates those spontaneous interactions that lead to better relationships with colleagues, new ideas, real innovation and deeper awareness. Digital offices designed in engine to mimic our real world spaces, plus future AR/VR capable wearables would combine to deliver a lower friction workstation that is portable and effective at work or home, that also allow for all those natural spatial interactions that increase agency and engagement - hand gestures, body language, free-wheeling discussions, leaving notes and diagrams for others via persistence, wandering into a spatial conversation that just so happens to be really relevant to you!

Whether at work or at play, it feels like 2022 is the year that many of these threads will start coming together in a way that justifies deeper investment in all of them. I’ve probably said that before.. But being positive isn’t a bad place to start; either way, it’s an exciting time to work in a rapidly changing space.