Spatial Computing - Ultimate Format of Metaverse [Primer]

With the convergence of physical and digital worlds, the metaverse is indeed pushing the boundaries of our digital capabilities. Everything from human identity, personality and reputation to assets, emotions and history can be interacted with, controlled and experienced in the virtual reality (VR) of the metaverse in entirely novel ways.

What interests me the most in the Metaverse scene, is the possibility to link up virtual experiences and assets to in-real-life utilities, stories and amazes. The limited possibilities empowered by "Spatial Computing" has intrigued me as the most inspiring and future-proof scenario - the human brain has evolved to deal?with?a?three-dimensional?physical?environment,?not 2D screens, and spatial computing taps?into this deep, embodied knowledge.?

For decades, technologists and futurists have dreamed of having the means to unite the digital and physical worlds, to enable a well-balanced interaction between humans, machines, and their environment. We’re inching toward making this type of computing – spatial computing – a reality. Spatial computing is a new perspective at the world around us, driven by advancement in computer vision, machine learning, and IoT. It enables the seamless interactions of humans, machines, and spaces to optimize processes, and foster real-time collaboration in the future of work.?

This series of articles is intended to explore and share my learnings and insights on this topic.

What is Spatial Computing?

Spatial computing is the digitization of activities of machines, people, objects, and the environments in which they take place to enable and optimize actions and interactions.Spatial computing is broadly synonymous with extended reality (XR) - an umbrella term for VR, augmented reality (AR), and mixed reality (MR). It is the practice of using physical space as a computer interface, in a way that machines no longer need to be tied to a fixed location.

The term spatial computing was defined by MIT Media Lab alumni Simon Greenwold in his very futuristic thesis in 2003. It is only until recently that we are able to make this thesis and vision possible, thanks to advances in these technologies: artificial intelligence (AI), camera sensors, and computer vision that track environments, humans and objects, Internet of Things (IoT) that monitors and controls products and assets, and augmented reality (AR) that provides the human user interface.

Spatial computing is the next step in the ongoing convergence of the physical and digital worlds as it augments our reality but also understands the space and therefore allows the content project to interact with the surroundings. Spatial Computing has benefited from the recent fast-pasted developments of computer-generated 3D environments of VR with visual enhancements and information that add value to the individual (AR).

Spatial computing does everything VR and AR apps do, then it combines these capabilities with high-fidelity spatial mapping to enable a computer to track and control the movements and interactions of objects as a person navigates through the digital or physical world. Spatial computing will soon bring human-machine and machine-machine interactions to new levels of efficiency in many walks of life, like transportation, health care, and the home. Major companies, including Microsoft and Amazon, are heavily investing in the technology.

Empowering humans through spatial capabilities

Billions of people around the globe use various applications of spatial computing daily—by using a ride-sharing app, GPS, social media check-ins, Pokémon Go. Scientists and researchers use spatial computing to track diseases, map the bottom of the oceans, chart the behavior of endangered species, and create election maps in real time. Drones and driverless cars use a variety of spatial computing technologies. Spatial computing works by understanding the physical world, knowing and communicating our relation to places in that world, and navigating through those places. It has changed our lives and infrastructures profoundly, marking a significant shift in how we make our way in the world.?

At its highest level, spatial computing is the virtualization of activities and interactions between machines, people, objects, and the environments in which they take place to enable and optimize actions and interactions.?

Spatial computing adds knowledge of relative location, for instance location with respect to other locations, to expand the concept of ‘traditional computing’. For example, an autonomous vehicle utilizes GPS, LiDAR, volumetric camera sensors and other technologies to triangulate its precise location and measure its proximity to objects in the driving environment.

The prevalence of automation, as well as machines working alongside humans, is increasing globally. Spatial computing will unlock synchronized operations between these humans and machines, as they work side by side. It is the ultimate way to optimize entire worksites occupied by humans and machines, including coordinating the work of every machine and worker involved in the process.?

Spatial computing enables coequal collaboration between humans and machines, but it also enhances each individually.

Augmented Reality at Architectural Scale

At architectural scale, AR moves into the territory of architects: rooms, buildings, neighbourhoods, cities, and all the things within those spaces. At this level, an augmented reality user interface serves multiple users, applications, and objects within multiple spaces.?

Spatial computing is the means to better understand how humans, machines, products (and more) move and relate to each other within a space.

At architectural scale, there is great opportunity for spatial computing to deliver more powerful insights. As is true of VR and AR, spatial computing builds on the “digital twin” concept familiar from Computer-Aided Design (CAD). In CAD, engineers create a digital representation of an object. This twin can be used in a variety of ways be it to 3D-print the object, design new versions of it, provide virtual training on it or join it with other digital objects to create virtual worlds.

Spatial computing makes digital twins not just of objects but also of people and locations—using GPS, radars, video, and other geolocation technologies to create a digital map of a room, a building, or a city. Software algorithms integrate this digital map with sensor data and digital representations of objects and people to create a digital world that can be observed, quantified, and manipulated.

In other words, spatial computing provides the tools for producing a digital twin of virtually every process, where the user can easily revisit and experiment any time with VR to modify the process as needed in the shortest time possible—and it will be more so when 5G networks become vastly available, because of the major improvement in latency, capacity, bandwidth, etc.

Spatial computing is reinventing how tech work regarding space

Think about the way you currently navigate a space that you’ve never been to before. There are foundational elements of the space, such as walls and doors, but also different types of markers to assist in moving around a space safely and efficiently, like visual printed signage or electric signals.?

One can think about these markers on a continuum from monumental static items like the wall, to changeable yet static items, like signage, to traffic lights, which change to convey information. These markers are stationary and don’t allow for much flexibility or fully represent the changing nature of a given space – that’s where spatial computing and augmented reality as a user interface in an architecture scale comes in.?

Technicians servicing oil rigs and refineries can now easily locate devices or components, operational instructions, equipment data, and IoT information using augmented reality. It's part of a growing business case for augmented reality, which is still struggling for a foothold in many industries.

The user interface for spatial computing will be completely different from the way most of us interact with computers today (i.e., via type, touch, and screen). For example, it will entail eye-controlled interactions, body or hand gestures, and voice controls– hardware will be invisible. The concept of fixed computers and staring at a flat screen will be looked back at by our descendants as absurd, just as we look back on the use of floppy disks to share files today.

A renaissance of interest in spatial computing and the merging of physical and digital personas, identities, and spaces, is now further propelled by several large tech companies boldly establishing their visions and claims for the Metaverse.?

Big tech, from Facebook to Google and Apple, are moving towards spatial computing and XR devices.

"Spatial Computing", the Ultimate Format of Metaverse?

The adoption of spatial computing in manufacturing?will happen much quicker than you might think.?

By 2030, the cutting-edge applications?of spatial computing now being adopted today will be components of much broader spatial computing initiatives.

With a shared understanding of the space and happenings around them, the actions of people and machines will be harmonious. Operators will use gesture control to direct the motion of machines. Machines will adjust their actions to account for the movements of people and products. Each will be aware of the goals and intent of the other through a shared perception of digital and spatial information.

In this vision, it’s not just the operations that are optimized, but the environment as well. For the first time, industrial engineers will have a complete and persistent view of the interaction of materials, people, and processes. This comprehensive knowledge will provide continuous feedback for the improvement of factory or warehouse layouts to resolve inefficiencies of time, motion, and the use of space. With spatial computing, each industrial environment will be as engineered, monitored, analyzed, and optimized as the processes conducted within it.