How the Holographic office will kill video-conferencing

TL;DR Holographic Computing will give rise to ambient omnipresence for the workplace. Regardless of where you and other members of your distributed team are situated physically you’ll occupy mutual holographic spaces, with all the collaborative affordances of shared concrete rooms. Holographic telepresence will be to video-conferencing what Snapchat is to the fax.

PREFACE: Virtual Reality can be thought of as Augmented Reality with 100% masking. They both exist on a spectrum of spatial digital experiences that will be observed through a single converged headset in years to come. Which is why Microsoft has done the world of tech a great service by eschewing this jargon altogether, with the introduction of the HoloLens headset, in favour of terminology the average person recognizes and comprehends thanks to years of use in popular culture?—?Holograms and Holographic Computing. True,HoloLens is aimed at the Augmented end of the spectrum at launch but there’s no doubt the same headset will cover the whole gamut of experiences by version 3. As will the Oculus Rift, Magic Leap, PlayStation VR, and so on. For that reason this essay sings from the same hymn-sheet as Microsoft. If you wish to argue the definition of Hologram please refer to this Oliver Kreylos essay first. And yes, you will wear a holographic headset in the near enough future, unless you’re the kind of person who gets a personal assistant to do your email. As Benedict Evans wryly notes , “The future has always looked like a toy that can’t be used for real work. So, each new computing platform will never be used for real work, but the platform gets better and the work changes to fit the new platform. In tech, ‘never’ seems to be 5–10 years.”

The Ghosts of Mars

When Microsoft announced HoloLens at the beginning of 2015 one image from the press kit that didn’t grab the attention it deserved was of NASA scientists collaborating remotely (see top). The Jet Propulsion Laboratory (JPL) worked with Microsoft to create OnSight, a tool that uses ‘holographic computing’ to enable scientists around the world to meet in a 3D simulation of the Martian environment, to plan the operation of NASA’s Curiosity Rover. That is, from the vantage point of standing beside it, virtually, on Mars.

In the B-roll video footage it’s clear to see the phantom outline of the remote scientist?—?John?—?watching his local colleague as he moves from a desk to the shared projection of the Martian soil. Not that we can see John’s eyes exactly, but his attention is easily inferred from the motion of his ghostly head while his gaze is emphasized by a dotted line. Of course the demo is likely just a mock-up using post-production effects but it’s nevertheless representative of the most exciting thing in Augmented Reality,… sorry, Holographic Computingthat no-one is talking about; people. Or rather, the spatial representation of remote colleagues in a shared (or merged), 3D digital environment.

Now, here’s where I deviate slightly from the hymn-sheet. The reason is Microsoft would apparently have us refer to all digital overlays seen through their headset as ‘holograms’. But for the sake of clarity in this essay I need another term to distinguish between the projection of non-humanoid and humanoid forms. The problem is with the preconceptions?—?in popular culture holograms are typically depicted as auras, projected from the floor or a wall, that enjoy a limited range of positional movement. Or none at all. But the point of this article is to consider the properties of humanoid forms that have unrestricted and continuous freedom of movement throughout, and indeed through, the environment. So I’ll use the label ‘ghosts’ to disambiguate.Ghosts are humanoid holograms, seen in outline, that move freely through the holographic space.

Why don’t I use ‘avatar’ instead of 'ghost’? Because of its long-term use in computer gaming, and association with stylized characters for fictional narratives, controlled with gamepads. More digital puppets than actualavatars (like the movie). Whereas ‘ghosts’, here, are motion-tracked and body-mapped stand-ins or surrogates (yes, like the movie).

My hope is that thinking about holographic humanoids as ghosts will give you a different perspective on the possibilities for holographic computing, beyond what is normally portrayed in fiction. That is, with the honorable exception ofDennō Coil, a truly visionary science fiction anime “depicting a near future where semi-immersive augmented reality (AR) technology has just begun to enter the mainstream”. In that TV series the boundary between the real and the virtual world was porous to such a disconcerting degree that it took viewers quite a time to figure out which domain a creature belonged to. The beings populating the world of Dennō Coil were ghost-like in their movement and passage between realms. They existed everywhere and nowhere. (Please do watch this video to better grasp the rest of this article).

Putting the presence into Telepresence

So, what about an OnSight for the rest of us? A workplace that works more like Dennō Coil than Cubicle Farm. In contrast to separation by partition, can the holographic office convince us that colleagues are alongside us, at all times, regardless of where they actually are, physically? What would that workspace look like and why might we want it? To answer that, lets start by looking at the changing nature of the labour market, as Loic Le Meur sees it —

“The Millenials want flexible work hours, they expect to be mobile and work from home/office/cafes at will, from anywhere really. They often work outside of normal business hours and prefer to freelance, work on flexible hours and collaborate online. 3 billion new minds are coming online within the next 10 years, many will learn and work exclusively online and never be a full-time employee of a corporation.”

[ASIDE: And speaking of millennials?—?in an article entitled ‘Why remote work isn’t just for millennials’, Susan Tenby lists Second Life among her favourite tools for remote workers?—?“for synchronous meetings with lots of people”. If you never understood the appeal of that virtual world platform (virtual reality without the goggles), Susan’s liking for it should make a lot more sense by the end of this article.]

In the words of Stephane Kasriel, CEO of Upwork, “You can totally see the emerging trend of people who don’t want a full-time job, period. And the best people really don’t, because they have a choice.” That begs the question?—?and it’s not a new one, I know?—?will work cease to be defined by an office? Or even by a place? Fast Company has more figures on the rise of the freelancer economy

“As of May 2015, 15.5 million people in the U.S. were self-employed, according to the Bureau of Labor Statistics?—?an increase of roughly 1 million since May 2014. That number is expected to keep growing at a steady clip. By 2020, a separate study estimates that more than 40% of the American workforce, or 60 million people, will be independent workers?—?freelancers, contractors, and temporary employees.”

The corollary of that being that —

“It’s not uncommon for independent workers to feel isolated. But the rise of co-working spaces in top urban centers is changing that, offering freelancers unprecedented support and resources. Co-working spaces are providing more than just a sense of community that comes from working around others. WeWork, for instance… has raised the bar in part by focusing on creating a collaborative ambiance you’d find at any cutting-edge startup.”

There’s an apparent paradox at play here. People want the freedom of working independently but don’t want to lose the commonwealth of togetherness. Co-working seems like the cure for now but it’s no panacea. Just a stopgap. While many independents miss the social ambiance of the office, few miss the many downsides of attending shared spaces that offset the benefits of professional freedom. Commuting for one! Distractions for another. And on and on. As Micah Baldwin succinctly puts it?—?“Spending some time in close proximity to other people, but not having to interact with them is probably sufficient for most freelancers”. How about having the best of both worlds? What if you could be among colleagues without being with them? You know, like a ghost.

The crux of the problem is that the real world is too binary. You’re either here or there. In a single place at a time. It’s black and white. While in contrast the digital world is… well, more analog. It’s many shades of grey. You can be ‘in’ a place to varying degrees and in multiple places at once. Yes, like a ghost.

However, it’s true to say that the boundaries of real world communication are more permeable. We move in and out of earshot, and view, all the time. Conversations change continuously according to who and how many are within vocal range. And often fork into (multiple) overlapping dialogues as groups expand. It’s organic and analog. Voices fade in and out as a function of physical movement, projection of speech both in volume and direction, and the shape, placement and material properties of objects in the space.

These are valuable properties and nuances of communication that we’ve failed to replicate in any meaningful way online. We can’t simply collapse the multi faceted communications fabric of meatspace into the narrow thread of cyberspace. Or the 2D portal of a video call.

We all know that video-conferencing has failed. Miserably. I long thought the failure was just a matter of time; time for Moore’s Law to work its magic. If the sound was surround enough, the video big enough, the lighting bright enough and the cameras… well, if they could somehow bend light to convey actual eye contact, then we’d surely feel as if we were in the same room, together. Surely, right? But then I got to actually experience a Cisco Telepresence system, with the very best of everything and… it was thoroughly disappointing. I still felt absolutely separated from the people at the other side of the call. The other side… I didn’t think about that until I wrote it down just now but it neatly highlights the real issue. Video-conferencing will always have a here and a there. Instead of just a here, as it needs to be.

And I’m far from alone in my disappointment with ‘videophones’. Paul Watson, former CTO at Storyful, laments -

“Video conferencing is woeful. Before you even get the call connected is painful, then the lag, drop outs, echoes, hiss, volume, looking at a ceiling, etc. 2D video is not going to make people feel like they are here… I reckon Microsoft’s Hololens is the way, until we have neural connections. Ditch physical screens all together.”

Paul identifies the key problem with video?—?regardless of how good it is it’ll never deliver a sense of social presence or co-presence. Photo-realism is not what we crave in ambient communication with distant colleagues. Spatial realism is. And the corporeal form, even in outline, with the unmistakable fluidity of human body language. Our animal brain is hardwired to sense when another creature is in ‘our’ space. And a flat reflection in the watering hole doesn’t fool us.

But what does fool us is Virtual Reality… sorry, Holographic Computing. Chief Scientist at Oculus, Michael Abrash explains how —

[Virtual Reality creates] a powerful sense of what’s known as presence… the sense of being someplace else while in virtual reality; many people feel as if they’ve been teleported. Presence is distinct from immersion, which merely means that you feel surrounded by the image of the virtual world; presence means that you feel like you’re in the virtual world. Presence is an incredibly powerful sensation, and it’s unique to VR. There’s no way to create it in any other medium. Most people find it to be kind of magical, and we think that once people have experienced presence, they’ll want it badly.”

So, while high-end video-conferencing may eventually deliver a sense of immersion it’ll never evoke a true sense of presence. That is to say, the photorealism of other people in video doesn’t compare even to matchstick men in VR. Not convinced? Take a moment to observe the giddy reaction to the social presence offered by Oculus’ Toybox demo, even with the most basic,ghostly avatars —

 

Then listen to the Voices of VR podcast, to hear the wild delight of others who tried the same demo. As host Kent Bye emphasizes -

“It was a watershed moment for so many developers to be able to experience social and emotional presence with another person within virtual reality. It became less about the technology and tech specs, and more about the experience of playing, having fun, and connecting to another human in ways that were never possible before. This Toybox demo felt like a real turning point and “Aha!” moment for a lot of VR developers to see how compelling social experiences in VR are going to be.”

Paul Watson’s comments on video-conferencing were particularly interesting to me because I knew he had previously explored ways to bring ambient (always-on, background) presence to his distributed team. One solution he investigated was Sqwiggle?—?“a tool for instant tap on the shoulder remote discussions”. Or “an always-on online workplace for your remote team to work together throughout the day and feel more connected”. But we understand now why products like Sqwiggle will never quite achieve that ambition while relying on video.

Furthermore webcams are unsuited to ambient presence because we have an instinctive unease at being spied upon, essentially through a peep hole. It’s a natural lizard brain reaction. Cameras are often misaligned, switched off and inconsistent placed so you can never quite be sure that someone isn’t actually staring at you. And that’s uncomfortable. Your only guide is a grid of postage stamp faces on a flat screen that you can’t really monitor in your subconscious.

In normal life we usually know when we’re being stared at (e.g. on public transport)?—?we’re hard-wired to ‘feel’ it because we have full spatial awareness of the orientation of people’s heads, what angle they’re held at, what direction their eyes are facing in. We can scan a room full of people with a dart of our eyes and immediately detect who’s looking in our direction. Video-conferencing doesn’t guarantee the conveyance of such clues. So the strong possibility that people are actually looking at you while it doesn’tappear they’re looking feels creepy. It feels off.

In addition there’s the nagging fear of forgetting that someone can see and hear you. When we don’t sense the physical presence of people, even in our periphery, we forget that microphones and webcams are switched on. That wont so easily happen when we have holographic ghosts around us. Even when they’re positioned behind us the spatial audio from a holographic headset reminds us with the sound of breathing, rustling and so on.

Holographic ghosts may be intangible but will take us across a cognitive threshold that video never can, and finally put the presence into telepresence.

Taking the place out of Workplace

Morgan Lovell has beautifully charted the ‘Evolution of Office Design’ through the emergence of open plan, the action office, the cubicle farm, hot-desking and the casual office. The future? Activity-based Working apparently -

“ Dynamic, activity-based working spaces to suit different working styles are key to moving away from Open Plan tedium. Contemporary, responsive spaces that remain open, but separate, allow for collaboration, inspiration, mobility, and the completion of specialist projects?—?without the worry of crowding or disruptions from one working style to another. Workers are presented with a range of logistically different workspaces to choose from to best suit their individual needs at the time?—?including their own desk, wide open meeting rooms with whiteboards, IT suites, or informal spaces with coffee and snacks. Agile working environments are truly the future. Allowing staff to work effectively in different environments within the same space is key to both productivity and workplace wellbeing.”

But notice the many contradictions therein. How can a solid building be truly dynamic? How can it sensibly remain open, but separate? Why strive for so many different environments for a work force that’s going increasingly distributed and independent? In particular, for a co-working facility what sense does it make to offer a configuration for every occasion when most professional relationships and collaborations are across the boundaries of the building rather than within? Morgan Lovell argues further that —

“This emerging shift towards adaptable workspaces to accommodate the most bizarre of ad-hoc working practices not only saves space and money, but also allows for employees to have the best of both worlds: collaboration and communication, and peace, quiet and privacy when they need it. Modern office spaces need to allow for a range of different numbers of people, from one-man phone booths, perfect for personal calls and personal work, to huddle rooms for small groups, loud rooms for informal, creative, collaborative work and quiet rooms for activities of a more reserved nature. This desire for collaborative, yet adaptive workspace has permeated modern offices significantly through the introduction of “third spaces,” or “in-between spaces;” work areas with no rigid purpose; but the ability to adapt to multiple styles of working on different project types.”

The fallacy is the notion that a physical building can ever be the container for a truly dynamic, adaptable and responsive work-space. Regardless of how clever the design or modular the furnishings, a fit-out configuration will remain largely static. A case of the tail wagging the dog, for in truth its the employee who must be adaptive and responsive by moving from place to place, throughout the day or week.

How ironic it is that we’ve digitized and subsumed a myriad of cubicle paraphernalia into applications on the Personal Computer and now endlessly re-arrange them for convenient access on virtual desktops, but the Activity-based Workplace would have us dragged around it’s floors like the icons on a background wallpaper. Morgan Lovell says, “The employee has become the centre of the office design blueprint”. But clearly not! Not until the office revolves around the employee rather than vice-versa.

 

In the era of Windows computing it made sense to stop at the edges of a ‘desktop’ for the UI metaphor. But in the age of Holographic computing there is no border. When Harvard Innovation Lab updates their ‘Evolution of the Desk’ video in 2020 it will end, I expect, with the desk and walls being sucked onto the screen, and the screen morphing into an holographic headset.Desktops? Where we’re going we don’t need desktops!

I’ve no idea what name we’ll put on this?—?the UI metaphor for Holographic Computing. But if ‘Desktop’ was apt for a world of flat screens and surfaces, then ‘Bubble’ may be a good candidate. Think of a Bubble as a 3D halo. Like a dome that surrounds you and your ghostly colleagues, pursuant to your group and privacy settings (like Google Circles in glorious 3D!). It inflates and deflates in accordance with the variable size of the group (as colleagues join and leave) and the nature of the relationships, in order to maintain a comfortable personal space as well as room for virtual objects, such as whiteboards and tables.

People can be in multiple Bubbles concurrently to be present with different colleagues and teams. So a holographic office is an effervescent foam of overlapping bubbles, inflating, deflating and popping as ghosts appear and disappear, through the walls. This is the Activity-based Workplace where the‘place’ is truly dynamic, adaptive and responsive. Achieved by getting rid ofplace!

Mike Alger is a VR Designer who received a lot of attention for his video presentation on VR Interface Design (below). It visualizes what a work environment might look like in VR mode at 12m30s and in AR mode at 14m02s. Mike wasn’t thinking about holographic bubbles but this still makes a useful reference point —

 The proposal is based on what’s doable with today’s technology but of course, as I’ve said, these different modes of operation will be achieved on a single headset with an unrestricted Field of View (FOV) in time to come. Unfortunately Mike’s video doesn’t visualize people, or teammates, in the office space but imagine the ghosts of Oculus Toybox projected therein. That image is very close to the ambient co-presence promised by Holographic Computing.

In that future the distinction between here and there starts to fade away. Your colleague is both a thousand miles away and right next to you. Your team is in a dozen countries but all in the same place, or bubble. In fact ‘place’ has lost its meaning. There’s no longer a here and a there, only a here. Distance is dead.

It’s a ghost.

Grooving with ghosts?—?a working scenario

Sarah, in Dublin, waits until her husband has left the apartment before donning her holographic headset to start the work day. As she walks to the kitchen and sits at the table with her coffee there’s an audible beep that lets her know she’s entered one of her preset demarcated zones, a sphere of diameter 8 feet that’s mapped to a shared virtual bubble in the holographic foam associated with her team.

The kitchen table is rectangular in reality but the virtual table is round and quite a bit larger than the solid surface on which her coffee rests. There’s no fear of spilling the drink because her headset has scanned the physical objects in this zone?—?the chair, table and cutlery?—?and mapped them onto corresponding virtual items in the holographic bubble, calibrated to the level of the table. Sarah chose an 8 foot zone when setting up the system, in order to share enough of her environment for meaningful replication but not so much as to clone the kitchen counter top and appliances. She’s careful about her family privacy and what parts of her home she transmits to her colleagues.

Sarah sees one of her colleagues seated opposite her, or rather his ghost. Rohit is physically seated at his dining room table, at home in Hyderabad, and sees Sarah’s ghost through his own holographic visor. They exchange greetings and talk excitedly about the international cricket match that India won yesterday!

Camila is in the office in Austin (yes, the company still has an actual building), sipping her tea. As she walks towards the kitchen counter she can see through her headset that she’s about to pass through a distortion in front of her that looks like the colourful film of a soap bubble. As she does so her ghost materializes to Sarah and Rohit like a Romulan warbird decloaking. They nod their phantom heads to acknowledge her and finish the cricket conversation. The virtual table expands slightly in diameter to accommodate Camila?—?it’s still important to observe personal space in virtual reality. Accordingly, the shared Bubble around them inflates slightly too.

The three colleagues are having a pleasant conversation when Sarah hears the doorbell. As she stands and moves towards the hallway she exits the demarcated zone in her kitchen, so Camilia and Rohit see her ghost walk through the shimmering wall, disappearing from their view and earshot. Sarah signs for the package and walks to the study where her second holographic zone is designated. The spherical volume here is larger, at 12 feet diameter, and currently mapped to the shared ‘open plan’ office space, corresponding to the largest bubble in the foam. The ghosts of Sarah’s colleagues are arranged around her in a pattern that you’d typically see in a physical open plan office. Although spacing is wider because each person has a full 360 x 360 degree of virtual monitor space.

At 11am Sarah has a meeting. Without moving a muscle she watches as the 14 ghosts around her vanish and 4 different ghosts pop into the bubble. This is a different bubble but no-one has moved physically. She’s now sitting at what looks like a boardroom table to which the surface of her study desk has been perfectly mapped. Greetings are exchanged and Sarah jumps to her feet, taking 4 steps towards the wall of her study and grabbing the whiteboard pen. As she does so a clone of the writing surface materializes in the holographic bubble so her colleagues can follow along as she takes notes. The sensors on her headset are constantly scanning to ensure that everything she writes is transmitted to them with high precision.

It’s a tough meeting and Sarah is drained by the time it finishes at 1.30pm. She removes her headset and her ghost disappears from her colleagues’ view. She walks to the kitchen, makes a soup and sits at the same stool she did in the morning. But because her headset is off there’s no beep and no hologram. And no fear of a ghost emerging through the walls. Sometimes it’s good to be alone.

Postface: As more people get to try Microsoft’s HoloLens we’re seeing greater evidence of how it, and devices like it, will enable the kind of scenario described above. Sam Murley, co-founder of Infinity Leap, got the opportunity to try out the HoloLens and said —

“It is a wonderful and engaging world where our physical and digital spaces become one. What I was most impressed with was the spatial scanning I performed through the device prior to starting the game. As I turned 360 degrees, the HoloLens projected a red wireframe onto the walls and instantly mapped all my physical surroundings smoothly and with complete accuracy. The potentiality of this instant mapping feature is massive to anchor digital content onto physical environments.”

Meanwhile, “the Surreal Vision team is working on early-stage prototypes thatmodel the world and reconstruct it inside your virtual reality headset.”

“In May Oculus teamed up with [the] computer vision company focused on real-time 3D scene reconstruction. The two companies are trying to build a technology that amounts to live streaming your actual environment into virtual reality.”

And “Facebook wants to build a device that allows you to be anywhere you want, with anyone, regardless of geographic boundaries”.

---

James Corbett is co-founder of Simvirtua, a Limerick, Ireland based start-up building cool things for Virtual Reality… sorry, Hologaphic Computing.