Lost in Automation

How We Struggle to Speak Robot

“You can’t go that way. It’s toward the road!” A passerby, squatting, whispered toward a robot the size of a cat.

The robot had neither ears nor microphones and couldn't control its own direction, even if it could hear and understand the warning. It was simply a piece of cardboard with large black eyes and a smile crafted with a marker. The only electronic components were a motor, batteries and a couple of wheels propelling the cardboard creature.

Named Tweenbot, this robot had a small flag announcing its intended destination. Its creator, Kacie Kinzer, released it in Washington Square Park in New York, expecting locals to crush, kick, or discard it. Instead, whenever the robot got stuck—whether in a pothole, under a bench, or in a blind corner—someone inevitably came to its aid.

Back in 2009, Kinzer's experiment revealed that we subconsciously assume robots can communicate the same way humans do, even when it's clear that won't work.

The Tweenbot didn’t just cross the park. It made it all the way to Manhattan: it’s now part of MoMA’s collection of design works.

Homo ex machina

Almost fifteen years later, we have much bigger bots roaming our cities. They have more wheels and are infinitely more adept at navigating from point A to point B. They can even take humans with them. They’re called robotaxis.

But they still occasionally get stuck, just like Tweenbots. And occasionally, a human tries to help. Now, what used to look really cute back in 2009, has many of us scratching our heads.

Earlier this month, rules changed at an intersection in San Francisco. A new “no left turn” sign was put up. But Cruise’s robotaxis didn’t notice. Maybe they don’t even read signs while driving around and instead rely on information in their maps (scary!). Or maybe they can’t understand what the new sign means (even more scary!). Perhaps there’s yet another reason they ignored the new rule, but the result was a gridlocked intersection with at least three robotaxis stuck there.

And so a human walked up and said, “You can’t go that way!”.

A person wearing a high-vis vest approached one of the stuck driverless cars. They went to the “driver-side” window and proceeded to knock on it. Yes, read these two sentences again if you need to.

This spontaneous act became a metaphor for one of our biggest challenges ahead of us: learning to communicate with autonomous technologies.

We subconsciously hope that—even though we know it’s just an object—a human will emerge from the machine to allow us to communicate with it. We don’t ask for a god-like power emerging from the machine to solve any problems, “deus ex machina”. We hope for a “homo ex machina”.

What’s next?

The field of human-computer interaction has explored these issues for years. Yet, we still don't know how to instruct a self-driving car effectively or how to tell a delivery robot not to drive through a crime scene. And I am still not sure how to tell my autocorrect to stop turning every “Dear” into “Dead” in my emails.

So here's a challenge. Let's develop intuitive ways to interact with autonomous technologies, as straightforward as talking to a cardboard robot or asking a driverless car to roll down its window. Let’s not wait for the next robot to get stuck at an intersection (or an epic autocorrect fail). If you have ideas, prototypes, or even just sketches on a napkin, now's the time to bring them into the daylight. Time is ticking; we can’t afford to go that (wrong) way anymore.

Prof. Marek Kowalkiewicz is a Professor and Chair in Digital Economy at QUT Business School. Listed among the Top 100 Global Thought Leaders in Artificial Intelligence by Thinkers360, Marek has led global innovation teams in Silicon Valley, was a Research Manager of SAP's Machine Learning lab in Singapore, a Global Research Program Lead at SAP Research, as well as a Research Fellow at Microsoft Research Asia. His upcoming book is called "The Economy of Algorithms: Rise of the Digital Minions".