HaptX and partners build first robotic hand to transmit touch across the Atlantic

HaptX Gloves lets human control robotic hand, feel what robot feels

HaptX is proud to announce the development of the world’s first haptic telerobotic system that transmits realistic touch feedback to an operator located anywhere in the world. The breakthrough system is the product of our collaboration with shadow Robot Company and SynTouch. ANA Holdings organized our collaboration as part of the AVATAR X initiative.

Our group demonstrated the system to WIRED, who published an article on the experience. “This is not haptics as you and I are accustomed to,” said Matt Simon, WIRED’s robotics writer. “It truly feels like you’ve reached your arm across the Atlantic, go go gadget style.” 

“It truly feels like you’ve reached your arm across the Atlantic, go go gadget style.” – Matt Simon, WIRED

The addition of touch to telerobotics marks a significant milestone towards the development of robotic avatars. This emerging field has the potential to fuse human dexterity and intelligence with the strength, speed, and robustness of machines.  

“Touch is a cornerstone of the next generation of human-machine interface technologies,” said Jake Rubin, Founder and CEO of HaptX. “We’re honored to be part of a joint engineering effort that is literally extending the reach of humankind.” 

Worldwide touch; a world-class team

It’s fitting that a system that transmits touch around the world was developed by a global team that collaborated in real-time from remote offices. The collaboration includes engineers from HaptX and SynTouch in California, and Shadow Robot Company in London and Madrid.   

Each company is the leader in its respective category: Shadow in anthropomorphic robot hands, SynTouch in tactile sensors, and HaptX in realistic haptic feedback gloves. The combination of the three technologies forms the most advanced telerobotic hand system achieved to date.  

“We’re honored to be part of a joint engineering effort that is literally extending the reach of humankind.” – Jake Rubin, HaptX Founder and CEO

The SynTouch BioTac sensor can sense everything your fingertip can feel including force, vibration, and temperature. It’s the most biomimetic tactile sensor available, and it’s a natural fit with the Shadow Dexterous Hand, the most anthropomorphic hand for robots. The two combined create the most human-like robotic hand available today.

HaptX Gloves provide the most realistic touch sensation ever achieved through a haptic wearable. The gloves apply detailed tactile feedback, powerful force feedback, and sub-millimeter motion tracking. While HaptX Gloves are primarily used for virtual reality, they are a natural fit for teleoperation of robotics.

“Shadow has been exploring what can be done with robot hands for over 20 years. Feeling this come together was a point where we knew that we had made a step change in what is possible,” said Rich Walker, Managing Director of Shadow Robot Company. “This teleoperation system lets humans and robots share their sense of touch across the globe. It’s a step ahead in what can be felt and done remotely.” 

“Hello, world”: A haptic telerobot’s first words

Near the beginning of our collaboration with Shadow and SynTouch, we performed a test that captured our team’s imagination: we typed the phrase “hello, world” from halfway around the globe.

Michael Eichermueller, Director of R&D at HaptX, wore one of the HaptX Gloves in our San Luis Obispo, California lab. Shadow’s Dexterous Hand robot in London mimicked Michael’s hand movements with a high degree of accuracy and little latency. Michael controlled the robot to type on a computer keyboard. As the robotic hand typed, SynTouch BioTac sensors on the robot’s fingers measured the pressure of each keystroke. Haptic information traveled over the network and back to California where tactile actuators in the glove applied corresponding touch feedback to Michael’s fingertips.

The result: even though Michael was 5,442 miles away from the keyboard and robot hand, he could feel the press of each key as he typed. As far as we can tell, that was the first time anyone has ever directed a robotic hand to type from across the Atlantic. We are certain it was the first time anyone’s ever been able to feel what they type through a robot.

Typing provides a useful illustration of why touch is critical to the way we function. When typing on a keyboard, we use sight to guide our hands into the proper position. Once set, we rarely look at our fingers while typing. This ability to type without looking would be impossible without the sense of touch. If we relied on vision alone, we would need to watch each of our fingers to peck out one letter at a time. Without touch, typing would be a tedious task. 

“This is the first time anyone has ever demonstrated a telerobot with such high-fidelity haptics and control,” – Dr. Jeremy Fishel, SynTouch Co-Founder

“We know from psychophysical studies that the sense of touch is essential when it comes to dexterity and manipulation,” said Dr. Jeremy Fishel, Co-Founder of SynTouch. “This is the first time anyone has ever demonstrated a telerobot with such high-fidelity haptics and control, which is very promising and would not have been possible without the great engineers and technologies from this collaboration.” 

Telerobotics and the future of work

Machines are expendable. People aren’t. However, people are often placed in hazardous situations: diffusing a bomb, decommissioning nuclear facilities, performing deep sea exploration, administering medical aid on a battlefield, repairing the exterior of the International Space Station. Telerobotics may one day let people perform these jobs from the safety of their office or home.

Likewise, there are many locations where a telerobot would add significant convenience. For example, cleanliness is crucial when working in a sterile environment like a semiconductor fabrication plant or pharmaceutical cleanroom. The potential of introducing foreign debris increases when people enter such facilities. Telerobotic systems can reduce the risk of contamination by enabling human operators to work without requiring them to enter the cleanroom.  

“We believe this technology will revolutionize the way the world connects, communicates, and shares resources with one another.”  Kevin Kajitani, ANA AVATAR Co-Director

ANA believes in the future of telerobotics and organized our collaboration to help catalyze a new industry around avatar technology. “As an airline, I believe our ultimate goal is to connect people and help the world communicate, collaborate, and learn from one another,” said Kevin Kajitani, ANA AVATAR Co-Director. “We believe that this technology will not only be a breakthrough in mobility and transportation but also revolutionize the way the world connects, communicates, and shares resources with one another.”  

We look forward to our further research and development into avatar technology and telerobotics, and to our continued collaboration with ANA, Shadow, and SynTouch.