Controlling Robots Remotely

The world has seen a change in work styles in recent years towards working remotely. This change was accelerated by the global spread of the COVID-19 pandemic, so it has now become customary to work away from one’s place of employment.

The change in working styles has not affected every industry equally, however. Many jobs, such as in engineering or medicine, are difficult to do remotely. Due to the technical difficulties inherent in automation and mechanization, there are cases where even dangerous work is still conducted manually, and humans remain needed in work areas due to physical, time and movement restrictions.

Japan is vulnerable to this problem due to the shrinking nature of its population and workforce—there are sometimes not enough workers on the ground available to do the jobs that need to be done.

Remote working that enables the performance of a task regardless of the worker’s location could be a partial solution to this problem.

NTT and Mitsubishi Electric have recently joined forces to develop a demonstration experiment that combines NTT's network and computing-based cooperative control technology with Mitsubishi Electric's remote-control technology.

The research partners placed a display and robot operation device at the NTT Musashino R&D Center in western Tokyo and a robot at the NTT Yokosuka R&D Center, located 50km away. They then set up an experimental network to enable remote control.

NTT focused on developing a network and computing high-speed, closed-loop control technology that would allow for stable and low-latency service by immediately switching to another path and another server process when the end-to-end delay time does not meet the performance requirements. Meanwhile, Mitsubishi Electric developed Visual Haptics, a process that converts remote haptic information into visual color information and transmits it in that form, instead of transmitting it to the user as a physical stimulus.

In theory, achieving advanced remote control of a robot using haptic information could lead to the safe, advanced remote control of robots. Doing so requires a connection that is robust and has low latency. NTT’s end-to-end delay quality management and control technology makes this possible. It not only reduces the delay time and jitter (displacement and fluctuation) of the connection, it also prevents any deterioration of service quality, even in the event of system failure or performance degradation.

Mitsubishi Electric’s Visual Haptics added information to the camera images taken by the robot and the test operator was able to operate the robot while visually confirming the haptic information.

The result? Success: advanced work could be performed remotely and has the potential to expand the scope of work, regardless of location.

In the future, NTT and Mitsubishi Electric anticipate the opening up of new types of activity, such as remote surgery that does away with regional disparities in medical care and remote equipment repair. Reducing or even eliminating the travel time of engineers could lead to sustainability benefits, while the technology holds the promise of giving employment opportunities to people with physical limitations.

NTT—Innovating the Future

?