Bridging the Gap Between Robotic Research and Industrial Application

In the dynamic robotics space, the gap between pioneering research and its practical application in industry can often be profound. There is a stream of novel results in foundational robotic skills such as planning, perception, and manipulation consistently emerging from the academic sphere. Yet, the journey of integrating these innovations into actual industrial operations is riddled with challenges: industrial robots and industrial processes see scant use as research platforms or target applications, and until recently there was no clear path for software built on research-focused software platforms like the Robot Operating System (ROS) to a software platform in the actual production environment.

This disconnect between research and industrial platforms creates friction in leveraging new technology where it is needed most, relegates robotics research to a whirlpool of closed successes and iteration, and hamstrings its ability to benefit to society. We believe that ForgeOS is the platform that creates this essential corridor from breakthrough robotics research to actual robots in actual factories.

Understanding the Challenges facing ROS in Industrial Deployments

ROS has been a key component in the success of robotics research in the last 15 years because it enabled researchers to use the same set of core libraries for their work and start to make robotics research reproducible. While this has been transformative in academia and private research, ROS has not had the effect in the industrial space that was expected, and with a few exceptions, most products that started based on ROS were wholesale re-written for industrial use. This means that even algorithms that were developed on ROS are hard to deploy on real industrial robots, let alone algorithms that were developed without ROS.

To illustrate this, let’s look at NVIDIA's NanoOWL, a cutting-edge object recognition algorithm optimized to run on GPUs, based on OWL-ViT by Matthias Minderer et al. at Google. NanoOWL exemplifies an out-of-the-box useful algorithm that traditionally would find a slow path into industrial use. This is an example of a ROS-compatible (with some work), research-focused algorithm that could have immediate use in the industrial space.

To build an end-user application with ROS to use NanoOWL and deploy that app on an industrial robot, just a few of the steps would be:

-????? Build the entire ROS environment on a Linux computer.

-????? Write custom code to tie in the ROS vision pipeline into NanoOWL so it can use that vision stream as the input for its object recognition.

-????? Find the ROS Industrial package for the desired robot, if it exists and is in a good state of maintenance. These packages usually offer limited support for the full capabilities of the industrial robot.

-????? If compatibility with more than one robot is necessary, repeat the previous step.

-????? Write an execution architecture for controlling the robot and moving it to the position of the object reported by NanoOWL or write more code to use a built-in ROS library.

-????? Build a custom user interface that sits atop the execution architecture.

Notice I didn’t mention safety, reliability, user testing, or several other considerations that would be required to put this task on an industrial robot in a factory setting.

Deploying Research Algorithms with ForgeOS

ForgeOS was built from the ground up to be an extensible platform for this type of integration of new technologies and algorithms. But above all, ForgeOS is a robust industrial platform that is being used every day in factories around the world. Many of the things we mentioned above are serious challenges that ForgeOS just solves:

-????? Build your application anywhere, on any system, in any language, because communication with ForgeOS is through REST

-????? The robust ForgeOS API and its Python bindings allow for far simpler integration at the Python code level.

-????? ForgeOS already has a task orchestration and execution layer, with a low-code user interface for creating the robot’s behaviors.

-????? Forge works with eight robot brands and more than 250 robot models, supporting their full capabilities – and the API is identical for every single one.

-????? ForgeOS already has an award-winning user interface for supporting the end user interacting with the system.

-????? ForgeOS works seamlessly with the safety systems on every industrial robot.

So how does this work in practice? We deployed NanoOWL onto a ForgeOS-controlled industrial system in a few hours, and here’s how it went:

? 1. Setting Up NanoOWL and Connecting to ForgeOS:

We started by cloning the NanoOWL project and connecting it to a stream from an Intel Realsense RGBD camera, using pyrealsense2.? These are two packages that were used out of the box – no custom code was written to get them running. From here we created a simple program in Python using our ForgeOS Python package.? This package provides a simple Python API for connecting to the REST endpoints in ForgeOS. With a few lines of code, we called the Python API for ForgeOS to get an object prompt from Forge, passed that to the call to NanoOWL, and sent the resulting coordinates back to ForgeOS, again using the API. This surfaces the location of the prompted object, e.g. “a white box”, as a waypoint in ForgeOS.

? ? 2. Building the Task in ForgeOS:

Once we had NanoOWL working as expected (again, out-of-the-box) and sending object data to ForgeOS, we tested this integration with a simple example of detecting an electronics enclosure box and having the robot place a mounting plate inside the enclosure. In ForgeOS, using our low-code programming application Task Canvas, we wrote a high-level program that would send a prompt “white box” to the NanoOWL, and when the box was found, receive back its position in space. That position was then used in the Task Canvas program as a reference location to pick and place the mounting plate. Our program in ForgeOS not only used the position from NanoOWL but also used the presence of the box as the interactive cue from the user to trigger the pick and place.

3. Deploying to Different Robots

We first deployed NanoOWL to a Doosan M1013 robot, but then also deployed that same code on a Fanuc CRX10iA robot in a different part of our facility. Both systems were set up within a few hours, and we had never deployed this code to any of our robots before. This illustrates the power of a platform like ForgeOS, where not only can a powerful algorithm like NanoOWL be deployed to real industrial hardware in a few hours but multiple completely different robots. The robust and usable API of ForgeOS and the fact that it is brand agnostic means that the friction that would normally exist for fielding an AI algorithm like this is almost entirely gone.? ?

Paving the Way for Commercialization

The advent of a platform like ForgeOS is not just timely but transformational. It serves as a vital conduit for channeling research innovations into commercial viability. The gap between robotic research and its practical industrial application has long been a significant barrier in the field. However, the introduction of ForgeOS marks a turning point. By bridging the gap between the academic advancements in robotics and their real-world deployment in industry, ForgeOS revolutionizing the way research is translated into tangible, industrial applications.

The example of NVIDIA's NanoOWL illustrates this evolution. Traditionally, integrating such an algorithm into an industrial setting would be a slow, laborious process. With ForgeOS, however, the deployment of NanoOWL on various industrial robots was achieved in a matter of hours. ForgeOS addresses the myriad of challenges previously faced in this domain – from compatibility issues with multiple robots to the ease of integration with external software. Its robust API, compatibility with a wide range of robot models, and user-friendly interface significantly lower the barriers to deploying advanced robotics technology in an industrial context.