Robots, AI and the promise of the future

At the start of NVIDIA GTC in San Jose, I had a plan for how my week was going to play out. Like many others, I was intrigued to learn more about the applications of Large Language Models (LLMs) and Generative AI, technology that has captured the public's imagination thanks to companies like OpenAI and their now seemingly ubiquitous ChatGPT chatbot.

Instead, the final section of the keynote presentation set me on a new course for the week. Jensen Huang captured MY imagination as he introduced a wealth of innovations and tools in the world of robotics.

Starting the journey via (autonomous) car

My week started with an excellent session delivered by Marco Pavone , who leads autonomous vehicle research at 英伟达 . A key piece of learning for me focused on the different steps in constructing foundational models. Specifically:

"The desired end capabilities inform the choice of data, model and training tasks"

This aspect on training (both self-supervised and supervised tasks) would prove to be key in drawing linkages and connections for myself in subsequent sessions during the week.

Robots Everywhere

The exhibition hall at NVIDIA GTC was packed with both humans (over 16,000 people attended during the week) and experiences. The experience was energizing, with countless in-person interactions. What drew my eye (and the attention of so many others), was the wide array of robots of all form factors and abilities on display.

I classify the robots in action into two types; those focused on mobility and those robots that are more task specific.

Coffee and Chess

There were many different task-specific robots at this year's conference. My two favorite examples from the showcase are the dueling chess bots and the Artly barista bot.

Coffee is always an essential part of my conference experience. As many a vendor knows, free coffee is a guaranteed way to draw attendees in to your booth. Combine that with the added appeal of a robot, and it was easy to explain the crowds that were gathered as Artly delivered me a delicious coffee to power me through my afternoon.

Mobile Robots: Different and complementary

The mobility and agility of robots like the Boston Dynamics Spot was on full display. Multiple exhibitors showcased the four-legged robot throughout the exhibition hall, with each instance drawing large crowds.

This year's conference brought a renewed energy to the humanoid robot. This was definitely highlighted by the announcement during the keynote of NVIDIA's Project Gr00T, a general purpose foundational model for humanoid robots.

One such robot on display was the Apptronik Apollo, featured during the keynote. Standing at 5'8", it is an impressive piece of technology up close, and promises to be equipped with capabilities delivered by Gr00T and the newly announced NVIDIA Blackwell architecture that debuted at the conference.

A fascinating shift appears to be underway in robotics, reinforced by Marc Raibert during his session "Robotics and the role of AI: Past Present and Future". When asked what’s new and different in the approach at RAI Institute from his time at Boston Dynamics , he said:

“Only half of our focus is now on the physical, with a new half on cognitive functions.”

The power of Human PLUS AI

I don't mind telling you that I'm a big kid at heart. That's why the session that brought me the most joy this week was "Breathing life into Disney's robotic characters with Deep Reinforcement Learning". Moritz Baecher and the team from Walt Disney Imagineering kept the packed room spellbound for the entire time. The team also produced the biggest flurry of cellphone selfies and videos to rival the furor that followed NVIDIA CEO Jensen Huang wherever he went this week.

Of course, the stars of the show, and the focus of all the attention were the Disney BD-X robots, who appeared on the keynote stage next to Jensen.

This session, for me, was where everything from the week came together. I'll explain.

Remember the training tasks I talked about at the top of this article, that help drive the desired capabilities of foundational models? These BD-X robots were trained on thousands of simulated runs, with many different randomized variables, including weight distributions and randomized terrain, in order to teach the models how to make the robots walk. Using capabilities from NVIDIA Isaac Sim, a simulation platform built to allow for the design, testing and training of AI-based robots, the Disney team were able to achieve in two days what would amount to years of training in the real world.

This reinforcement learning is also key in Project Gr00t. Isaac Lab, announced during the GTC keynote, is a new application for robot learning built on the Isaac Sim platform. All of these capabilities assist in minimizing the Sim2Real gap in bringing new systems to the real world.

The Human Aspect

So far, the focus here has been all about the technology. This is a tech conference, after all! The final, and for me, most crucial aspect of these BD-X robots is the human element.

For all of the powerful technology applied to the creation of these robots, the essence of the appeal of the BD-X droids is due to the creativity and skill of their human animators and operators. This is a shining example of using AI to supplement human capabilities, not to replace them.

You can watch this video of the Droids in Training at Star Wars: Galaxy's Edge to see more of these remarkable robots in action. I'm sure it will leave you with a smile on your face. It certainly does for me!

I know we are heading in the right direction with leveraging AI when we focus on making the human experience better. That focus is what gives me unbounded hope for the future.

#AI #GenAI #Robotics #Futurist #TechLeadership #NVIDIA #GTC24