ChatGPT-based autonomous agents: what's the trick?

We start to get familiar with ChatGPT to make the most out of this conversational LLM AI (Large Language Model). Meanwhile, a wave of another order of magnitude is on the horizon: ChatGPT-based autonomous agents.

Microsoft's Jarvis project is under construction, and the BabyAGI and Auto-GPT projects are currently generating buzz. The principle behind these autonomous agents is always the same: use ChatGPT or any other LLM-type AI as a controller that understands, decomposes, delegates, and assembles. Because despite the power of LLM-type AIs, they cannot memorize, interact with an online service, generate a video, etc. In fact, ChatGPT is "just" able to answer any question extremely relevantly (most of the time). These autonomous agents use ChatGPT as a smart controller and then delegate the execution of tasks to the multitude of services and tools already at our disposal.

The question that must then emerge is: how can these autonomous agents interact so effectively with ChatGPT? What's the trick? Here is a striking example as an explanation.

I ask the following question to ChatGPT-4:

You are the autonomous agent of an online train ticket booking agency. Your role is to break down a customer request to extract the requirements related to the desired date and time of departure, the departure station, the arrival station, the desired class (1st or 2nd class), the preferences related to the desired location (for example: near a window, standalone seat, close to the snack bar), and the possible possession of a discount card (for example: frequent traveler card). I expect a JSON-formatted response, such as:

{
  from: "Macon",
  to: "Paris",
  departure_time_constraints: "tomorrow early afternoon"
  seat_location_constraints: "on the 2nd floor"
  class: "1st class"
  card: ""
  other: "I think I'll be taking my cat with me"
}
{
  from: "Marseille",
  to: "Toulouse",
  departure_time_constraints: "May 18th after 6 am"
  seat_location_constraints: ""
  class: "2nd class"
  card: "youth card"
  other: "and I hope I don't forget my keys this time"
}        

Please break down the following request:

Hi buddy, I need to take the train tomorrow around noon after dropping off my son at the babysitter's. I'm going to leave from Strasbourg and arrive in Paris, God willing. I'm going to travel in 1st class this time, and unfortunately, I don't have a subscription card. On the other hand, there's no way I'm dealing with an SNCF strike, or I'll lose my mind. Please assign me a seat near a window.

And here's the response:

{
  from: "Strasbourg",
  to: "Paris",
  departure_time_constraints: "tomorrow around noon",
  seat_location_constraints: "near a window",
  class: "1st class",
  card: "",
  other: "I need to drop off my son at the babysitter's before leaving. I'd like to avoid SNCF strikes."
}        

Well, that's the trick, and we use big words like "Prompt Engineering", "In-context learning", and "Chain-of-thought prompting" to name what's done in this trick. And this trick is... crazy, with economic implications that are still impossible to assess.

This is what we're leveraging at Reviewpad (part of Snyk) to build our own Agent, named Robin.