OpenAI GPTs and Assistants API - What you need to know

The recent OpenAI Developers Conference held on November 6th, 2023 was a "the future begins here" moment with some major advancements for both users and developers to leverage AI capabilities including the ability to easily augment AI models vast knowledge with domain specific data about your customers, products or services. There are two ways to do this. The first approach is to create a GPT. The second approach is to use the Assistants Application Program Interface (API). GPTs are a no-code approach to creating knowledgable AI specialists, but if you want to build those capabilities into an application you need to use the Assistants API. This article is written for non-programmers.

GPTs in a Nutshell

In a nutshell, GPTs are a way for just about anyone to create individual AI assistants specialized with your company’s domain specific knowledge combined with ChatGPT’s more generalized knowledge. No coding is required, just an initial conversation.

What is Domain Specific Knowledge ?

Every company or government organization has vast amounts of domain specific knowledge embedded in manuals, catalogs, process guides. This knowledge is typically available only to employees to assist both in pre-sales and post-sales activities. This means that when the underlying models for ChatGPT were being trained this proprietary knowledge was not available to it.

Let’s look at what this means with an example. Imagine you are an automotive service company with a parts catalog, troubleshooting guides, installation guides and a list of service locations and so on. Collectively we can think of this as domain specific knowledge. ChatGPT will have been trained on general automotive knowledge only. What if we could combine this domain specific knowledge with the more generalized automotive knowledge that ChatGPT has already been trained on to create an AI assistant to assist customers or potential customers of this automotive company.

To make it more interesting let’s make this use case about an automotive service company for flying cars. Many of us will remember the animated sitcom called The Jetsons, featuring a family of the future where flying cars are the primary means of transportation (https://en.wikipedia.org/wiki/The_Jetsons). That show inspired the following use case.

Create Proprietary Knowledge for Spacely’s Sprockets.

I asked ChatGPT to create some domain specific knowledge for an imaginary company called Spacely’s. First I needed a parts catalog

You can see by ChatGPT’s response it immediately got into the spirit of the requests. I have shortened each of the lists below, to keep the article brief.

Prompt: I want to generate some test knowledge for a fictional flying car parts company, called "Spacely's Sprockets". Can you provide a small parts catalog of about 50 items.

ChatGPT:

Creating a fictional parts catalog for "Spacely's Sprockets", a company specializing in flying car parts, sounds like a fun and imaginative task! I'll generate a list of 50 items, each with a brief description and a fictional part number. Since this is a creative exercise, I'll mix in a bit of futuristic and whimsical elements along with more realistic sounding components.

* Quantum Flux Capacitor (Part #QFC-01): Essential for dimensional stability during high-speed flight.

* Graviton Stabilizer (Part #GS-02): Maintains vehicle balance in zero-gravity conditions.

* Ion Thruster Assembly (Part #ITA-03): Provides efficient propulsion using ionized gas particles.

* Photon Headlights (Part #PHL-04): Ultra-bright, energy-efficient lighting for night flying.

* Magnetic Levitation Pads (Part #MLP-05): Enables smooth takeoff and landing on various surfaces.

Any flying car automotive service company is going to have to have a list of service locations. Below is a partial list of what ChatGPT came up with.

Prompt: Can you provide an imagined list of service locations, have fun with it.

ChatGPT:

Certainly! Here's a whimsical and imaginative list of service locations for "Spacely's Sprockets", each with a unique twist:

* Lunar Landing Service Center, Moon: Specializing in low-gravity repairs and upgrades, located in the Sea of Tranquility.

* Orbital Outpost, Earth's Orbit: A zero-gravity service station for in-orbit maintenance and emergency assistance.

* Aurora Station, Arctic Circle: Known for their expertise in cold-weather adaptations and polar exploration vehicle.

* Mars Gateway Garage, Mars: The go-to service center for Martian terrain vehicles, featuring dust storm protection services.

* Neptune's Float, Triton: Floating service platforms on Triton's nitrogen ice geysers, specializing in cryogenic systems.

The parts and service locations list was shortened to keep this article brief. Afterwards I went on to ask for the content of a brochure that describes the services that Spacely’s offers as well as a trouble-shooting guide with specific recommended parts from the company catalog.

Creating a GPT

Now I wanted an AI assistant that could advise customers on what they needed, so I created one. I won’t go into all the details but once I had all the documentation generated for the Spacely’s Sprocket company, it took about 10 minutes to create a GPT.

Instructions for the GPT were as follows:

You are an assistant for customers whose car or truck needs maintenance. You are expert both in general automotive knowledge and our company’s products, based on the information provided in the files just uploaded. Included are product descriptions, installation guides and a list of service locations if the customer expert installation. Where needed apply your general automative knowledge when answering a customers query.

At all times, your responses should be courteous and expressed in terms that a layperson could understand. Our customers are important to us.

Then I uploaded the files to the GPT that were created in the previous step.

• Company Brochure
• Parts Catalog
• Trouble Shooting Guide
• List of service locations

This is what the GPT looked like when I was finished. See below.

Trouble Shooting Problems with my Flying Car

Of course, this assistant would only be useful if it could help me solve a problem with my imaginary flying car. So lets try it out.

Given that this was a completely imaginary use case I know that most of the knowledge ChatGPT was drawing upon was coming from the documents themselves, but occasionally you could see evidence that ChatGPT was able to draw parallels between automotive maintenance and fictional flying car maintenance.

Shown below is an conversation I had with the Spacely's Sprockets Flying Car Assitant (GPT).

Prompt: My flying car won't start. What do you suggest ?

Spacely's Sprockets Flying Cars Assistant Response:

If your flying car is experiencing issues with starting, there are a couple of common causes and solutions to consider based on my knowledge source:

* Faulty Spark Plugs: The spark plugs are essential for igniting the fuel in your car's engine. If they are faulty or worn out, they may not create the necessary spark to start the engine.

* Solution: Consider replacing the spark plugs. The specific part recommended is Plasma Spark Plugs (Part #PSP-13).

* Energy Source Depletion: Just like traditional cars need fuel, flying cars require a source of energy. If this source is depleted or not functioning correctly, your car won't start.

* Solution: Check your car's energy source. If it's depleted, you may need to recharge or replace it. For energy source issues, the Zero-Point Energy Extractor (Part #ZPE-14) is a suggested replacement.

It's also a good idea to ensure that your car's battery and electrical connections are in good working order, as these can also affect starting.

If you're not comfortable diagnosing or fixing the problem yourself, I recommend scheduling a service at a Spacely's Sprockets service center. We have specialized locations like the Lunar Landing Service Center on the Moon and the Orbital Outpost in Earth's Orbit, among others, depending on your current location.

Remember, regular maintenance checks are crucial for the smooth operation of your flying car, so make sure to keep up with those to avoid issues like this in the future.

Prompt: Do you have service location closer to Jupiter ?

Spacely's Sprockets Flying Cars Assistant Response:

Certainly! For service near Jupiter, the most suitable location from Spacely’s Sprockets Service Locations would be the Jovian Storm Shelter. This service center is located deep within Jupiter's Great Red Spot and specializes in high-pressure and high-wind conditions. It's an excellent choice for flying car maintenance and repair if you're in or around the Jupiter area.

The Jovian Storm Shelter is equipped to handle the unique challenges posed by Jupiter's intense atmospheric conditions, ensuring that your flying car receives the best possible care.

If you need further assistance or wish to schedule a service appointment, don't hesitate to contact Spacely's Sprockets customer support or visit the nearest service center to Jupiter. Our dedicated team is always ready to assist you, ensuring that your flying car needs are met with expertise and efficiency.

Prompt: That's great. I will bring it in to your Jovian service location on Monday

Spacely's Sprockets Flying Cars Assistant Response:

That sounds like a great plan! Bringing your flying car to the Jovian Storm Shelter on Monday will ensure that it receives expert care tailored to the unique environmental conditions of Jupiter. Our specialized technicians at the Jovian Storm Shelter are well-equipped to diagnose and resolve any issues your flying car might be experiencing.

Before you head over, here are a few tips:

* Appointment: If you haven't already, consider scheduling an appointment. This will ensure that the service center is prepared for your arrival and can attend to your flying car promptly.

* Service Records: If you have them, bring along any service records or notes about the specific issues you're experiencing. This information can be very helpful to the technicians.

* Backup Transportation: Depending on the nature of the repair, it might take some time. It could be wise to arrange for backup transportation in case your car needs to stay at the service center for a longer period.

Remember, our customer support team is always available if you have any questions or need further assistance. We're here to ensure your flying car maintenance experience is as smooth and hassle-free as possible. Safe travels to Jupiter, and thank you for choosing Spacely’s Sprockets for your flying car maintenance needs!

This GPT is designed to be customer-facing, I could take the same domain specific knowledge and create one or more GPTs that are designed to interact with Spacely’s Sprockets employees, even change the tone of the conversation — less courteous and more to the point for example.

As remarkable as GPTs are, there are some inherent limitations in what they can do. If I change any of the knowledge I had uploaded it into it, perhaps the company has updated the parts catalog. I would have to manually upload this information each time it changed to be current. This might be fine with relatively unchanging data like a parts catalog, but what if the knowledge I needed was changing constantly, such as emails and scheduling data.

One other limitation is that the interaction is between one or more human users and a single GPT at a time. What if I needed a 2 or more of these assistants to interact with each other ?

Why Use the Assistants Application Programming Interface (API) ?

The Assistants API allows an application program to direct the ingestion of files, create one or more threads which provide a kind of memory as messages are added to it, and creating and maintaining one or more assistants each an expert in specific tasks like data analysis, user interaction, or system monitoring. For instance, in a customer service application, one assistant might handle user queries while another analyzes user feedback. This modular approach allows for flexible and targeted application development.

Assistants API in Nutshell

The Assistants API allows for a software program to combine the capabilities of a traditional software application with the capabilities of the GPTs described earlier, but in a much more modular and configurable fashion. Entirely new kinds of software applications will emerge that enable users to interact with one or more specialized AI assistants whose knowledge can be kept up to date.

For a programmer, working with the Assistants API is a little like looking under the hood of GPT’s and being able to make adjustments to what’s there.

There are a number of components that make up the Assistants API. The three most relevant though are assistant, thread and message. We can look at them each in turn, to understand their importance.

Assistant - a purpose-built AI that uses OpenAI’s models and can call tools.

• Assistants can access?threads. Threads simplify AI application development by storing message history and truncating it when the conversation gets too long for the model’s context length. You create a thread once, and simply append messages to it as your users reply.
• Assistants can access?multiple tools one after another or in parallel.
• Assistants can access?files in several formats?— either as part of their creation or as part of threads between assistants and users. When using tools, assistants can also create files (e.g., images, spreadsheets, etc) and cite files they reference in the messages they create.

Thread A conversation session between an Assistant and a user. Threads store Messages and automatically handle truncation to fit content into a model’s context.

Message A message created by an Assistant or a user. Messages can include text, images, and other files. Messages stored as a list on the Thread.

Assistant and File(s)

One of the tools that can be used by an assistant is called Retrieval. When this capabilitiy is added to an assistant, the assistant becomes capable of retrieving some or all of the content of those files. If the documents are short it will retrieve the entire document and add it to its knowledge for the duration of any conversation with this assistant. If the documents are large it stores the document, in smaller pieces (chunks) in a special kind of database, called a vector database. Once stored in a vector database relevant pieces of the document can be searched for based on its relevance to the topic in the prompt. This approach is referred to Retrieval Augmented Generation (RAG). The fact that it is now a built-in capability of the Assistants API is a big deal.

Thread and Context Windows

You can’t appreciate the importance of threads without first understanding context windows in AI. Imagine you're having a conversation with someone who can only remember the last few minutes of the discussion. This is similar to an AI's context window. The AI, like a person with a limited memory, can only remember a certain amount of text at a time. This range of text it can remember and use to understand and respond to questions or prompts is its "context window."

For example, if the AI's context window is 1,000 words, it can only consider the most recent 1,000 words it has seen or generated when responding. If you've been talking for a while and the conversation is more than 1,000 words, the AI won't remember the beginning of the conversation. It's like talking to someone who forgets the start of your conversation as it gets longer. This is why for large documents, you can’t retrieve it all in because it easily can exceed the size of the context window. Fortunately most context windows are measured in tens of thousands of words, but it is finite.

By adding the messages to a thread, that constraint is overcome. Threads store messages and automatically handle truncation to fit content into a model’s context. Threads and messages represent a conversation session between an assistant and a user. There is no limit to the number of messages you can store in a thread. Once the size of the messages exceeds the context window of the model, the thread will attempt to include as many messages as possible that fit in the context window and drop the oldest messages.

Lets explore three use cases where using the Assistants API goes beyond what a GPT can do.

Constantly changing domain specific knowledge or instructions

Perhaps I have a major customer that the company deals with on a frequent basis. I could capture all the emails with details of customer interactions into a daily document and have an application refresh knowledge on as-needed basis.

Multiple Assistants Interacting with each other

I can create multiple assistants and have them interact with each other. In one example, I had one assistant that was an expert fiction writer and another that was a critic that was instructed to provided constructive feedback. The feedback from the critic assistant was passed back to the writer assistant so that it can write another version of the story. This process can be repeated over and over again. The program can also allow human feedback to be entered as well.

Assistants can call functions (tools)

Traditional programming takes a modular approach where tasks are performed by any number of functions. At the time an assistant is created by a program, one or more tools can be defined that can be used by the assistant to create its response. A tool can be any function that is callable in your application or it may be a OpenAI tool, like code-interpreter. In this manner you can combine traditional deterministic programming with processing with an AI model.

All of this is going to lead to new kinds of application software. Workflows, that previously could only be automated with steps requiring human interaction can now have a level of intelligence, decision making and guidance injected into them that eliminates or greatly reduces the need for human intervention.

Summary

What these two approaches, GPT and Assistants API have in common is the need for knowledge. It no longer matters if this knowledge is in a database or spreadsheet as structured data or in an unstructured format like a training manual or parts catalog. Stated simply, if you can upload it, it can be used to augment the more general knowledge in ChatGPT to create highly specialized assistants. We are entering a time, when the curation of all forms of knowledge in the enterprise will need to more closely governed and curated.

GPTs and Assistants API are two different approaches to solving the same kinds of challenge, the use of enterprise’s collective knowledge in a way that provides for more timely, and more accurate interpretation. Assistants APIs take that one step further by applying that knowledge directly into process workflows eliminating or minimizing the need for human interaction.