AI Agents don’t need data and training. They need knowledge and rules.

How a Gen-AI Agent differs from “older” approaches to AI and how that makes AI Agents accessible to more organizations.

?? I am the co-founder and CPTO of OpenDialog AI - Our mission is to transform the way users and organizations engage with business systems and processes by automating interactions through conversation. I write about product development, Conversational AI technologies, and, more generally, the impact of AI on the way we live and work.

Introduction

In talking with customers one of the questions we often get is how an AI Agent is trained to do a specific task. How does it learn? Customers are concerned that they will not have enough training data or that it will involve a complex process and highly specialised skills. These questions are completely understandable and we have only ourselves to blame for them.

For many years the narrative around AI (that we pushed as practitioners) was consistent. The task for a business, in very broad terms, was to identify a useful dataset, pick an appropriate ML technique and generate an ML model, trained on that data, that will be able to perform a classification or prediction task. The ML technique discovered the classification rules and expressed them through the model. The language we used referred to “training” and “learning” (as flawed as those concepts are in the context) and how the dataset was collected, curated and used to achieve this.

Generative AI, however, changes the task significantly, especially in the context of Conversational AI Agents. You are not "training" against a dataset. You are providing explicit knowledge and explicit rules that the agent should follow in applying that knowledge to a scenario. It is important to understand the difference and the benefits it brings for a clearer conversation about what it takes to create AI Agents.

In the next few paragraphs I will spend some time walking through the changes and explain how they open up access to automation and Conversational AI Agents to a much broader set of organizations and users.?

Disclaimer: Of course, LLMs can still be trained (e.g. through fine-turning) and for some use cases there may still be a need for raw training data. What I am arguing for here is that for a large portion of use cases clear rules around their process and knowledge in terms of documents is more than enough for organizations to realize value from AI Agents.

Conversational AI: The old way

Intent Classification

Imagine you are designing an an AI Agent that is going to assist users to perform changes to their insurance policy. In the “old days” (i.e. more or less just 12-18 months ago!) the task would be to train an ML model to distinguish between different types of phrases.?

Say, for example, you want to identify when a user wants to update their address. You would need a dataset of those types of dialogs with phrases such as?

• "i want to update my address"
• "where i live has changed"
• "the current address on file is no longer valid"

and so on. You would then train a model to classify such phrases as falling under the intent UpdateAddressDetails,e tc.?

As you can see there is a significant variation in how a user might say the same thing so you need a big enough and reliable enough dataset to properly distinguish between intents. In addition, you have to keep updating this model as your business process and terminology changes. All quite challenging.?

Content

Following intent classification, in traditional Conversational AI solutions you then had to worry about what content you present to the user. In most cases this was either static content or the result of a search against a predefined set of Question / Answer pairs with answers coming back as static content that was always the same. The challenge for organizations was once more generating this content in an appropriate way, with a lot of curation and editing necessary to get their knowledge into appropriate Question / Answer pairs with the answers tailored for a conversational interface.?

Conversational AI: The new way

Fast forward to the Generative AI time and both how we do (and if we do) intent classification changes and how we generate content changes.?

Semantic Intent Classification

LLMs are powerful language calculators and one of the tasks they excel at is classification. We can provide the LLM with instructions around what classes of phrases we want to distinguish and allow it to get to work.

At OpenDialog we call this Semantic Classification. Now, you may be wondering “do you even need to classify intents” when using LLMs. You can certainly interact with a user through a prompt without any classification, but for us that is analogous to driving in the dark with your headlights off. You may not care to capture every detail and different type of intent of what the user says but in order to provide an appropriate level of control and safety we still prefer to have an understanding of the user utterance before we hit any of the rest of our AI Agent with a request.?

You can certainly interact with a user through a prompt without any intent classification, but for us that is analogous to driving in the dark with your headlights off.

A Semantic Classifier enables to introduce structure into the conversation, at the level that is appropriate for a given context.? However,? instead of training an ML model what you are doing is providing rules and knowledge as shown in the (simplified) example below.

We’ve made this a hierarchical classifier so that you can capture the high-level intent (PolicyChange) and then dive into a more specific lower-level intent (ContactDetails) as required.?

Semantic classification, powered by LLMs, means that you do not need to worry about large datasets to perform classification of user queries, the process is much faster, and you can focus on a more appropriate set of categories. With some care around your instructions you can use a smaller LLM to decrease latency and by taking advantage of prompt caching (the input is almost always exactly the same) costs are minimal.

Since you can combine different semantic classifiers within a single AI Agent you can adapt your classifier to the context.

Interestingly, because of how semantic classifiers work with LLM-powered prompts that manage conversation you are often looking to detect things you do not want to deal with rather. In other words, your semantic classifier becomes a filter or a semantic firewall if you will, stopping unwanted statements hitting the rest of your AI Agent.?

Content Generation

Having classified a statement and decided that we want to deal with it lets move to content generation. While before we had to curate large sets of question / answer pairs in order to answer user questions we can now take advantage of Retrieval Augmented Generation (RAG) and use prompts that combine instructions around how to formulate an answer together with the knowledge they should draw on to generate the answer.?

In OpenDialog we support RAG through a combination of building blocks. First, we have RAG services where you upload documents, organize under topics and vectorize (and set on a re-vectorization schedule). These knowledge sources are then accessible to be queried from any prompt you define in OpenDialog. Any prompt that needs access to knowledge (from the prompts that powered intent classification above) to our LLM actions that power the behavior of the AI agent can access knowledge services.?

This flexibility enables us to combine and include or exclude knowledge sources or topics from within a prompt when generating content.?

Now, crucially, the task here was not to define Question / Answer pairs, instead it was to provide the source material and then allow the LLM through instructions in a prompt construct a contextually relevant answer. As such, what becomes critical are the rules and the knowledge you place around access to your content and that you actually have content that can be usefully queried.?

Another disclaimer: I don't want to pretend that there is no work involved. For a good result you will still want to tightly control context and instructions as well as the knowledge we feed into topics so that the output if something that works for our user and complies with any regulatory requirements. However, these tools significantly free us up to focus on outputs and overall user experiences in ways that simply were not possible before.

Conclusions

The shift from traditional AI approaches to Generative AI in Conversational AI Agents is a true step change in how we build and deploy these systems. No longer constrained by the need for large datasets and complex training processes, modern AI Agents thrive on knowledge and rule-based frameworks. By leveraging LLMs for both semantic classification,? content generation and conversational reasoning, businesses can automate processes more efficiently and effectively, even without extensive AI expertise. This new approach democratizes access to conversational automation, making it scalable, adaptable, and faster to implement for a wider range of organizations.