Rise of the Agents:From 007 to AI Agent

In last month’s TDA newsletter edition we talked about advancements in all aspects of Movie-making facilitated by the AI & GenAI technology revolution.? So much has happened in the realm of AI since then - Meta released Llama3, GPT 4o released by CTO Mira Murati at Open AI, Gemini Pro 1.5 with 2M tokens, Astra and more? innovations announced by Sundar Pichai at Google I/O and IBM open-sourcing Granite Large Language Models with CEO Arvind Krishna envisioning AI scaling with hybrid cloud and Research SVP Dario Gil proclaiming “Future of AI is open” and pushing for renewed focus on trustworthiness, transparency and AI adoption for the enterprise at the recently concluded? Think24 conference. Even Microsoft launched an AI PC and new Copilot ?harnessing the power of AI beyond the cloud on the edge last week at Build2024.? It seems it's Everything, Everywhere and all AI all at Once!?

In between, the case of AI ethics and governance got stronger last week with popular actress Scarlett Johanasson slamming OpenAI for using a clone of her voice for the GPT4o “Sky” feature despite rejecting Sam Altman’s offer last year. This has resulted in public re-awakening to matters of privacy now with film and artist industry body SAG-AFTRA backing Johansson against OpenAI. All this clearly reads like a script for an action-packed blockbuster movie!?

From 007 to AI Agent

An AI generated trailer of upcoming James Bond movie was released last month and it went viral with over 4M views..except that it was a fake! While the British secret agent 007 may not have embraced AI yet, there is another agent that is gaining popularity. Yes, its the AI Agent and lets focus on understanding it

What is an AI Agent?

While traditional software applications act as tools to help you get a job done and apps make the work easier, but you still have to do the work. Agents on the other hand are different.

Agents are autonomous software systems that can reason, make decisions, and pursue goals with creativity and flexibility, all while staying within the bounds that have been set for them.

Whereas apps help you do the work, agents get the work done for you. AI agents have the potential to be so useful that they will soon be ubiquitous and available everywhere and for everyone.?

Humans set goals, but an AI agent independently chooses the best actions it needs to perform to achieve those goals. For example, consider a contact center AI agent that wants to resolves customer queries. The agent will automatically ask the customer different questions, look up information in internal documents, and respond with a solution. Based on the customer responses, it determines if it can resolve the query itself or pass it on to a human.

The concept of an agent has been around in academia for decades, but only recently has it become possible to build sophisticated agents that you can converse with and which can get useful things done in the real world. Behind this breakthrough are recent advances in artificial intelligence, and large language models (LLM) in particular. For example, using an AI agent, a bank manager can go beyond simply chatting with an LLM on what to do in the case of a fraud alert to actually taking action on blocking the transaction and sending the customer the recording and remediation forms.

But, how does an AI Agent work?

The inherent capabilities of LLMs – natural language understanding, language generation, and reasoning – make it possible to build sophisticated AI agents. But an LLM by itself is not an AI agent. Sophisticated AI agents consist of a number of components, all working together to enable an agent that can converse fluently, answer nuanced questions, and solve complex problems. Think of this like an orchestra, with each component serving a distinct and important role and, by working in concert with others, enabling something greater than the sum of the parts. To build an agent, you need more than an LLM- memory, tools and actioning capability.

By combining many separate LLM calls, each for a special purpose, agents can achieve far higher performance than would be possible with a single LLM call. Agents can also use long-term memory to answer questions or solve problems. In order to be able to do anything, agents need to be able to take action and interact with systems and services around it. For example, for an e-commerce retail store, no matter how powerful the latest LLM is deployed to improve customer experience, it can’t possibly know where a shipped package is, or the status of a subscription, or the last time a customer interacted with them. This type of information lies only in the company’s own internal systems and must be integrated.

Giving an agent access to “tools to use” via APIs – interfaces that enable inter system? information exchange possible– enables an agent to interact with these systems and take action.

Can we trust AI Agents ?

AI agents are by definition autonomous and rational. They can converse, reason, and take action – all on their own. However, the newsflashes of a bot going rogue never fail to amuse us -? be it Microsoft’s foulmouthed twitter bot? TY or more recently when a brand new Chevrolet vehicle was offered ?for a $1 or GM bot recommending a competitor’s product?

While as a technology in the abstract, autonomous agents sound really powerful. But if we are putting an AI agent in front of customers, a key question is: can we? trust it?

With hallucinations, bias, and risk of sensitive data leaking into models, the answer very clearly is no, not without significant guardrails. And given the fuzziness around exactly how and why transformers work, explainability is very important for building trust. Trust starts with being able to define and enforce guardrails for an agent. Transparency and explainability are a must have. Monitoring and observability needs to be built in as well. We need AI agents? to express and enforce deterministic rules and business logic where needed. Tools for monitoring and auditing need to be an integral part of the solution. Also, features like secure data grounding and data masking to Zero Retention Prompts, audit trails, citations, and toxicity filters need to be incorporated. Thereby a well architected agent should be able to log reasoning traces for each decision it makes, making it possible to correct and improve its performance and behavior over time.

What does the future entail?

While I don't have a clear crystal ball,?with the rapid pace of innovation in this space, we seem to be moving into the age of AI agent ubiquity. A term CaaS (cognition as a service) is being pitched by venture capital firm Mayfield that defines the framework for this category where startups such as Sierra.ai and Sema4.ai are couple of leading names in the AI Agent race. I see a future where every person gets a digital co-worker, enabling humans to behave like superhumans.

At the very least, I would love to see an AI Agent? that simplifies my travel — finds me a new exotic location I have never visited or? that perfect vacation, books my hotel and flight, remembers to ask if I need a car or SUV, records and communicates my diet preferences, checks me into my flight ensuring my vegetarian meal has been provisioned, and delivers my boarding pass to my email or phone- all without reminding or calling travel agent, any customer service representative or? my office human assistant.

Key questions: While fear mongers?are continuing to sound sirens on AI Agents taking over the human world and destroying it,? my 3 immediate worries are:

1. ?When will my future travel planning, booking and execution be fully autonomous through AI Agents?
2. When will my tax returns be autonomously filed? and paid without me fuming and fretting during tax season especially with my blood pressure rising on the April 15 deadline every year??
3. Will the AI agent be as smart and good looking as 007?

(Please do share your comments and subscribe to this newsletter and click ?? if not already done so)