Copilot, Copilot, er, Agents?

Welcome to Grounding AI. It’s been a busy time these past few weeks, with Microsoft Build and TechCon365 conferences in Seattle. In this edition, we’ll discuss AI Agents, as announced at Build, Agencies, and share a Copilot tip to help you on those busy meeting days.

The Rise of the Agents

We finally heard our first official mention of AI agents at Microsoft Build this month with the Copilot Agents announcement. AI agents have been around for a while but have been the purview of professional AI developers. The ability to create agents using low code Copilot Studio will make this process much easier. You’ll be hearing more about agents over the next 18 months.

What is an AI Agent?

Artificial Intelligence (AI) agents are designed to perform tasks on our behalf. They can use AI capabilities to understand and conduct specific actions, making our lives easier.

Imagine having a virtual assistant that helps you with tracking your project work, which manages your calendar, or even finds the best recipes for your daily meals to keep your diet on track. That's what an AI agent does – it helps you by doing tasks automatically and efficiently.

How are Agents Different from Copilot?

While AI agents and copilot systems might seem similar, they have distinct roles. An AI agent works independently to complete tasks without intervention from you. It can manage processes on its own, making decisions based on its programming and data it receives. On the other hand, a copilot system is designed to help you interactively. It works with you, offering suggestions and guidance as you perform tasks. Think of it as a smart assistant that gives you real-time advice while you’re working, whereas an AI agent is more like a background worker doing its job quietly.

Types of AI Agents

AI agents come in diverse types, each with specific functions and expertise. In my experience, I think of agents as a mix of process and expertise capabilities. While there are many classifications, the classification comes down to a combination of process capabilities of the agent, the expertise necessary for the agent and whether any of these two aspects will improve as the agent gains experience.

Process

Some agents are created with the capability to manage and automate processes. For example, they can manage repetitive tasks like data entry, scheduling, or managing emails. They are great at making routine work more efficient and error-free.

The complexity of the agent is determined by how they manage those processes. Simple agents may only respond to specific conditions, requiring low expertise. For example, we may have an agent that turns on the lights in our office, without regard to the outside lighting conditions. It works, it’s fast and dependable, but there’s potential room for improvement.

Some agents use a set of goals with some level of specialized expertise to manage a process. A scheduling agent would manage your calendar based on the goals of maximizing the number of attendees for a meeting, while not scheduling meetings after hours.

Others are utility focused where both the goals and the need to optimize outputs are necessary. They use multiple criteria for decision-making, can make trade-offs and incorporate user preferences into the process. Agents that make investment recommendations, for example, tend to be more complex and require a high level of expertise.

Expertise

The expertise required for agents will determine what data is needed, how these agents are grounded, and how often the data is updated. An office light management agent may only need updates on an as needed basis, perhaps quarterly. A medical expertise agent that helps doctors diagnose diseases by analyzing patient data and suggesting possible treatments will need very timely data to maximize its utility.

This Grounding AI edition sponsored by MarqueeInsights.com .

Elevate Your Data Analysis with Power BI with a private 2-day training class! Contact us to book a class for your organization at [email protected]

Local Agents

Both Microsoft and Apple have announced AI on your device. If we were to reimagine an operating system in a world of AI, one of the tenets would be that the OS should automatically adjust to the needs of the user as they are working. To do this, the AI needs context information as to what the user is doing. Microsoft Recall and Apple Siri’s screen awareness will provide this context to the onboard AI agents. Data security is a major concern, so I expect this use case to evolve quickly over the next year.

Team/Organization Agents

Team level and organization level agents will be where most companies will start. The investments require major capability increases or cost savings. I feel most organizations should start thinking about what processes would be good candidates for making agents.

Examples of Uses

AI agents are used in many fields to make tasks easier and more efficient. Here are some examples:

• Customer Service - AI agents can answer customer queries, resolve issues, and provide information without human intervention.
• Healthcare - AI agents assist in diagnosing diseases, managing patient records, and even predicting health trends.
• Education - They can personalize learning experiences, grade assignments, and provide tutoring support.
• Business - AI agents help with data analysis, financial management, and automating business processes.

Evaluating Automation Risks

Two aspects need to be considered when evaluating whether an agent should be used to automate a given need. We know that LLM technologies upon which these agents are built can hallucinate. So, you should weigh two factors when an agent makes a mistake. First, what is the impact of a mistake? If a scheduling agent makes a mistake, it’s annoying but not highly impactful. If a medical insurance recommendation agent tells a person that an expensive procedure is covered but it’s not, it may be life changing as the procedure may bankrupt the person.

You should also determine how likely it is to make mistakes. Azure AI is releasing several LLMOps tools to help you do this type of testing in an automated way.

Agencies (AKA Agentic Workflows)

Once you have agents, you’ll want to create teams from those agents. The academic term for this is Agentic workflows. However, I prefer the term Agencies as it describes the situation more clearly and sounds much less pretentious. The term Teams is so overused that Agency is a good substitute.

What Are They?

Agencies are systems where multiple AI agents work together to complete complex tasks. They can be self-organizing and self-collaborating, depending on the framework used. There are many available, like Autogen, CrewAI, and AgencySwarm.

How Are They Related to Agents?

Just like how different specialists work together in a team, AI agents in an agency cooperate to achieve a common goal. For example, in a business setting, one agent might oversee data collection, another might analyze the data, and yet another might generate reports. Together, they streamline the workflow and enhance productivity.

Flexibility

Agencies can be created for long- or short-term use. Much like human teams, agencies can be created easily to address a specific need. The agency frameworks today can create needed agents on your behalf, based on the roles needed. This offers great flexibility to address emerging needs. However, those agents don’t possess any knowledge beyond what GPT already knows. I may have an immediate need to create an agency that incorporates an internal agent with detailed knowledge of our shipping processes to address a customer need. Today, that’s not easily possible. Already, work is ongoing to enable incorporation of existing internal agents such that an agency can be created that uses your internal agents as well as creates new agents for specific generic roles.

Impact of Agencies

Of all the advances, agencies will require rethinking how we structure and execute work within our organizations. Today’s Robotic Process Automation (RPA) projects are brittle by nature. All the logic is hard coded by the creators. When the business changes, it’s a large effort to update the workflows. Agencies have the potential to be self-adjusting, giving us workflows that can learn and evolve. We’ll be able to automate entire capabilities and make them available to employees to use. To get to this stage though, we must ensure we’ve built the appropriate agents within our companies.

Copilot Tip

This edition’s Copilot tip came from a discovery after spending six hours in meetings. We had a client issue that required several people to be involved. Rather than trying to figure out the email thread and the Teams chat, I asked Microsoft 365 Copilot, “What’s the latest in the conversation between Heather Waters, Jane Oso, and James Bell on the Contoso problem we are discussing?" Copilot was able to summarize all the related Outlook mail and Teams chats to provide me with an instant synopsis of where matters stand.

That's it for this edition. More real-world uses of AI to come in the next edition.