Build Copilots using Semantic Kernel

An AI copilot is an artificial intelligence-powered assistant (agent) designed to help users with various tasks, often providing guidance, support, and automation in different contexts.

These AI copilots can be found in applications such as productivity tools, customer support systems, navigation systems, virtual assistants, and software development environments. At its most basic level, an AI copilot is an AI assistant that can help you accomplish routine tasks faster than before.

Copilot Patterns

1. Beside: The Copilot as a helper within an app. It’s represented as a sidebar on the right side of a grey application interface. This suggests that Copilot can assist users while they are using a specific application.
2. Center: The Copilot is integrated into the central interface of an application. The text “Copilot as app foundation” indicates that Copilot can serve as the core functionality or feature within an application.
3. Outside: The Copilot’s ability to work across different applications and tasks simultaneously. Multiple application windows are shown with Copilot represented outside these windows. The text “Copilot orchestrates across apps and tasks” highlights its multitasking ability.

Copilot is everywhere

AI Copilot is designed to be accessible and helpful in a wide range of scenarios. It can assist with tasks such as coding, writing, learning new topics, answering questions, and much more. Its goal is to make your tasks easier and more efficient. However, please note that while it’s a powerful tool, it’s not literally “everywhere” in the sense of being omnipresent or having access to personal or sensitive data unless explicitly provided. It respects user privacy and confidentiality.

Agentic AI

Agentic AI, referred to as GenAI 2.0 goes beyond mere automation; it represents a paradigm shift in how we approach artificial intelligence within enterprises. Agentic AI systems designed to autonomously pursue complex goals and workflows with limited direct human supervision.

AI Agents exhibits the following critical capabilities:

• Autonomy: The ability to take goal-directed actions with minimal human oversight.
• Reasoning: Contextual decision-making, allowing it to make judgment calls and weigh trade-offs.
• Adaptable Planning: Dynamic adjustment of goals and plans based on changing conditions.
• Language Understanding: Comprehending and following natural language instructions.
• Workflow Optimization: Fluidly moving between subtasks and applications to complete processes efficiently.

Requirement for new practices

Each technological era brings new challenges and opportunities, requiring new tools and methodologies. This progression is a testament to the constant evolution and advancement in the field of technology.

The above diagram represents the evolution of development tools and engineering approaches over time, depicted as climbing a ladder. Each rung of the ladder signifies a new technological era. Here’s a breakdown:

1. Mainframe: This is the first rung on the ladder. The mainframe era was dominated by languages like Cobol and Fortran, and used Job Control Language (JCL) for task automation. The primary engineering approach was the Waterfall model and Structured Programming.
2. Desktop: The second rung represents the Desktop era. Integrated Development Environments (IDEs) became popular during this time, and languages like C++, Java, and .Net were widely used. SQL databases were common, and Object-Oriented Programming was the prevalent engineering approach.
3. Internet: The third rung signifies the Internet era. Web technologies like JavaScript, CSS, and HTML5 were essential during this period. NoSQL databases emerged as alternatives to traditional SQL databases. Agile Methodology became the preferred engineering approach, promoting iterative development and collaboration.
4. Mobile + Cloud: The fourth rung represents the Mobile + Cloud era. iOS and Android became the leading mobile platforms. Containerization, Microservices, Serverless architectures, and Continuous Integration/Continuous Deployment (CI/CD) practices became increasingly important in this era.
5. AI: The top rung of the ladder indeed represents the current era of Artificial Intelligence (AI). This era is characterized by rapid evolution and change, necessitating the development and use of new toolsets and tool chains. These tools work in conjunction with foundational models to facilitate the creation, training, and deployment of AI applications. This reflects the ongoing innovation and advancement in AI technologies and methodologies.

Sematic Kernel

Semantic Kernel is an open-source SDK that lets you easily build agents that can call your existing code. As a highly extensible SDK, you can use Semantic Kernel with models from?OpenAI,?Azure OpenAI,?Hugging Face, and more! By combining your existing C#, Python, and Java code with these models, you can build agents that answer questions and automate processes. The SK extensible programming model combines natural language?semantic functions, traditional code?native functions, and?embeddings-based memory?unlocking new potential and adding value to applications with AI. Semantic kernel enables developers to augment their applications with advanced capabilities.

Semantic Kernel is at the center of the agent stack

At the center of this stack is an AI orchestration layer that allows Microsoft to combine AI models and plugins together to create brand new experiences for users.

To help developers build their?own?Copilot experiences on top of AI plugins, we have released Semantic Kernel, a lightweight open-source SDK that allows you to orchestrate plugins (i.e., your existing code) with AI. With Semantic Kernel, you can leverage the?same?AI orchestration patterns that power Microsoft's Copilots in your?own?apps.

Semantic Kernel Architecture

Semantic Kernel? integrates Large Language Models (LLMs) like?OpenAI,?Azure OpenAI, and?Hugging Face?with conventional programming languages like C#, Python, and Java. Semantic Kernel achieves this by allowing you to define?plugins?that can be chained together in just a?few lines of code.

What makes Semantic Kernel?special, however, is its ability to?automatically?orchestrate plugins with AI. With Semantic Kernel?planners, you can ask an LLM to generate a plan that achieves a user's unique goal. Afterwards, Semantic Kernel will execute the plan for the user.

New Paradigm for App Development

The?Semantic Kernel?represents a new paradigm in app development, particularly for building intelligent AI applications. It is a conceptual model that leverages the power of semantics, or the study of meaning in language, to enhance the functionality and user experience of apps. By understanding and interpreting user inputs in a more nuanced and context-aware manner, apps built on a semantic kernel can provide more accurate, personalized, and intuitive responses. This approach goes beyond traditional keyword-based searches and commands, enabling a deeper understanding of user intent and a more natural interaction between the user and the application. As a result, developers using a semantic kernel can create AI apps that are not only more intelligent but also more user-friendly, paving the way for a new era of sophisticated and intuitive digital experiences.

AI Agents

An agent is an artificial intelligence that can answer questions and automate processes for users. There's a wide spectrum of agents that can be built, ranging from simple chat bots to fully automated AI assistants.

• A copilot is a special type of agent that is meant to work side-by-side with a user.
• Unlike an agent, a copilot is?not?meant to be fully automated. Instead, it is meant to help a user complete a task by providing suggestions and recommendations.
• For example, a copilot could be used to help a user write an email by providing suggestions for what to write next. The user can then choose to accept or reject the suggestion.

Build Agents (Copilots)

An agent is made up of three core building blocks:?plugins,?planners, and its persona.

These building blocks are what allow an agent to retrieve information from the user or other systems, plan how to use that information, and use that information to respond to a user or perform an action. For example, a copilot that helps a user write and send an email. After getting instructions from a user, the copilot would need to generate a plan using the available plugins to complete the task. This plan would include steps like...

• Get the user's email address and name
• Get the email address of the recipient
• Get the topic of the email
• Generate the subject and body of the email
• Review the email with the user
• Send the email

Plugins

At a high-level, a plugin is a group of functions that can be exposed to AI apps and services. The functions within plugins can then be orchestrated by an AI application to accomplish user requests. Within Semantic Kernel, you can invoke these functions either manually or automatically with function calling or?planners.

Planners

Planner is a function that takes a user's ask and returns back a plan on how to accomplish the request. It does so by using AI to mix-and-match the plugins registered in the kernel so that it can recombine them into a series of steps that complete a goal.

This is a powerful concept because it allows you to create atomic functions that can be used in ways that you as a developer may not have thought of.

Personas

Personas add a human touch to Large Language Models (LLMs), transforming generic outputs into customized responses that resonate with users.

They can be thought of as a “meta prompt” or “instruction” that influences how the agent responds to stimuli. In the simplest cases, you can use the persona to change its personality. This concept is particularly handy in applications like customer service bots and virtual assistants. It allows for the creation of unique AI personas without hefty costs or time investments. Overall, Semantic Kernel Personas represent a significant advancement in the field of AI, enabling more personalized and engaging user experiences.

Conclusion

In conclusion, the rise of AI agents, particularly copilots, is a significant development in the realm of intelligent applications. The Semantic Kernel, with its ability to seamlessly integrate Large Language Models with conventional programming languages, plays a crucial role in this evolution. It not only underscores the transformative potential of Agentic AI but also highlights a new paradigm in application development. This paradigm is characterized by more accurate, personalized, and intuitive responses, made possible by the Semantic Kernel. The process of building AI agents, with a focus on plugins, planners, and personas, is also elucidated. Ultimately, the Semantic Kernel emerges as a key instrument in advancing AI, paving the way for more personalized and engaging user experiences. This marks a promising step towards a future where technology is more closely aligned with individual needs and preferences.

References

• What is Semantic Kernel? https://learn.microsoft.com/en-us/semantic-kernel/overview
• Understanding AI plugins in Semantic Kernel: https://learn.microsoft.com/en-us/semantic-kernel/agents/plugins
• Automatically orchestrate AI with planners: https://learn.microsoft.com/en-us/semantic-kernel/agents/planners
• Getting started with Semantic Kernel: https://github.com/microsoft/semantic-kernel
• Architecting AI Apps with Semantic Kernel: https://devblogs.microsoft.com/semantic-kernel/architecting-ai-apps-with-semantic-kernel/