On AI Agents ...

Another month has passed, and it is time for a new Generative AI Solution Newsletter edition. :)

As the hottest subject in Gen AI is AI Agents, it is not surprising that so many of you would like to keep up with the latest improvements and news on this topic. This could be because you are an AI practitioner and want to know how to build AI agents and integrate them into products and services, or an AI Leader interested in finding out about opportunities on how AI Agents could be applied in your organisation or for your client's products and services. Knowing about types of agents, their use cases, design patterns, and how to develop and use them is a must if you are following the recent advances in AI.

Good news. I have recently watched recordings of an online conference on Building AI Agents with LLMs: Harnessing the Power Of Generative AI with Autonomous Agents (https://learning.oreilly.com/videos/building-ai-agents/0636920961598/) on the O'Reilly Platform. Although it is from late 2023, I still found the content quite educational, so I wanted to share a summary here as all presentations were fantastic and the order in which they were executed to build a story could not be better. The full duration of this conference, which is six presentations, is about two hours as a single video.

I have learnt

• The different types of AI agents and when and how to use them
• Which agents to use based on particular use case
• The limitations of AI agents and how to contain the scope of their work
• The latest tools that have been developed to help build own AI agent

Presentations

Kence Anderson: Intelligent Autonomous Agents

Kence Anderson, founder and CEO of Composabl shared a sample use case after explaining two design patterns for AI Agents.

Kence also shared the use cases for intelligent autonomous agents covering a variety of areas such as Intelligent Process Control, Robotics, Factory Machine Control, Formulation Discovery, and Autonomous Vehicle Control, stating that the common thing across all these is the decision-making provided by the agent, and the follow-up action planning and execution.

I particularly enjoyed learning about the structure of intelligent autonomous agents.

Kence shares two design patterns: the Copilot Pattern, in which the LLM is used before the perception layer, and the Research Pattern, in which the LLM is included as part of the agent's perception layer.

Yohei Nakajima: Building an Autonomous Agent: The Evolution of BabyAGI

Yohei showed the history of developing his personal project, BabyAGI, a framework that can be used to build self-learning autonomous agents. He also discussed various approaches to LLM-based task creation, including skill and tool management for agents.

I like how Yohei started from a simple concept of using agents for task planning autonomously, and worked through several iterations to construct and advance the framework. I also enjoyed the naming convention used, starting with BabyBee and moving to BabyCat, BabyDeer, BabyElf, etc. :) I later checked it out, and the most recent version stores and executes functions from a database, It is available here: https://github.com/yoheinakajima/babyagi

Jenny Kaehms: Building Your First AI Agents

Jenny's presentation was more general as she went through the use cases for AI Agents and their types. She explained the 5 types of AI Agents and their levels of adaptability, starting from Reflex-based on one end of the spectrum to object-centric on the other.

Jenny gave a few examples of use cases and explained their type. For example, in Healthcare, using an LLM can return the information based on the query. However, AI Agents can monitor and help maintain personalised insulin levels, being of type Reflex-based. And in the other side of the spectrum, for example, in Education, LLM directly can suggest answers to test questions and homework, in contrast with the AI Agents being able to generate lesson plans adopting specific learning goals and being of object-centric or curious type.

Div Garg, the founder of MultiOn: Building Human-like AI Agents

Div shared why AI Agents are required and how MULTI.ON capability to have complete read/write access to the internet provides human-like activities that AI Agents can perform.

Div also discussed the 5 levels of autonomy for agents, stating that the autonomous agents were on the level 2/level 3 boundary, where a human is available in the loop to monitor/ fallback in case automation fails.

Shawn “Swyx'' Wang: Anatomy of Autonomy

The Swyx presentation was the most interesting for me personally. It started by discussing the levels of AI that Div introduced in the previous talk. He believes that AI Agents are sitting at Level 2/3 of autonomy. He further explained why he believes BabyAGI developed by Yohei is an interrupt-based Level 3 Agent.

Swyx referred to one of his blogs to explain the anatomy of autonomy and how he has used it in his development project Smol, which enables you to code in English, for example, use bullet points to code nesting!

He also suggested his latest research to switch the LLM Core/Code Shell solutions, such as RAG, with LLM Shell/Code Core solutions, such as Copilot, stating that the LLM Core apps are constrained by the LLM capabilities.

Arjun Bansal: Evolution of LLM Agents and the need for Robust Evaluations

The final presentation by Arjun Bansal was a perfect ending to the conference. He talked about how the early agents were limited to one API call, which have evolved into a dynamic chain that enables them to perform more-complex human-like tasks.

Arjun comprehensively discussed four types of LLM evaluation: Metric-based, Tool-based, Human expert in the loop, and Model-based.

He finished by introducing robust evaluation methods for LLM, such as use of Agents, Fine-tuning, Distillation, and Model Orchestration.

Final Thoughts

Hope you enjoyed this edition. AI Agents are developing fast, and as I was writing this Newsletter, I also found out that OpenAI has released OPERATOR (only to monthly paid members), covering several areas in which agents take over human tasks. There is a nice first look at OPERATOR available on You Tube, released by Cassie Kozyrkov called: Full Unbiased Demo of Operator (Agent from OpenAI) . Watch it below

https://www.youtube.com/watch?v=xtiab6jTWWQ

The future is Agentic. Keep learning, keep discovering, and stay curious! :)