Mixture of Agents (II)

Two weeks ago, we started discussing the Mixture of Agents. We tackled the systems of thinking and how we'd like to translate them into LLM operations.

We've also highlighted the Mixture of Experts approach as the prelude to how we want to achieve long-inference models.

If you're not familiar with the topic, here's part one.

Today's chapter will be focused on the actual Mixture of Agents approach.

Without further ado.

What is Mixture of Agents

A good way to use LLM as a productivity booster is to treat it like a rookie assistant.

A rookie with broad knowledge but unable to channel it if not asked specifically. Eager to do, but without supervision, you have no idea what quality the outcome will be. LLM, on the other hand, will be very confident that he did the best possible job.

That's why prompt engineering is "the thing," as it's our main source of control and guidance over what the model produces.

If you want the output to be good, you must iterate with the LLM.

However, we seek the model autonomy to perform the task by completing iterations according to a plan and knowing the goal and what was done earlier.

The approach that chains models, making them collaborate as a team and use the collective expertise of different LLMs, is called the Mixture of Agents.

How Mixture of Agents works

As you see in the illustration, we have a framework that goes as follows:

1. An input prompt is fed into the first layer - the proposer
2. The first layer consists of three models (A1,1; A1,2; A1,3)
3. Each of the models produces an output and feeds it forward to the next layer along with the initial prompt
4. The process repeats on layers two and three - also proposers
5. In the final layer - the aggregator - we've got one LLM that aggregates what has been done and produces a final output

Each agent on each layer receives the same input to answer and additional information (let's call it more context) produced by the previous layer.

The actual inspiration for this is a Mixture of Experts method highlighted in the previous chapter, but here, the experts within a model were replaced by full-fledged LLMs.

The paper that introduced the MoA approach presents how it enhances the output quality in the following illustration:

Additionally, MoA scored 65.1% on AlpacaEval 2.0 using only open-source LLMs, compared to GPT-4 Omni’s 57.5%.

Now, the first thought that pops up is, "alright, the performance is slightly better, but we have to run multiple LLMs to get the output." That's true, but benchmarks show it's actually more cost-efficient than frontier models, and the quality-to-cost ratio is much better.

Orchestrating agentic systems

Okay, so that was the theoretical part. Let's review a quick example of how such a system can be built.

First, what do we want it to do?

Complete a goal.

What does it need to complete a goal?

Agents that have the following qualities:

- a profile or a purpose that is well-defined so the other agents can call it whenever necessary

- a memory of passed-over messages and the collective memory of the goal and progress made toward it

- an ability to plan both the steps and the approach to achieve them

- toolkit and a set of actions that can be done with it

How can we execute it?

We can use one of the two most common patterns:

- centralized, where a "supervisor LLM" decides on steps and delegates tasks and tools based on the analysis of progress

- decentralized, where each agent can decide what to do, when to call other agents and when to end its part of the execution

For each pattern, three core elements - training, controller, and critic - most affect system performance.

Centralized pattern approach

Centralized Training involves training all agents using a shared set of data and a central controller that has access to all agents' global state and actions.

Centralized Controller is responsible for making decisions and distributing tasks to the agents. It collects information from all agents, processes it, and then issues commands.

Centralized Critic is a global value function that evaluates the overall system performance. The centralized critic updates the agents’ policies based on the joint actions and observations, leading to a coherent strategy across agents

Decentralized pattern approach

Decentralized Training occurs when each agent is trained independently with its local data and observations. Agents do not have access to the global state but rely on their perceptions and interactions with the environment. This setup is more scalable and can effectively handle dynamic and large-scale environments.

Decentralized Decision-Making is when each agent makes decisions autonomously based on its local information and objectives. This method reduces the communication overhead and increases the system's robustness since there is no single point of failure.

Decentralized Critic means that each agent has its own critic that evaluates its performance. This allows for a more flexible and adaptable system where agents can learn and adapt individually to their local conditions.

Final thoughts

The Mixture of Agents approach is very promising, and will definitely be adopted by many organizations.

How I like to think about them is that they will govern other AI systems, forming an ecosystem. As AI is much more than GenAI, various agents sitting on top of the data governance platform will aid upper-level management in:

- aggregating and redistributing data coming from various sources,

- feeding other parts of the system (e.g., Computer Vision systems),

- confronting the data with predictions (e.g., predictive maintenance systems connected with AI Quality Control systems and IoT readings),

- or decomposing data silos.

For more ML and AI insights, subscribe or follow Sparkbit on LinkedIn.

If you're looking to start an AI project, you can book a free consultation with our CTO here: https://calendly.com/jedrek_sparkbit/ai-consultation