Multi Agent Systems | Shared Persistent State

A significant amount of application architecture knowledge is required when we design generative AI based solutions. It becomes even more important when we are designing multi agent systems. This is probably the first time that machine learning and application architecture are coming together in a significant way. All these days, most organizations had separated application and machine learning architecture teams. Data scientists and ML engineers hardly cared about application architecture and application engineers never had to think of ML engineering process. But in the era of Generative AI, it is important to blend them together to design the most optimal solution architecture.

The motivation to write this article comes from a question that I faced recently

Why do we need a shared persistent state in a multi agent system?

The person who asked this question comes from a data science background and the concept of a shared persistent state comes from application architecture pattern; more precisely it is one of the famous patterns in an event driven architecture. We called it event sourcing. To understand the role of a shared persistent state, let us first go back in the past to refresh our memory on Event Sourcing.

Event sourcing is a design pattern where state changes are logged as a sequence of events. These events are immutable and appended to a log, which serves as a single source of truth for the system.

A Quick Refresher on Event Sourcing

Event sourcing is a design pattern where every state change in an application is logged as a sequence of events. These events are immutable, meaning they cannot be altered once recorded, and they are stored in an append-only log that serves as the system's single source of truth.

In most applications, data is central. If data could manage itself, we wouldn’t need complex applications to interact with it. Typically, we manage data through CRUD (Create, Read, Update, Delete) operations. However, this approach has a significant drawback: it only shows us the current state of the data, losing the entire history of how it evolved to that point. Event sourcing solves this problem by recording every operation and state change in an event store, allowing us to replay these events to reconstruct the current state. This capability is crucial in regulated industries where we need to audit and explain the decisions made based on the data.

Shared Persistent State in Multi-Agent Systems

In a multi-agent system, a shared persistent state is essentially an implementation of the event sourcing pattern, but with added functionality. Beyond just providing an audit trail, it serves as a mechanism for passing context from one agent to another, enabling seamless collaboration between the agents within the system. Additionally, it allows for the replaying of events, giving us insights into how the agents responded to specific inputs. This feature will likely become integral to LLM observability products, enabling developers to trace and optimize the behavior of multi-agent systems.

Both LlamaIndex and LangGraph have already implemented the concept of shared persistent state. In LangGraph, for example, developers can define the data structure of the shared state, while LlamaIndex manages it through a context object that persists across workflow steps.

Designing the Shared Persistent State

When architecting a multi-agent system, designing the structure of the shared persistent state is critical. It must be extensible, allowing for the addition of new agents without requiring a complete overhaul of the existing structure. Performance is also a key consideration; the data structure needs to be efficient for both read and write operations.

As multi-agent systems evolve, it's clear that architects have a significant role to play. While innovation is essential, we should also draw on the best practices and lessons learned from existing application and data architecture patterns. We, the architects have a lot to do to in the coming days to design the applications and the business processes of the future. I imagine reliving the days of "monolithic to microservices" paradigm that happened few years back. But now it will be the evolution to an "Agency" architecture that will bring together the semantic and syntactic world to expand the horizon of automation.