Multi-Agent Collaboration for Long-Context Tasks: The Chain-of-Agents(CoA) Approach

Disclaimer:?the opinions I share are solely my own and do not reflect those of my employer.

Have you ever tried to read a long book or article and found yourself forgetting details from the beginning by the time you reached the end? That's similar to a problem that computers, specifically Large Language Models (LLMs), face when dealing with long pieces of text. These models are great at understanding and generating human-like text but have trouble with very long inputs. The Chain-of-Agents (CoA) approach is a new method that helps LLMs overcome this challenge, as discussed in the paper below.

In the paper "Chain of Agents: Large Language Models Collaborating on Long-Context Tasks," researchers from Penn State University and Google Cloud AI Research present a new framework called Chain-of-Agents (CoA). This framework uses multiple agents that work together to handle long-context tasks more effectively. By collaborating, these agents can break down complex tasks into smaller parts, making managing and improving overall results easier.

What's the problem with long text for LLMs?

LLMs have a limited "context window," meaning they can only process a certain amount of text simultaneously. When the input is too long, they have to either:

• Truncate it: Cut off the text's beginning or end, meaning they might miss important information.
• Use Retrieval Augmented Generation (RAG): Break the text into chunks and try to retrieve only the most relevant parts, which can also lead to missing key details.
• Extend the context window:?Some models try to handle more text, but even these models struggle to focus on the relevant information in very long texts.

How Chain-of-Agents (CoA) Works

The Chain-of-Agents (CoA) framework tackles this problem by using multiple LLM "agents" to collaborate to process the long text cleverly. Instead of trying to read the entire document simultaneously, CoA breaks it into smaller parts and assigns each to a "worker" agent. These agents then work together in a sequence, like a team passing a baton:

• Worker Agents:?Each worker agent reads a portion of the text and message from the previous worker and extracts the key information. This information is summarized into a message passed to the next worker in the chain. This way, each agent builds upon the previous one's understanding.
• Manager Agent: Once the last worker agent has processed its text portion, the final message is passed to a "manager" agent. The manager agent takes all the information gathered by the worker agents and generates the final answer, summary, or code completion.

This differs from previous methods, which either try to reduce the input or extend the context window. The CoA approach is inspired by how people interleave reading and processing extended contexts under our limited working memory constraints.

Benefits of Chain-of-Agents:

• Improved Accuracy: CoA makes sure that all parts of the long text are processed, leading to better results than methods that might miss important details.
• Better Reasoning:?CoA enables complex reasoning across the extended context by having multiple agents work together. The agents can build on each other's work, and later agents can use information from earlier ones.
• Cost-Effective: It's less computationally expensive than feeding the entire long document into an LLM, as the text is processed in chunks. The time complexity is reduced from O(n2) to O(nk), where n is the number of input tokens and k is the context limit of the LLM.
• Works with different LLMs:?CoA is flexible and can be used with various LLMs. It is also task-agnostic, meaning it can be applied to answering questions, summarizing, and completing code.
• Overcomes the "Lost-in-the-Middle" Issue: Traditional LLMs often struggle to retain information in the middle of long documents. However, CoA's sequential agent communication helps preserve information flow, improving accuracy.

In simpler terms:

Imagine you're making a cake. You wouldn't try to bake the whole cake at once. Instead, you'd break it into steps: get the ingredients ready, mix them, bake them, and then decorate them. CoA does something similar with long texts. It breaks them into smaller pieces and uses "agents" to process each piece sequentially so that a final manager can complete the task.

So, CoA is like a team of readers working together to understand long texts and answer questions correctly and efficiently... It's a new and improved way for computers to handle long stories and do it better than before.

Limitations

While the Chain-of-Agents approach is a significant step forward, it also has some limitations:

• Agent Communication: The way agents communicate could be improved for more efficiency.
• Information Loss: Some information might get lost as it's passed from one agent to the next.
• Slower than RAG: Because of the sequential processing, CoA is currently slower than RAG.
• Limited to text: Currently, CoA is limited to text-based tasks and is not integrated with vision or multimodal models.

Conclusion

The Chain-of-Agents approach is a promising framework for processing long texts with LLMs. Using multiple collaborating agents can overcome traditional approaches' limitations, achieving better accuracy, improved reasoning, and greater efficiency. As research progresses, we can expect to see further improvements in how agents communicate, further optimizing this approach for various tasks. I recommend going through the paper for further insight into CoA,