Multi-Agent AI Query System

Introduction

Recently, I set out to build a tool that could help me learn from both LlamaIndex and LangChain documentation, as well as search the web for related information. The idea was to create a system that would allow me to gather all this information in one place and then use AI to answer my questions based on that data. Here’s a look at how I did it and why I think it’s a useful approach.

Why I Built This Tool:

Sometimes, while working and learning, I found myself needing to see proper implementations from both LlamaIndex and LangChain. Switching back and forth between them was time-consuming, so I thought it would be more efficient to combine them with web search in one place. For personal use, I decided to build a system that could automatically gather information from both LlamaIndex and LangChain documentation, as well as related web content, and then help me find answers quickly using AI.

I was also talking with some students who were interested in #AI, #LangChain, and #LlamaIndex and were looking for a small gateway project to get started. This idea of creating something that could help both them and myself in my work seemed like a perfect solution.

What is an Agentic System in AI?

An agentic system in AI refers to a setup where autonomous agents operate within a defined environment to achieve specific goals. These agents are designed to perform tasks, make decisions, and interact with their environment (and sometimes with other agents) without human intervention.

Agentic systems can: Automate Complex Tasks, Scale efficiently to handle large datasets, Usually designed for specific roles or tasks

Type: There are several types of agentic system, few of them are as follows:

Reactive Agents: Reactive agents operate purely on a stimulus-response basis. They do not store past experiences or have memory. Use Case: Automated planning systems, etc.

Conversational Agents: These agents are specialized in handling human-like interactions. They use natural language processing (NLP) to understand and respond to user queries. Use Case: Chatbots, virtual assistants, etc.

Multi-Agent Systems: Here, multiple agents interact or collaborate to solve complex problems. Each agent may have different roles, and together, they work towards a common goal.

Hybrid Agents: Here we combine features of different types of agents. For example, an agent might be both goal-oriented and reactive,

Step-by-Step Guide

Colab Notebook: Google Colab Code Implementation

1. Setting Up the Environment

Why Python?: Python is straightforward and has all the tools we need. We'll use libraries like requests to fetch web content, BeautifulSoup to parse that content, and Streamlit to build a user interface.

Key Libraries:

1. requests: For fetching web pages.
2. BeautifulSoup: To help us clean up and extract the useful parts of those web pages.
3. Streamlit: Allows us to create a simple interface where we can input URLs and queries.
4. langchain: A framework that lets us build powerful AI applications using langchain agents.
5. Chroma: Stores the information we gather so that it can be easily searched later{can be any of the vectorstore}

2. Building the Web Crawler

• What is a Web Crawler?: Imagine you’re visiting a website, reading each page, and taking notes. A web crawler does this automatically, visiting pages, grabbing content, and moving on to the next link.
• Why Store the Data?: By storing the text from these pages, we can later search through it quickly without needing to visit the website again. This is especially helpful when we want to compare information from both LlamaIndex and LangChain documentation.

3. Using AI to Search the Data

• What are AI Agents?: Think of AI agents as smart assistants that can search through the data we've collected and answer questions based on that data.
• The Three Agents:You can use any i was experimented with these three, specifically using Zero-Shot React Agent
• Zero-Shot React Agent: Good for answering questions when you don’t need any prior knowledge or context.
• Conversational React Agent: Useful when you want to have a back-and-forth conversation, as it remembers what you’ve talked about before.
• Structured Chat Zero-Shot Agent: Best when you need detailed, structured answers.

4. Creating the User Interface with Streamlit

Why Use a UI?:

A user interface (UI) makes it easy to input URLs and queries without needing to write code every time. Streamlit is perfect for this because it’s simple and lets us build a web app with just a few lines of code.

How It Works:

• You input URLs you want to crawl, and the app gathers and stores the information.
• You can then ask questions, and the AI agents will search through the stored data to give you answers.

5. Managing the Server and ngrok

• Why Manage the Server?: During development, you might need to restart the server to apply changes. The app includes a feature to stop and restart the server easily.
• Using ngrok: ngrok allows you to share your local server with others by creating a temporary web address. This is useful if you want to access your app from different devices or share it with others.

Example:

Running the Application on Google Colab: Step-by-Step Guide

By following these steps, you’ll be able to set up and run the application in Google Colab

Step 1: Set Up Your Google Colab Environment

1. Open the ntebooks from below emntioned github repository in Google Colab

Code URL : Github

Google Colab URL: code implementation

Step 2: Set Your API Keys

Set Your OpenAI API Key: Replace "your_openai_api_key_here" with your actual OpenAI API key

Set Your ngrok API Key: Replace "your_ngrok_api_key_here" with your actual ngrok API key

Step 3: Run each cell in colab step by step

Expose the Server with ngrok:

• Start the ngrok tunnel to expose your Streamlit app to the internet:
• The URL printed will be your Streamlit dashboard where you can interact with the application.

public_url = ngrok.connect(8501)
print(f"Streamlit dashboard is live at: {public_url}")
        

Additional development if needed:

Work with Different Agents

• Using Different AI Agents: You can experiment with the different agents provided in the application (Zero-Shot React, Conversational React, and Structured Chat Zero-Shot). Each agent has a unique way of handling queries, so try them out to see which one works best for your needs.

Experiment with Different LLMs

• Switch Between LLMs: The application uses OpenAI's GPT-4 by default, but you can switch to other language models (LLMs) by modifying the ChatOpenAI instance in the code.Replace model="gpt-4" with the model of your choice.

Use Different URLs for Documentation

• Crawling Multiple Documentation Sources: Input different URLs for LlamaIndex, LangChain, or any other documentation you wish to include.The application can handle multiple URLs, crawling and storing the content from each.

Explore Vector Stores and Multi-URL Implementations

• Detailed Implementation with Different Vector Stores: The default vector store is ChromaDB, but you can experiment with other vector stores by adjusting the code. Add multiple URLs and compare how the AI agents perform when querying across different documentation sources stored in the vector database.

Conclusion:

Building this tool has not only streamlined my learning process but also given me deeper insights into how AI can be used to enhance knowledge discovery. If you’re looking to learn from multiple sources efficiently, I’d recommend trying a similar approach. It’s a great way to leverage the power of AI in your learning journey.

** This is in-progress work and sharing the basic implementation.

PS: Article thumbnail is generated using AI

Thank you for reading! ?? ?? Connect with me: Satyam's LinkedIn , Satyam's Github

Also, visit my blogs where I share my work implementations and learning to write: Satyam's Blogs

Thanks to everyone for teaching the concepts in details on youtube, blog-posts :);