Building Agentic Systems & LLM Applications the Right Way!

Welcome to the world of agentic AI systems! Today, we will dive into how to build these systems. This guide will walk you through the process of understanding how AI and Agentic systems work. We will explore the new world of multi-AI agent systems by creating a team of AI agents that can automate complex workflows. Let's also see how to build agentic systems and LLM applications the right way using highly efficient tools/platforms.

Understanding AI Agents

The AI agent market is predicted to grow by 45% annually, reaching $47 billion worldwide by 2030. Microsoft CEO Satya Nadella has even suggested that agentic applications could replace traditional SaaS.

Simply put, AI agents are specialized software systems that can perceive their environment, make decisions, and take actions autonomously.

Also, an AI agent can be understood as a computer program that can perform tasks or make decisions on its own, based on its goals and the information it receives.An AI agent will primarily have three main components - Perception, Brain and Action.

These three components work together: the perception gathers information, the brain figures out what to do, and the action carries out the plan. This process allows AI agents to perform a wide variety of tasks, from answering questions to controlling complex systems.

Imagine a team of digital workers, each with unique skills, collaborating to solve a complex problem. This is the essence of AI agents—working together through sophisticated communication protocols to achieve a common goal.

AI agents are not limited to interacting with a static “environment” on behalf of a user — they can also form dynamic multi-agent systems. Such system’s capabilities go far beyond what a single agent can do.

LLM Agents vs LLM Workflows

When it comes to agentic solutions, we talk about agents and workflows.

LLM agents and LLM workflows differ conceptually and operationally.

LLM workflows are predefined, static processes designed to perform specific, linear tasks. They operate based on a structured pipeline where each step is explicitly defined and executed in sequence, with little to no flexibility or adaptability.

For instance, as illustrated in Figure 2 above (right side), a typical workflow involves a chain of prompts using multiple LLMs, combined with a Retrieval-Augmented Generation (RAG) pattern for accessing domain-specific knowledge.

In this setup, one LLM might process the query to determine intent or refine context, while another LLM, equipped with retrieved knowledge, generates the final response. The workflow’s reliance on fixed steps ensures consistency but limits flexibility.

LLM agents stand apart due to their ability to reason, adapt, and refine their actions in response to changing environments and complex goals, making them well-suited for advanced, dynamic applications.

This distinction underscores the versatility and intelligence that define LLM agents as compared to traditional LLM workflows. Know more about the agentic systems in this paper.

Understanding Multi AI Agent Systems

Multi AI Agent systems represent a significant advancement in the field of artificial intelligence. These systems consist of various autonomous AI agents that collaborate to accomplish specific tasks or solve complex problems. Unlike single AI agents, which operate independently, Multi AI Agents work collectively, leveraging their unique capabilities to enhance performance and efficiency.

Multi-Agent RAG systems utilize multiple specialized agents, each with its own knowledge base. The user prompt is processed sequentially by these agents, with each one contributing its specific expertise.?

This cascade of agents allows for a high degree of customization and potentially better output quality, as each agent can focus on different aspects of the task.?

Multi-Agent systems offer more flexibility and can be configured for various use cases, making them adaptable to complex or specialized queries.

Key Characteristics of Multi AI Agent Systems

• Collaboration: Multiple agents interact and share information, allowing them to tackle challenges that would be difficult for an individual agent.
• Autonomy: Each agent operates independently, making decisions based on its programming and the data it processes.
• Specialization: Agents can be specialized for specific tasks, enhancing their efficiency and effectiveness in achieving their goals.
• Scalability: New agents can be added to the system as needed, allowing for easy scaling of operations.

Understanding Agentic Workflows

An agentic workflow is a system where autonomous agents work together in a coordinated manner to accomplish tasks, with each agent having specific responsibilities and some degree of independent decision-making capability. The term "agentic" comes from the concept of agency - the capacity to act independently and make decisions.

Breaking down the workflow diagram:

At the top level, we have an Actor who interacts with a Supervisor. The Supervisor acts as an orchestrator or manager for three agents (Agent 1, Agent 2, and Agent 3).

The workflow structure has several key components:

• Task Distribution:

Tasks 1-5 are distributed among the three agents

The Supervisor appears to coordinate which agent handles which tasks (shown by the blue arrows)

Each agent can handle multiple tasks (e.g., Agent 1 handles Task 1 and Task 3)

• Agent Layer:

Three agents work in parallel

They appear to be semi-autonomous but under the Supervisor's oversight

The agents can interact with various tools and data sources (shown by the arrows pointing downward)

• Tools Layer: The bottom section shows the resources available to the agents:

External Data - RAG (Retrieval-Augmented Generation) which includes:

• VDB (Vector Database)
• Document storage

Three main tools:

• Search capability
• Computational tools
• Other miscellaneous tools

This type of architecture is commonly used in AI and automation systems where complex tasks need to be broken down and handled by specialized agents, each with access to different tools and capabilities, while maintaining a coordinated workflow under central supervision.

The design follows a hierarchical structure that allows for both centralized control (through the Supervisor) and distributed execution (through the individual agents), making it suitable for handling complex, multi-step tasks that require different types of processing or expertise.

Companies Using AI Agents

Here’s a story about the five early users of AI agents:

1. At New Jersey-based Johnson & Johnson, AI agents are being used to help the healthcare giant with the chemical synthesis process in drug discovery.

2. AI agents are becoming key players in research at Moody’s, the New York-based financial analysis and software company. Moody’s agents are given specific instructions, personalities and access to data and research.

As a result, they can come to different conclusions, especially for complex topics like analyzing the financial fitness of a company that appears healthy, but is facing geopolitical risk.

3. EBay is using AI agents to help write code and create marketing campaigns. The company also plans to roll out agents that can help buyers find items and sellers list goods.

4. Telecommunications giant Deutsche Telekom, which has roughly 80,000 employees in Germany, has rolled out an AI agent for its employees to ask any question about internal policies and benefits, and for its service staff to ask questions about its products and services.

5. The Spanish company Cosentino, which makes countertop surfaces and other stone materials for homes and buildings, has brought on a “digital workforce” of AI agents to fill the gaps in its customer service staff.

Cosentino calls its agents digital workers because they’re treated that way—they need to come with a set of basic skills, but they also get training when they first arrive on the job. They’re also instructed to follow a strict process, and the company knows if they go off the rails.

Know more in this in-depth article.

Multi-Agent Application Example

A multi-agent application is a system where multiple #AI agents work together collaboratively to accomplish complex tasks that would be difficult for a single agent to complete. Each agent has specific roles and responsibilities, and they communicate and coordinate with each other to achieve common goals.

The Code Generation and Review Application, built using LangGraph and CrewAI frameworks, demonstrates this concept through a streamlined workflow for code development. The process begins when a user submits a question or coding task. This input is processed by a Node that initiates the code generation phase, where an agent creates the initial code solution.

The generated code then moves to another Node for code review and quality checking. At this point, a critical decision is made: if the code passes the review, it proceeds directly to become the final Code Solution; however, if any issues are detected, the code is sent back through a feedback loop to the Code Generation phase for improvements.

This cycle continues until the code meets the required quality standards. LangGraph manages the workflow and state transitions through its graph architecture, while CrewAI handles the coordination between different agents, ensuring efficient task allocation and collaboration. This automated approach enhances code quality through continuous review and improvement, making the development process more efficient and reliable.

Know more about multi-agent application implementation in this research paper.

Here is my step-by-step video on building multi-agent AI systems.

Bonus Use Case of a Multi-Agent System

CrewAI is an open-source framework designed to help developers build and orchestrate multi-agent AI workflows. It enables the creation of autonomous agents with specific roles, goals, and tools, allowing them to collaborate on complex tasks. The beauty of crewAI lies in its simplicity and effectiveness in managing interactions between agents.

Setup CrewAI

python -m venv crewai-env

source crewai-env/bin/activate? # On Windows use: crewai-env\Scripts\activate

pip install crewai

Now that we have crewAI set up, let’s outline the workflow we want to automate. Our goal is to create a multi-agent system that writes a book through various stages:

• Planning: Developing the book's concept, outline, characters, and settings.
• Writing: Drafting the chapters based on the outline.

Defining Agents

We will create five agents, each responsible for one of the stages mentioned above:

• Planning Agent: This agent will handle the development of the book's concept and outline.
• Writing Agent: Responsible for drafting the chapters based on the outline provided by the planning agent.
• Editing Agent: This agent will refine the written chapters for coherence and grammatical accuracy.
• Fact-Checking Agent: It will verify the factual accuracy of the content.
• Publishing Agent: Finally, this agent will format the manuscript for publication.

Let's start configuring our agents using CrewAI.

Below is my complete video on building multi-AI agent systems.

Here is the complete notebook code you can try.

Building LLM Applications Made Easy

During my exploration of LLM development tools, I encountered Arato - a platform that notably improved how I approach AI projects. Previously, I struggled with disorganized prompt iterations, inconsistent evaluation methods, and the tedious process of tracking experiments across spreadsheets. Arato addressed these pain points by offering a centralized workspace to systematically design and test prompts, analyze model outputs, and document iterations.?

What I appreciate most is its balance of structure and flexibility: the interface avoids unnecessary complexity while providing robust version control and data-driven insights. It’s particularly useful for identifying patterns in model behavior, which has reduced my debugging time and helped refine prompts more efficiently. While no tool eliminates all challenges, Arato has become a practical asset in my workflow—not by promising magic, but by streamlining the incremental, often messy work of developing reliable LLM applications.

Arato streamlines the process of managing data pipelines, running experiments, and visualizing results to ensure high-quality and reliable outputs. Its features include dataset, prompts, and model configuration management, input validation mechanisms, evaluation frameworks, and a powerful interface for comparing model runs and analyzing performance metrics.

Whether you're building a customer support bot, generating creative content, or optimizing decision-making models, Arato's collaborative environment helps teams iterate quickly and align AI capabilities with business goals. By bridging the gap between AI experimentation and real-world applications, the Arato Platform accelerates innovation and ensures effective, responsible, and impactful AI deployment.

Try Arato for Free!

Building effective agentic systems and LLM applications requires careful prompt management and experimentation. This is where platforms like Arato become invaluable. As we continue to explore the potential of AI Agents, it is clear that their ability to work together will redefine how we approach complex tasks in various industries. Embracing these technologies will pave the way for more intelligent and responsive systems in the future.

Also, having a robust data platform like SingleStore to handle not just the vector but any type of data for your AI applications is highly recommended. SingleStore also supports features like hybrid search, reranking, real-time analytics, semantic cache, millisecond latency, ultra fast ingestion, etc. Try SingleStore for FREE!