The Power of AI Agents: A Practical Guide to Building Smarter, Autonomous Systems

Welcome to the latest edition of #AllThingsAI newsletter. This edition is the 2nd part of our comprehensive series on #AIAgents, where we are discussing the different levels of agentic behaviour and the careful considerations while building them.

If you find the article thought provoking, please like the article, share your perspective in comments, and repost to spread the AI knowledge.

In recent months, one question has surfaced repeatedly among AI practitioners, developers, and enthusiasts: What exactly is an agent? The rapid evolution of large language models (LLMs) has brought about systems that are increasingly capable of reasoning, decision-making, and interacting with external sources of data in ways that seem almost human-like. These systems, commonly referred to as "agents," represent a key shift in how we think about artificial intelligence. But what makes a system agentic, and why does this matter for the future of AI?

What is an Agent?

It is a frequently asked question to define what an agent is. Simply put, an agent is a system that uses an LLM to control the flow of an application. This can range from simple decision-making, such as routing data between two paths, to complex autonomous behaviors that require iterative processes and dynamic adaptation.

An agent is a system that uses an LLM to decide the control flow of an application.

But here’s where it gets tricky: the term agent conjures different ideas depending on who you ask. To some, an agent is synonymous with advanced, autonomous AI—systems that function like robots capable of handling complex tasks on their own. For others, an agent might be as simple as a system that uses an LLM to choose between two different actions.

So, what’s the right answer? In truth, there’s no universally accepted definition of what an agent is. Instead, we should focus on the degree to which a system exhibits agentic behavior. This idea of degrees of agentic capabilities is something Andrew Ng aptly highlighted in a recent tweet: just like self-driving cars have levels of autonomy, LLM-based systems also exist on a spectrum of autonomy, or agentic capabilities.

The Spectrum of Agentic Behavior

So, what does it mean to be agentic?

In simplest terms, a system becomes more agentic as the LLM plays a larger role in determining its behavior. The more decisions the LLM makes—whether it’s about routing data, determining next steps, or even executing tasks autonomously—the higher it ranks on the agentic spectrum.

Let’s break it down:

1. Router: A system where an LLM directs inputs to downstream workflows exhibits some agentic behavior. For instance, if an LLM routes between two or more predefined paths, it shows a level of decision-making but lacks complex reasoning.
2. State Machine: If the system takes multiple routing steps and loops until a task is completed, it enters the realm of state machines. This adds complexity, allowing the system to handle tasks that require ongoing decisions and dynamic feedback.
3. Autonomous Agent: Systems like Voyager, which build tools and apply them to future steps, sit at the top of the spectrum. Here, the agent not only makes decisions but also adapts, evolves, and creates tools on its own. These are the most "agentic" systems, with high levels of autonomy and control.

Why Does Being Agentic Matter?

When building LLM-based systems, it’s important to understand where your system sits on the agentic spectrum. Why? Because how agentic your system is will directly influence the complexity of its development, the tools required to manage it, and the strategies needed for testing, monitoring, and scaling.

Let’s consider a few key factors:

• Development Complexity: The more agentic your system is, the more difficult it becomes to design and run. As agentic systems involve complex decision-making processes and dynamic feedback loops, developers need robust orchestration frameworks to keep everything running smoothly. A simple router may not need much, but a fully autonomous agent requires deep control over branching logic, cycles, and decision flows.
• Durability and Execution: As systems become more agentic, they often involve tasks that take longer to complete, increasing the need for durable execution environments. A system that is midway through a task shouldn’t fail or break if an error occurs—it should be resilient enough to recover, making error-handling and background job execution critical.
• Interactivity and Observation: Agentic systems can behave unpredictably, making it important for developers to have visibility into what’s happening at every step. You need tools that allow you to observe, modify, and nudge an agent back on track if it deviates from its intended behavior.
• Evaluation and Monitoring: Given the inherent randomness of LLMs, agentic systems need robust evaluation frameworks that test not only the final outcome but also the intermediate steps. The more agentic the system, the more these intermediate evaluations become critical to understanding the efficiency and efficacy of the agent's decisions. Monitoring should be dynamic, allowing for queries based on specific steps or behaviors of the agent.

A New Era Requires New Tools

As LLMs become more central to AI systems, and as we push the boundaries of what these systems can achieve, it’s clear that traditional tooling won’t suffice. Highly agentic systems require specialized infrastructure—this is where tools like LangGraph and LangSmith come in. LangGraph serves as an agent orchestrator that helps developers build, run, and interact with LLM-driven agents, while LangSmith provides a testing and observability platform tailored to the needs of these sophisticated systems.

What’s truly new here is the need to rethink how we support and scale these increasingly agentic systems. Pre-LLM tools and infrastructure weren’t built with this level of autonomy or unpredictability in mind. As we move further into the agentic era, the ecosystem surrounding LLMs must evolve to keep up.

What Does the Future Hold for Agentic AI?

The rise of agentic AI is opening up a world of possibilities, from autonomous tools that build and improve upon themselves, to decision-making systems capable of tackling problems without human intervention. But this also raises important questions about the role of human oversight in increasingly autonomous systems. How do we maintain control and ensure that agentic AI behaves in ways that align with our goals? How do we address the inherent risks of systems that can operate in unpredictable ways?

As we explore these new frontiers, it’s critical for developers, researchers, and organizations to consider where they want their systems to fall on the agentic spectrum. Should AI agents remain simple routers, or is it time to push for full autonomy?

Your Thoughts

Where do you think the line should be drawn in agentic AI? Should we strive for more autonomous agents, or is there value in maintaining human oversight at critical points? Share your thoughts in the comments below. I’d love to hear where you stand on this exciting—and sometimes controversial—evolution in AI. ??

Found this article informative and thought-provoking? Please ?? like, ?? comment, and ?? share it with your network.

?? Subscribe to my AI newsletter "All Things AI" to stay at the forefront of AI advancements, practical applications, and industry trends. Together, let's navigate the exciting future of #AI. ??