Building LangChain ReAct Agents with create_json_chat_agent

LangChain offers a powerful way to create agents that use tools. This article focuses on using the create_json_chat_agent function, which allows agents to interact using JSON-based responses, a useful format for integrating tool-based actions. The goal is to show how to implement create_json_chat_agent using OpenAI, while tools are the main focus. In the previous article, we used ZeroShotAgent.from_llm_and_tools. In this article we will use create_json_chat_agent.

This article is a continuation of the previous article:

Key Components

1. Tools: Functions that agents can invoke for tasks like calculations or external queries.
2. LLMs: Although any model can work, we’ll use OpenAI for simplicity.
3. Prompt: The instructions the model uses to format responses as JSON.

Step-by-Step Implementation

1. Install Dependencies

First, make sure you have LangChain installed:

pip install langchain openai        

2. Define Tools

Create simple tools the agent can invoke. We will use the tool that we used in the previous article:

from langchain.tools import BaseTool

class last_name_for_Rany(BaseTool):
    name = "Last name for Rany"
    description = "Use this tool to get the last name of Rany"

    def _run(self, expression: str):
        return "ElHousieny"
      


last_name_for_Rany_tool = last_name_for_Rany()        

The description of a tool is essential because it tells the language model (LLM) how and when to use the tool correctly. Clear descriptions guide the LLM's decisions during task execution. In addition, prompt engineering is crucial for refining how the model interacts with the tools. By adjusting the prompt, developers can enhance the model's understanding and performance, especially in cases where the LLM might make mistakes or attempt to use tools inappropriately, like handling multiple tasks at once. This process helps improve overall accuracy and efficiency. So, we might go back and adjust the description as we start using the tool.

Let's test the tool:

last_name_for_Rany_tool.run("Rany")        

Now, add it to the tools:

tools = [last_name_for_Rany_tool]        

3. Create the Prompt Template

Prompt Engineering is very important. That is why I started my course with it. Please go back to my prompt engineering article, if needed.

We need a prompt that instructs the model on how to format responses as JSON:

system="""
You are tasked with solving problems step-by-step, responding with a JSON structure in markdown format.
The JSON should contain:
- thought: your reasoning
- action: the name of the tool
- action_input: the parameters for the tool

You can use the following tools: {tool_names}

Descriptions of these tools:
{tools}

If you have enough information, use the "Final Answer" tool with the solution as its input. Otherwise, continue using the available tools to gather more information.
"""

        

System messages are used to set the behavior, rules, or context for the conversation. They typically define the AI's role, instructions on how it should respond, and the boundaries within which it should operate. System messages guide the AI on how to interact throughout the conversation. They are not visible to the end-user in most interfaces and are primarily used to configure the AI's behavior. This prompt has several distinct sections, each serving a specific function in guiding the agent's behavior:

1. Purpose Overview: The agent is informed that its job is to solve tasks step-by-step. Responses must follow a structured format, using JSON to represent the decision-making process.
2. JSON Structure: The agent's responses must include:
3. Tool Definitions: The prompt provides placeholders for:
4. Decision Logic:

Purpose of the Sections:

• System Initialization: This section defines the structure and expectations, ensuring the agent knows how to format its responses correctly in JSON.
• Tool Descriptions: By defining the tools in advance, the agent can choose the right tool for the task based on the descriptions provided.
• Task Flow Control: The prompt includes decision-making logic, guiding the agent to keep iterating through tools until it has enough information to provide a final solution.

This format is designed to ensure that agents using the create_json_chat_agent function can interact with tools in a systematic and consistent way while maintaining clear reasoning behind their actions.

human="""
After each markdown snippet, include the word "STOP". Example:

```json
{{"thought": "<your reasoning>",
 "action": "<tool name or Final Answer to give a final answer>",
 "action_input": "<tool parameters or the final output"}}
```
STOP

This is my query="{input}". Write only the next step needed to solve it.

Remember to add STOP after each snippet.

"""        

In the context of LangChain and agents, the human message acts as the input provided to the agent. It defines the task or query that the agent needs to solve, outlining the structure for how the agent should respond. It’s essentially the instruction layer that helps guide the agent on how to interact with tools, process data, and return outputs.

Breakdown of the Human Message:

Explanation of the Parts:

1. Instruction for JSON Structure:

- "After each markdown snippet, include the word 'STOP'": This ensures that each JSON response ends with the word "STOP," which acts as a separator, allowing the agent to send and execute actions step-by-step.

2. JSON Example:

- This section shows how the agent should format its responses to compl with ReAct Agents. The structure includes:

- thought: Describes the agent's reasoning process.

- action: Specifies the name of the tool the agent will use or signals that the final answer is ready.

- action_input: The parameters or inputs needed for the tool or the final result.

3. Query Input:

- "{input}": This placeholder represents the actual query or problem posed by the user. The agent must process this query based on the available tools and previously gathered information.

4. Step-Based Instruction:

- "Write only the next step needed to solve it": Guides the agent to approach problem-solving incrementally, responding one step at a time, rather than jumping to a final answer immediately.

Why Is It Necessary?

- Structure & Clarity: The human message helps the agent follow a structured format, ensuring the use of JSON responses, which makes the agent's actions clear and traceable.

- Incremental Problem-Solving: By instructing the agent to solve tasks step-by-step, it prevents the agent from making assumptions and encourages reasoning for each action.

- Tool Guidance: By explicitly outlining how and when to use tools, the human message helps optimize the agent’s tool selection process.

In summary, the human message plays a critical role in ensuring that the agent interacts logically and incrementally while providing structured, predictable outputs.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        MessagesPlaceholder("chat_history", optional=True),
        ("human", human),
        MessagesPlaceholder("agent_scratchpad"),
    ]
)        

4. Initialize the Agent

We’ll use OpenAI's gpt-3.5-turbo model and create the agent:

from langchain_community.chat_models import ChatOpenAI
LLM = ChatOpenAI(model="gpt-3.5-turbo")  # Use OpenAI for simplicity        

from langchain.agents import create_json_chat_agent, AgentExecutor

agent = create_json_chat_agent(
    tools = tools,
    llm = llm,
    prompt = prompt,
    stop_sequence = ["STOP"],
    template_tool_response = "{observation}"
)        

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=True)        

agent_executor.invoke({"input": "What is the last name for Rany?"})
        

Let's check Galileo traces:

As you can see, the chain is longer. The previous image shows the high level part which includes the first input and the last output:

You can see the use of the tool in trace:

Here is a better look into the flow:

Conclusion

In this article, we demonstrated how to create a JSON-based chat agent using create_json_chat_agent in LangChain. We showed how to set up tools, define prompts, and initialize the agent using OpenAI. This framework enables agents to perform various tasks by leveraging external tools, making them highly adaptable for real-world applications.