Build your first Chat GPT like app[Beginner]

Introduction:

As the world moves at a rapid pace, why shouldn't you keep up? You can either be left behind or ride the wave—the choice is yours. I'm talking about AI, which is making headlines every day. We should all feel honored and proud to witness this technological revolution. Who knows, you might be sharing stories with your children or grandchildren about the dawn of AI.

Putting stories aside, why not dive in and get hands-on by building a quick, small app using an LLM (the core of ChatGPT or Gemini)?

Are you ready? Okay lets do it and yes you will be able to launch your app in 10 mins.

I will be using Langchain framework to build apps powered by LLMs

Pre-requisite:

You must have basic understanding of

• Python programming
• Background on AI and its ecosystem(If you are not aware of LLM, Prompt, RAG etc then please refer to this article)

Code Repo:

You can clone the repo and

https://github.com/gauravsinghaec/langchain-llm-apps

I will be using a tool Poetry to manage my python dependencies.

• Clone the repo and cd into the cloned directory
• Install Python 3 , pipx and the Poetry tool (which requires pipx).
• Install the app dependencies using poetry as below:

poetry install        

• to add any python package using poetry use

poetry add <package name>        

Development:

I will divide this article into 3 sections:

1. Launching a simple app powered by LLM(here open AI model) -- you can find the code on main branch
2. Build a RAG app (What is RAG? please check above article) -- you can find the code on langchain-rag branch
3. Build an LLM app using an Agent -- you can find the code on langchain-agent branch

• Since I will be using OpenAI api for this app, request you to signup and generate your OpenAI secret key here.

1. Building a simple application with LangChain

We will be using LangChain framework to build the application powered by LLMs.

LangChain is a framework for developing applications powered by large language models (LLMs).

To build my first app, I will need to

• set up relevant env variables. For example,

export OPENAI_API_KEY="sk-..."        

• use an LLM model,
• create a prompt
• build the chain and
• finally serve the app with LangServe. It helps developers deploy LangChain chains as a REST API.

Below is the 30 lines of code to launch your first app.

Using an LLM model

Now you need to set an environment variable to store you OpenAI secret key so that we can use the same in our app as shown below:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI()        

The above line of code assumes that your OpenAI API key is set in your environment variables. You can explicitly mention it otherwise as you can see in my code snippet. You can also chose which model you wanna use e.g you can mention the latest open ai model gpt-4o (based on your open ai subscriptions)

llm = ChatOpenAI(model="gpt-4o")        

Creating the Prompt:

You can play around with this template, e.g lets say if you

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are the world database, the keeper of all the knowledge."),
    ("user", "{input}")
])        

The {input} variable gets its value from the UI once you launch this app. What if I change this template to below

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are the famous fitness coach and should guide people and asnwer their query related to fitness"),
    ("user", "{input}")
])        

You can try yourself and see if the answers to you query is same in both or not.

Build the chain:

Chains refer to sequences of calls - whether to an LLM, a tool, or a data preprocessing step.

User input is used to build the prompt -> that goes to LLM -> the output of LLM can be returned to user as it is or parsed in nicer format.

chain = prompt | llm | output_parser        

Create the app:

from fastapi import FastAPI

app = FastAPI(
  title="LangChain Server",
  version="1.0",
  description="A simple API server using LangChain's Runnable interfaces",
)        

Define the runnable:

from langserve import add_routes

add_routes(
    app,
    chain
)        

Run the app

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="localhost", port=8000)        

Uvicorn is an asynchronous server gateway interface(ASGI) implementation for Python app. You might have use WSGI implementations earlier for python as Django and Flask both utilize WSGI

Launch the app:

Run below code from the repo

> poetry run python -m main        

Here is your first app running and you can access this at https://localhost:8000/playground/

Go and launch your first LLM app and do not stop there but go and see what all you can do with it.

You can also see the docs which LangServe exposes as REST API abount LangChain here https://localhost:8000/docs

Here completes your 10 minutes... If you want, you can go ahead or stop here, grab some tea, coffee or whatever and enjoy.

2. Building a RAG App with LangChain

[You can find the code in langchain-rag branch]

As I have explained RAG in the other article here. You can bring your own data to add it in the context of LLM so that the query can answers question specific to your usecase.

So basically here a new component is called retriever, which we will build using LangChain and is added to to chain (which answers your query).

retrieval_chain = setup_retrieval | prompt | llm        

and then add this chain to route

# Adding chain route
add_routes(
    app,
    retrieval_chain,
)        

To build the RAG app, I just have to add the retriever and other steps are same as in the first LLM app like below

• use an LLM model,
• create a prompt
• build the chain and
• finally serve the app with LangServe. It helps developers deploy LangChain chains as a REST API.

Building Retriever:

Lets build the retriever. Here I am trying to load the news mentioning my company Accern's latest partnership deals from web.(Its called web scrapping)

Extract(Load)

we need a data source( can be a file or some URL or database).

Check loaders for different sources here..

from langchain_community.document_loaders import WebBaseLoader

loader = WebBaseLoader(
    "https://finance.yahoo.com/news/accern-emerges-premier-nlp-leader-124500667.html"
)
docs = loader.load()        

Transform: Then we need to transform that data into embeddings so that LLM can use it to find the answers in it.

from langchain_text_splitters import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
documents = text_splitter.split_documents(docs)        

Load or Embed(in this context): Finally load the data into a vector DB

Here we are using FAISS as vector store, it is a library — developed by Facebook AI. Given a set of vectors, we can index them using FAISS and search for the most similar vectors within the index.

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
vector_db = FAISS.from_documents(documents, embeddings)        

retriever = vector_db.as_retriever()        

I am using OpenAIEmbeddings which is an embedding model from Open AI to calculate the embedding for my documents. You can pass your own embedding model from open ai

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")        

You can see the same in above ETL diagram where we are building a vector db.

Prompt:

There is a new variable in this prompt called {context} which comes from the retriever.

retrieval_prompt = ChatPromptTemplate.from_template(
    """Answer the following question based only on the provided context,if you dont know the answer, please think outside of context then answer.:
    <context>
    {context}
    </context>
Question: {input}"""
)        

I have used some formatter to do basic formatting , you can change as per your owns but other remains same. Here is the full code .

Launch the app:

> poetry run python -m main        

Here you go... https://localhost:8000/playground/

3. Build an LLM app using an Agent and Tools

So far, we have seen that we need build a chain which is basically a sequence of steps which is like below:

retrieval_chain = setup_and_retrieval | retrieval_prompt | model | output_parser        

Before we go to this, lets try to understand few term here.

Agents is the brain of the most of the real world LLM apps. It uses another LLM(s) under the hood to decide the sequence of steps (as opposed to chain where the steps are predefined) and which tools to use to answer the user query.

The core idea of agents is to use a language model to choose a sequence of actions to take. In chains, a sequence of actions is hardcoded (in code). In agents, a language model is used as a reasoning engine to determine which actions to take and in which order.

Tools are interfaces that an agent can use to interact with the world. Now as shown below in the green boxes, the tool can be a simple function or a retriever or built-in tool(made for specific tasks) to get the answers of user query.

Let's take a real-world example: imagine you're playing a game of Who Wants to be a Millionaire (known as KBC in India). You have a lifeline called "Phone a Friend," and you've asked your best friend (the Agent) to be ready to assist you with the right answers when needed. When you need help with a question, you read it to your friend (the Agent), who must listen carefully and provide the correct answer. They can either answer from their own knowledge or use any books, newspapers, or other resources (Tools) to find the correct answer quickly. Your friend needs to think fast and accurately to help you win.

This analogy helps illustrate the concept of an Agent and Tools.

Building Tools:

Lets build some tools and see how it works. I have created 4 tools for my example

• A retriever for the latest Accern's partnership
• a built-in tools to get the answers from stackexchange
• another built in tool to get the answers for graphql query and
• a custom function to count the character in the text.

While building any tools we need to make sure(consider your character counter) we have to describe function, the argument to function(schema for input to func) etc in a way which helps the Agent while deciding which tool to use when user asks a question.

Retrieval Tool:

First, let's set up a tool for the retriever we just created in RAG app:

retriever_tool = create_retriever_tool(
    retriever,
    "accern_news_search",
    "Search for information about Accern. For any questions about Accern's latest partnership,if you dont know the answer, please think outside of context then answer",
)

        

Built in tools

from langchain.agents import load_tools

built_in_tools = load_tools(
    ["graphql", "stackexchange"],
    graphql_endpoint="https://swapi-graphql.netlify.app/.netlify/functions/index",
)        

Custom Tools

from langchain.tools import StructuredTool
from langchain.pydantic_v1 import BaseModel, Field


def get_character_count(input_str: str):
    return len(input_str)


class GetCharacterCountInput(BaseModel):
    """
    Pydantic arguments schema for get_character_count method
    """

    input_str: str = Field(..., description="The text to be counted for")

char_count_tool = StructuredTool.from_function(
    func=get_character_count,
    args_schema=GetCharacterCountInput,
    description="Function to count the number of charaters in a text.",
)

        

Lets merge all the tools together now that we want to work with.

tools = [retriever_tool, char_count_tool] + built_in_tools        

Building Agent:

from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_openai_tools_agent

prompt = hub.pull("hwchase17/openai-functions-agent")
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
agent = create_openai_tools_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)        

Adding the agent executors in route

class Input(BaseModel):
    input: str


class Output(BaseModel):
    output: str


add_routes(
    app,
    agent_executor.with_types(input_type=Input, output_type=Output),
)

        

Lets now look at the app and ask some questions.

• Lets see the 1st question and we can see the agent has chosen the custom function to answer this question.

• Now lets check another tool

Meanwhile you can see the response to your query in the CLI as well.

You can go over the whole list of built in tools by langchain or build your own tool or agent.

I hope this will encourage you to make your hand dirty and build some cool apps. You can go through LangChain documentation to build more cool stuffs.

Happy Reading - Gaurav

#LangChain #LangServe #AIDevelopment #LLMApp #OpenAI #RAGApp #TechInnovation #MachineLearning #AIEngineering #NaturalLanguageProcessing #AIRevolution #BuildWithAI #AIApplications #TechCommunity #AIProjects #InnovationInTech #AIFrameworks #FAISS #VectorDB