Getting started with Langchain and LLMs

Large Language Models are in the technology spotlight – and for good reason. In this set of posts, I’m going to show how LLMs and Python can change the way we approach a lot of problems.

Now, let’s have some fun.

LangChain Part 1

LangChain is?a framework specifically built for developing applications and data pipelines that use LLMs. LangChain is an open-source project developed by Harrison Chase, and was first released in October 2022. Its uses include?chatbots, summarization, and querying documents.

In this post we will look at using LangChain to set up a conversational chatbot, run templated queries, and develop provide some fun grammatical translation capabilities.

Getting started

Refer to the previous post for setting up OpenAI: https://www.dhirubhai.net/pulse/exploring-large-language-models-mark-killmer/

Before we get started in this post, the LangChain and openAI libraries will be used. You may need to install them:

Don’t forget – just like in the previous post we need to set our OpenAI license:

import os
os.environ["OPENAI_API_KEY"] = 'sk-xxxxxxxxxxxx'        

Simplified Chatbot programming

One of the key features of LangChain is to simplify and improve chatbot features. In the following example, we use the LangChain ConversationChain and ConversationBufferMemory to write a simple conversation prompt:

from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
chat = ChatOpenAI(temperature=0)
conversation = ConversationChain(
? ? llm=chat,
? ? memory=ConversationBufferMemory(),
? ? verbose=False
)
conversation.run("Alice has a parrot. ")        

The LLM response should be something like:

'Interesting! What kind of parrot does Alice have?'        

In our code snippet we created an LLM model, named chat, and a conversation object, named conversation. Context memory for the conversation is provided using ConversationBufferMemory.

ConversationBufferMemory is a memory mechanism that stores every chat interaction into a buffer which is used to provide context to the LLM.

To add dialog into the conversation we use the run function.

If we continue the conversation:

conversation.run("Alice also has a black cat. What animals does Alice have? ")        

We should get something like:

"So Alice has a parrot and a black cat. Those are the only animals I know of that Alice has. Would you like me to look up more information about Alice's pets?"        

We can see the context from the first interaction ( “Alice has a parrot”) remains, without us manually saving aspects of the conversation.?

We can look under the hood to see what is happening if we set the Verbose parameter to True:

from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
chat = ChatOpenAI(temperature=0)
conversation = ConversationChain(
? ? llm=chat,
? ? memory=ConversationBufferMemory(),
? ? verbose=True
)
conversation.run("Alice has a parrot. ")        

With Verbose enabled, the output follows:

> Entering new ConversationChain chain..
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:

Human: Alice has a parrot. 
AI:

> Finished chain.

Out[1]:
'Interesting! What kind of parrot does Alice have?'.        

At this point, we can see “Current conversation:” is Blank. When we run the next statement...

conversation.run("Alice also has a black cat. What animals does Alice have? ")        

We can see:

Prompt after formatting
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: Alice has a parrot. 
AI: Interesting! What kind of parrot does Alice have?
Human: Alice also has a black cat. What animals does Alice have? 
AI:

> Finished chain.

Out[2]:
"So Alice has a parrot and a black cat. Those are the only animals I know of that Alice has. Would you like me to look up more information about Alice's pets?":        

?

Now we can see “Current conversation:” holds the previous conversations, made up of the human and AI generated text.

ConversationBufferMemory ?is used to store the “Current conversation:”? which in turn provide context. As we continue questioning, each subsequent conversation will be appended until the token buffer is reached. ?

If we set the Verbose flag back to false we can run a simple loop to mimic chatGPT:

from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
chat = ChatOpenAI(temperature=0)
conversation = ConversationChain(
? ? llm=chat,
? ? memory=ConversationBufferMemory(),
? ? verbose=False
)
while True:
? ? myInput=input(">:")
? ? if myInput=="":
? ? ? ? break
? ? print(conversation.run(myInput))        

Our "chatGPTmimic" output:

>:what color is a fire truck?
A fire truck is typically red in color, although some fire departments may use different colors or designs. The red color is meant to make the truck easily visible and recognizable as a fire truck.
>:what else is often that color?
The color red is often associated with danger, warning, and urgency. Other things that are often red include stop signs, emergency exit signs, warning labels, and some types of emergency equipment. Additionally, many fruits and vegetables are red, such as apples, strawberries, and tomatoes.        

LangChain Tempates

Templated queries allow us to generate a standardized prompts to query the Large Language Model. Similar in concept to an SQL Stored procedure, a templated query accepts parameters and runs a pre-saved query.

Here's an example:

from langchain import OpenAI
from langchain import PromptTemplate
llm = OpenAI(model_name="text-davinci-003")
template = "What are 3 good and funny one word names for a company that makes {product} ?"

prompt = PromptTemplate(
? ? input_variables=["product"],
? ? template=template,
)
completion = llm(prompt.format(product="running shoes"))
print(completion)        

The response should be something like:

1. Strides
2. Joggers
3. Pacesetters        

We can re-run the templated query with a different parameter:

completion = llm(prompt.format(product="dog food"))
print(completion)        

Now the output is:

1. Chowhound
2. MuttMeal
3. KibbleKart        

This abstraction from the original query allows a prompt engineer to craft the query, in the same way a DBA might craft a stored procedure!

Few shot prompt Template

A further extension of the prompt template is the Few Shot prompt Template. By providing just a few examples, the model can generalize and produce similar results.

Here’s an example of a Few Shot Prompt Template:

from langchain import OpenAI
from langchain import PromptTemplate, FewShotPromptTemplate
llm = OpenAI(model_name="text-davinci-003")
examples = [
? ? {"source": "daleks", "translated": "Skaro"},
? ? {"source": "vulcans", "translated": "Vulcan"},
]
example_template = """
Source: {source}
Translated: {translated}\n
"""
example_prompt = PromptTemplate(
? ? input_variables=["source", "translated"],
? ? template=example_template,
)
few_shot_prompt = FewShotPromptTemplate(
? ? examples=examples,
? ? example_prompt=example_prompt,
? ? prefix="Given the source",
? ? suffix="source: {input}\translated:",
? ? input_variables=["input"],
? ? example_separator="\n",
)
completion = llm(few_shot_prompt.format(input="ewoks"))
print(completion)        

The LLM successfully returns :

Endor        

In this code sample, the examples for the prompt are 2 science fiction species (daleks and vulcans) as a source, and their home planet as the translation (Skaro and Vulcan). The prompt is then integrated into a FewShotPromptTemplate.

To confirm the prompt is working, we can try:

completion = llm(few_shot_prompt.format(input="Thor")
print(completion))        

The LLM returns:

Asgard        

(this is a great example of flexibility as we have moved from species to characters!)

Let’s refactor this into a function and try some examples:

from langchain import PromptTemplate, FewShotPromptTemplate
def fewShot(examples,fewShotInput):
? ? example_template = """
? ? Source: {source}
? ? Translated: {translated}\n
? ? """
? ? example_prompt = PromptTemplate(
? ? ? ? input_variables=["source", "translated"],
? ? ? ? template=example_template,
? ? )
? ? few_shot_prompt = FewShotPromptTemplate(
? ? ? ? examples=examples,
? ? ? ? example_prompt=example_prompt,
? ? ? ? prefix="Given the source",
? ? ? ? suffix="source: {input}\translated:",
? ? ? ? input_variables=["input"],
? ? ? ? example_separator="\n",
? ? )
? ? completion = llm(few_shot_prompt.format(input=fewShotInput))
? ? print(completion)        

Our examples and our function call:

examples = [
? ? {"source": "Spandau Ballet", "translated": "new romantic"},
? ? {"source": "david bowie", "translated": "glam rock"},
? ? {"source": "elvis", "translated": "Rock and roll"},
? ? {"source": "B-52s", "translated": "New wave"},
]
fewShot(examples,"Backstreet Boys")        

Successfully returns:

Pop        

In this example, we can "translate" why people are famous:

examples = [
? ? {"source": "Frank Gehry", "translated": "Designing the Walt Disney Concert Hall"},
? ? {"source": "Leonardo Da Vinci", "translated": "Painting the Mona Lisa"},
? ? {"source": "Neil Armstrong", "translated": "First person to walk on the moon"},
? ? {"source": "William Shakespeare", "translated": "Wrote Romeo adn Juliet"},
? ? {"source": "Florence Nightingale", "translated": "Founded modern nursing"},
]
fewShot(examples,"Marie Curie")        

The LLM returns

Discovered radium and polonium        

Because GPT3 has been trained on such wide variety of inputs,?we can use the LLM for surprising Few Shot Queries - In this example we remove all vowels:

examples = [
??? {"source": "happy", "translated": "hpp"},
??? {"source": "difficult", "translated": "dffclt"},
??? {"source": "experimental", "translated": "xprmntl"},
??? {"source": "finally", "translated": "fnll"},
??? {"source": "elephant", "translated": "lphnt"},
??? {"source": "aardvark", "translated": "rdvck"},
]
fewShot(examples,"beautiful")        

The LLM correctly returns:

btfl        

Try something of your own, see what works and doesn’t work.

That wraps up this post - I hope to see you next time where we will load up documents and create a chatbot!