Exploring Large Language models

Large Language Models are in the technology spotlight – and for good reason. In this set of posts, I’m going to show how LLMs and Python can change the way we approach programming and problem solving. These posts will be bite sized, focusing on a single aspect.

Now, let’s have some fun.

Getting started with LLMs + Python

First things first

To follow along with this code, you will need to have an API Key from openAI.com. If you don’t have an OpenAI key, you’ll need to create an account at https://openai.com and then select “Create new secret key”. Save your OpenAI key in a safeplace, we will need it in a second.

OpenAI is not expensive, but it is not free. Writing this post used approximately $0.40 in OpenAI token charges.

Alternatively, you can modify this code to use one of the free models available - if you do, let me know – I’d like to see your results!

The ?OpenAI library will be used so you may need to install it:

pip install openai # the OpenAI library         

We have… words!

To get started we will use the "text-davinci-003" GPT-3 engine and make sure everything is running OK.

import openai
# Set the OpenAI key - this is where you will need to insert yours# To get started we will use the text-davinci-003 (GPT-3 Model)
openai.api_key? =? "<Your OpenAI API key>"

# We will use the "text-davinci-003" GPT-3 engine and
# look at the different engines later in this post

model_engine='text-davinci-003'
prompt = "Alice has a bird and a dog. How many pets does Alice have?"
completion = openai.Completion.create(engine=model_engine,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? prompt=prompt,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? max_tokens=1024,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? stop=None,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? temperature=0.7)
message = completion.choices[0].text
print(message)?)        

If everything is working, the response should be something like:

Alice has two pets: a bird and a dog.        

Fantastic it worked!?

A quick rundown of the parameters we just used to create the response:

engine??????(what engine used for the response)

prompt??????(the prompt, eg “Alice has a bird…”)??????

max_tokens ??(the maximum number of completion tokens for the response)

stop ???????(the constraint to stop generating output eg new line)

temperature ?(creativity of the output, between 0 - low creativity and 1-ma)

?

Remember I mentioned the OpenAI service had a cost? ?Cost is calculated in tokens, which are roughly 125% of word count of the input and output. The tiktoken library can translate text to tokens - we'll cover that in different post.

?Let’s see find out the tokens used to create this output:

print(completion.usage)?        

produces:

{
? "completion_tokens": 5,
? "prompt_tokens": 15,
? "total_tokens": 20
}        

prompt_tokens indicates 15 tokens were created by the phrase "Alice has a bird and a dog. How many pets does Alice have?".

completion_tokens refer to the 5 tokens of generated text.

total_tokens confirms that 20 tokens were used to run this query.?

?

The full pricing model can be found here:

?

GPT 3.5 model - the "Chat GPT Model"

The GPT 3.5 engine is similar to the davinci model, but optimized for chat. It has different pricing with similar capabilities with a slightly different syntax.

Gpt-3.5-turbo is recommended when chat-centric capabilities are needed. The Completion function has been changed to ChatCompletion. The new syntax divides the prompt into 2 roles; the system role describes the overall approach to the chat, while the user role holds the main message:

model_engine="gpt-3.5-turbo"
prompt = "Alice has a bird and a dog. How many pets does Alice have?"
completion = openai.ChatCompletion.create(model=model_engine,?
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?max_tokens=1024,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stop=None,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?temperature=0.7,
? ? ?messages=[{"role": "system", "content": "Reply with 'OK' unless there is a question"},
? ? ? ? ? ? ? ?{"role": "user", "content": prompt}],
? ? ?)
print(completion.choices[0].message.content)        

Running this code results in:

Alice has two pets.        

Let’s keep talking…

Still using gpt-3.5-turbo, let's keep talking about Alice and her pets...

prompt2 = "Alice has a goldfish. How many pets does Alice have?"
completion = openai.ChatCompletion.create(model=model_engine,?
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? max_tokens=1024,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? stop=None,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? temperature=0.7,
? ? messages=[{"role": "system", "content": "Reply with 'OK' unless there is a question"},
          ? ? {"role": "user", "content": prompt2}],
    )
print(completion.choices[0].message.content)        

Surprisingly, the response is something like this:

It is stated that Alice has one pet, which is a goldfish. Therefore, Alice has one pet.?        

Well, that’s awkward! – our model has no memory! ?Each session we raise with the engine is independent to the previous one. To provide a memory, we’ll need to manually chain our conversation together with the previous prompts.

prompt2 = prompt+"\nAlice has a goldfish. How many pets does Alice have"
completion = openai.ChatCompletion.create(model=model_engine,?
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? max_tokens=1024,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? stop=None,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? temperature=0.7,
? ? messages=[{"role": "user", "content": prompt2}],
? ? )
print(completion.choices[0].message.content)        

?Now we get the expected result!

Alice has three pets: a bird, a dog, and a goldfish.?        

We can continue manually chaining together our conversation until we hit the token limit of the model. For GPT-3.5, the token limit is 4096. If this limit is exceeded an error is raised and the completion is not run.

Making our own chatbot

We can add a loop to our prompt at accept user input to make our own chatbot:

prompt='
while True:
? ? user_input = input("Enter some text: ")
? ? if user_input == "":
? ? ? ? break
? ? prompt += user_input
? ? completion = openai.ChatCompletion.create(model=model_engine,?
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? max_tokens=1024,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? temperature=0.3,
? ? ? ? ? messages=[{"role": "system", "content": "Reply with 'OK' unless there is a question"},
? ? ? ? ? ? ? ? ? ?{"role": "user", "content": prompt}],
? ? ? ? )
? ? print(completion.choices[0].message.content)'        

This results in this conversation:

Enter some text: Alice has a bird and a dog. How many pets does Alice have
Alice has two pets.
Enter some text: Alice also has a pet frog
Alice has three pets.
Enter some text: Alice also has a cat
Alice has four pets.

Enter some text:         

The same code produces this:

Enter some text: what are the main types of Pokemon?
There are 18 different types of Pokemon: Normal, Fire, Water, Electric, Grass, Ice, Fighting, Poison, Ground, Flying, Psychic, Bug, Rock, Ghost, Dragon, Dark, Steel, and Fairy.

Enter some text: which of these are weakest to water attack?
The main types of Pokemon are Normal, Fire, Water, Grass, Electric, Ice, Fighting, Poison, Ground, Flying, Psychic, Bug, Rock, Ghost, Dragon, Dark, Steel, and Fairy. Among these types, Fire, Ground, and Rock types are weak to Water attacks.

Enter some text: what are some fire types?
The main types of Pokemon are Normal, Fire, Water, Electric, Grass, Ice, Fighting, Poison, Ground, Flying, Psychic, Bug, Rock, Ghost, Dragon, Dark, Steel, and Fairy.
Fire, Ground, and Rock types are generally weak to Water attacks.
Some Fire-type Pokemon are Charmander, Vulpix, Growlithe, Ponyta, Cyndaquil, Torchic, Chimchar, Tepig, Fennekin, and Litten.

Enter some text:         

and even this!

Enter some text: what are the ingredients for barbeque sauce
Barbecue sauce typically includes ingredients such as tomato sauce or ketchup, vinegar, brown sugar, Worcestershire sauce, mustard, garlic, and various spices.

Enter some text: can you give me a recipe?
Sure! Here's a simple recipe for homemade barbecue sauce:

Ingredients:
- 1 cup ketchup
- 1/4 cup brown sugar
- 1/4 cup apple cider vinegar
- 1/4 cup Worcestershire sauce
- 2 tablespoons Dijon mustard
- 1 tablespoon smoked paprika
- 1 teaspoon garlic powder
- 1/2 teaspoon salt
- 1/4 teaspoon black pepper

Instructions:
1. In a medium saucepan, combine all ingredients and whisk together until smooth.
2. Bring the mixture to a simmer over medium heat, then reduce the heat to low and let it cook for 10-15 minutes, stirring occasionally.
3. Remove the sauce from the heat and let it cool to room temperature.
4. Use immediately or store in an airtight container in the refrigerator for up to 2 weeks.        

It's certainly not ChatGPT, but it is only 13 lines of code, so - I think it's not a bad result.

Costings

Different GPT 3 engines have different pricing and different capabilities.

• davinci - Most capable GPT-3 model. Can do any task the other models can do, often with higher quality.
• curie - Very capable, but faster and lower cost than Davinci.
• babbage - Capable of straightforward tasks, very fast, and lower cost.
• ada - Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost.
• gpt-3.5-turbo - similar to the davinci model, but optimized for chat. It has different pricing with similar capabilities with a slightly different syntax.

Model usage costings aren’t much when we’re just playing the tool. But watch out!?These costs can add up!

GPT 3 model costings:

The davinci model (text-davinci-003) costs $0.02 per thousand tokens. Since our query was 20 tokens, our query cost:

$0.02*20/1000 = $0.0004 = 0.04 cents

The curie model (text-curie-001) costs $0.002 per thousand tokens and gave the same response. (0.4 cents for this query)

The babbage model (text-babbage-001) costs $0.0005 per thousand tokens and gave the same response. (0.1 cents for this query)

The ada model (text-ada-001) costs $0.0004 per thousand tokens, but only gave the correct response when the Temperature was lowered. (0.08 cents for this query)

GPT 3.5 model costings:

The gpt-3.5-turbo model (gpt-3.5-turbo) costs $0.0002 per thousand tokens and gave the same response. (0.04 cents for this query)

Wrapping it up

This post has shown some simple code producing some sophisticated results.

We've seen how to:

• Get registered for an OpenAI API key and get started with the OpenAI LLMs
• Send prompts to an OpenAI engine and receive the response
• Use GPT v3 and GPT v3.5
• Add a context memory to engine prompts
• created a simple chatbot with a memory
• Understand the costs associated with the prompts, responses and different engines
• Produce an awesome sounding recipe for BBQ sauce.

I hope you check back again and look for the next post in the series.

The next post in this series will look at the LangChain library, and how to query a set of documents.