Toy Tuning a Model to match Cat Breeds and Human Temperaments with OpenAI

A model can be trained to become an assistant tailored to your needs. Fine-tuning allows so by providing a number of customised examples to a pre-trained model.

For this toy fine-tuning, I tapped into the OpenAI library to have created and make use of a customised version of gpt-3.5-turbo. This is achieved by uploading a jsonl file with some examples and asking OpenAI to fine-tune the model remotely. Usually, this technique is used to train chatbots, and the number of samples uploaded – with historical use cases or tried-and-true solutions – is enormous. In my case, there are just 16 examples in the jsonl file, which were in turn created by ChatGPT 3.5 itself.

Here follow the various steps in Python. As usual, the code starts by uploading a few libraries.

import json
import os
import openai
import requests        

Afterwards, it reads the OpenAI key and the jsonl file with the examples.

key_location = '/Users/silvi/Downloads/key-location/genaikey.txt'

with open(key_location, 'r') as file:
    key = file.readline().strip()

openai.api_key = key

# The with keyword automatically closes the file when you are done
with open("./data.jsonl") as f: 
print(f.read())        

The upload of the jsonl file and the creation of a fine-tuning job are then performed using the OpenAI library.

file = openai.files.create(
  file=open("data.jsonl", "rb"),
  purpose="fine-tune"
)

openai.fine_tuning.jobs.create(
  training_file=file.id,
  model="gpt-3.5-turbo"
)        

The response to the second call returns an id which allows to get updates on the status of the fine-tuning job launched.

So, to retrieve the state of the job, I repeatedly ran these lines.

fine_tuning_job_state = openai.fine_tuning.jobs.retrieve("ftjob-lk2CcFQT4BgWyagYob8Dwie0")

print("fine-tuning job status: ", fine_tuning_job_state.status)
print("fine-tuning job error: ", fine_tuning_job_state.error)
# Eventually, fine_tuned_model gets from None to ft:gpt-3.5-turbo:my-org:custom_suffix:id
print("fine-tuning model: ", fine_tuning_job_state.fine_tuned_model)        

Eventually, the fine-tuning job finished, and I got the string to univocally reference my customised model. The final state of the job is also sent via email to the address associated to the OpenAI account.

Lastly, I test the fine-tuned model by writing a question which is the the same contained in the first line of the jsonl. Here, the temperature will determine the "creativity" allowed, on a scale from 0 to 2.

completion = openai.chat.completions.create(
  model="ft:gpt-3.5-turbo-0125:bitdrops-ai::8y3MbQfN",
  messages=[
    {"role": "system", "content": "Cat expert."},
    {"role": "user", "content": "What is the best cat breed for someone living in an urban location and which has a relaxed temperament?"}
  ],
# how stocastic on a scale of 0 to 2; suggested 0.7
  temperature=0.7
)
print(completion.choices[0].message.content)        

The answers with low temperature are, unsurprisingly, the same or very close to the one gpt-3.5-turbo was fine-tuned with. When the temperature is 2, instead, the resulting completion is incomprehensible.

If you want to run this locally, the Jupyter Notebook and the jsonl can be found here.

This very simple chatbot has allowed me to explore a topic that I find very fascinating. Hope cats will be grateful...