Optimizing Large Language Models: Harnessing Hyperparameters for Fine-Tuning Excellence

Fine-tuning a language model can significantly enhance its performance and make it adaptable to specific tasks. Hyperparameters play a crucial role in fine-tuning a machine-learning model to achieve optimal performance. They are settings or configurations that are not learned from the data but must be specified before training begins. Here are some hyperparameters that affect the process of fine-tuning a model:

1. Learning Rate (LR)
2. Batch Size
3. Number of Epochs
4. Number of Layers (Architecture Hyperparameter)
5. Number of Neurons in Each Layer (Architecture Hyperparameter)
6. Dropout Rates (Architecture Hyperparameter)
7. Activation Functions (Architecture Hyperparameter)
8. Regularization Strength (Regularization Parameters)
9. Regularization Technique (Regularization Parameters)
10. Optimization Algorithm
11. Momentum (Optimization Algorithm)
12. Decay Rates (Optimization Algorithm)
13. Epsilon (Optimization Algorithm)
14. Data Augmentation Parameters

In this blog, we will be focusing on three significant Hyperparameters:

1. Learning Rate (LR)
2. Batch Size
3. Number of Epochs

But before that, let us focus on the basics of the LLM fine-tuning process.

Understanding Large Language Models

Before we delve into the fine-tuning process, let's briefly touch on the models that serve as the foundation for this endeavor. Large Language Models (LLMs), few are initially pre-trained on extensive datasets to grasp language patterns and semantics. This pre-training forms a robust base for further customization.

Preparing Your Dataset

Fine-tuning begins with selecting a base pre-trained model and preparing your dataset. Your dataset should be tailored to your specific requirements. The quality of your dataset plays a pivotal role in determining the model's performance. It should include relevant examples, prompts, and instructions to guide the model's learning process.

Task Adaptations

Task adaptations involve customizing the model for specific tasks by training it on specialized datasets. To fine-tune your LLM effectively, you'll need to pair it with a dataset that aligns with your objectives. This dataset will serve as the blueprint for shaping your model's behavior.

Fine-Tuning Process

Fine-tuning an LLM is an iterative process involving multiple training cycles and hyperparameter tuning. Techniques like Reinforcement Learning from Human Feedback (RLHF) are used to refine the model's behavior continually. Each cycle refines the model's understanding and adaptability to your specific task.

Hyperparameters: The Key to Fine-Tuning

Hyperparameters are critical in shaping the fine-tuning process. Three key hyperparameters to consider are:

1. Epoch

Epoch refers to how many times the model processes the entire dataset. Increasing the number of epochs can help the model refine its understanding. However, excessive epochs can lead to overfitting, where the model becomes too specific to the training data and struggles with generalization.

1. Learning Rate

The learning rate controls how quickly the model updates its parameters during training. A higher learning rate accelerates learning but may result in instability. A lower learning rate ensures stability but prolongs the training process. Optimal learning rates vary based on the task and model architecture.

1. Batch Size

Batch size determines how many data samples the model processes in a single iteration. Larger batch sizes can speed up training but require more memory. Smaller batch sizes can help the model thoroughly process each record. The choice of batch size should align with your hardware capabilities and dataset size.

Finding the Right Balance

Finding the right balance for these hyperparameters is crucial. Monitoring validation performance can help you identify when to stop training to avoid overfitting or underfitting. Experimenting with different hyperparameter values to optimize your fine-tuning process is recommended.

Quick Walk-Through

Sample Dataset

This is a data set that we have created. It's a sample data set, just a random one, and it has 12 records.

Following are the steps we followed to fine-tune the model:

1. Get credentials for respective LLM (API key)

api_key ="sk***************************************"

openai.api_key = api_key

1. Create training data

Make sure to end each prompt with a suffix. According to the respective LLM reference, you can use ->.

Also, make sure to end each completion with a suffix as well; I'm using .\n.

data_file = [{

????"prompt": "Prompt ->",

????"completion": " Ideal answer.\n"

},{

????"prompt":"Prompt ->",

????"completion": " Ideal answer.\n"

}]

1. Save Dict as JSONL:

The next step is to convert the dict to a proper JSONL file. JSONL file is a newline-delimited JSON file, so we'll add a \n at the end of each object:

file_name = "Training_Data_prepared.jsonl"

with open(file_name, 'w') as outfile:

????for entry in data_file:

????????json.dump(entry, outfile)

????????outfile.write('\n')

1. Check the JSONL file:

!openai tools fine_tunes.prepare_data -f Training_Data.jsonl

1. Upload training data

Now that you checked the improvement suggestions let's upload the training data:

upload_response = openai.File.create(

??file=open(file_name, "rb"),

??purpose='fine-tune'

)

upload_response

1. Save File name:

file_id = upload_response.id

file_id

file_id = 'file-*****************************'

1. Fine-tune model:

The default model is Curie. But if you'd like to use DaVinci instead, then add it as a base model to fine-tune like this:

# Define your model parameters

model_params = {

????"model": "Davinci", # Replace with the desired model name

????"n_epochs" : 30,

????"batch_size": 1,

????"learning_rate_multiplier": 0.3

}

confirm = input("Do you really want to fine tune the model ?")

if confirm == 'YES':

????# fine_tune_response = openai.FineTune.create(training_file=file_id)

????fine_tune_response = openai.FineTune.create(training_file=file_id, **model_params)

????fine_tune_response

Do you really want to fine-tune the model?YES

1. Check fine-tuning progress:

You can use two methods to check the progress of your fine-tuning.

### Option 1

Check the progress and get a list of all the fine-tuning events:

fine_tune_events = openai.FineTune.list_events(id=fine_tune_response.id)

fine_tune_events

<OpenAIObject list at 0x243d8ff7ec0> JSON: {

??"data": [

????{

??????"created_at": 1692720417,

??????"level": "info",

??????"message": "Created fine-tune: ft-*****************",

??????"object": "fine-tune-event"

????}

??],

??"object": "list"

}

### Option 2

Check the progress with the following method and get an object with the fine-tuning job data:

retrieve_response = openai.FineTune.retrieve(id=fine_tune_response.id)

retrieve_response

<FineTune fine-tune id=ft-*****************************************************> JSON: {

??"created_at": 1692720417,

??"events": [

????{

??????"created_at": 1692720417,

??????"level": "info",

??????"message": "Created fine-tune: ft-*************************************",

??????"object": "fine-tune-event"

????}

??],

??"fine_tuned_model": null,

??"hyperparams": {

????"batch_size": 1,

????"learning_rate_multiplier": 0.3,

????"n_epochs": 30,

????"prompt_loss_weight": 0.01

??},

??"id": "ft-***********************",

??"model": "Davinci",

??"object": "fine-tune",

??"organization_id": "org-*********************",

??"result_files": [],

??"status": "pending",

??"training_files": [

????{

??????"bytes": 12311,

??????"created_at": 1692029999,

??????"filename": "file",

??????"id": "file-***************************",

??????"object": "file",

??????"purpose": "fine-tune",

??????"status": "processed",

??????"status_details": null

????}

??],

??"updated_at": 1692720417,

??"validation_files": []

}

1. Save fine-tuned model:

### Option 1:

if fine_tune_response.fine_tuned_model != None:

????print("Model available")

????fine_tuned_model = fine_tune_response.fine_tuned_model

### Option 2:

if fine_tune_response.fine_tuned_model == None:

????fine_tune_list = openai.FineTune.list()

????fine_tuned_model = fine_tune_list['data'][0].fine_tuned_model

### Option 3:

if fine_tune_response.fine_tuned_model == None:

????fine_tuned_model = openai.FineTune.retrieve(id=fine_tune_response.id).fine_tuned_model

fine_tuned_model

1. Test the new model on a new prompt:

Remember to end the prompt with the same suffix as we used in the training data; ‘->’:

new_prompt = "Which studio is behind the movie 'Avatar: The Way of Water'?"

Answer = openai.Completion. create(

??model='davinci:ft-smartbots-2023-07-23-07-13-27',

??prompt=new_prompt,

# max_tokens=10, # Change the amount of tokens for longer completion

??temperature=0.4

)

print(answer['choices'][0]['text'])

###

?Fox Studios, which is behind the movie 'avatar'.

In the realm of language models, optimizing for excellence hinges upon the precise tuning of hyperparameters. It's a fusion of art and science, where the selection of learning rates, batch sizes, and a number of epochs is a delicate craft. These parameters are the brushstrokes that shape the model's performance, balancing the fine line between overfitting and underperformance.

In conclusion, optimizing language models through hyperparameter tuning is a testament to the synergy of technology and human expertise. It results in models poised for linguistic excellence, prepared to transform the way we comprehend and interact with the world through the medium of natural language processing. Ready to explore more about the world of language models? - Talk to us.