Fine-Tuning LLaMA2 with Alpaca Dataset Using Alpaca-LoRA

Alpaca-LoRA provides a way to efficiently fine-tune large language models like LLaMA2. By leveraging LoRA, it achieves similar results to the Stanford Alpaca model and can even be executed on devices as compact as a Raspberry Pi for research purposes.

Note: This article is part of the following article:

Key Components:

1. Low-Rank Adaptation (LoRA): A technique that alters a small part of the model's weights, making fine-tuning large models more resource-efficient.
2. Hugging Face’s PEFT and bitsandbytes: These tools are utilized for efficient fine-tuning and training.

Note: his article is part of the following Medium article:

Note: We will be using Google Colaboratory Python notebooks to avoid setup and environment delays. The focus on this article to get you up and running in Machine Learning with Python and we can do all what we need there. The following article explains how to use it:

Google Colab GPU

If you are using Google Colab, make sure to select the GPU

Local Setup:

• Clone the git repo.

!git clone https://github.com/tloen/alpaca-lora.git        

%cd alpaca-lora        

Install dependencies (https://github.com/tloen/alpaca-lora/blob/main/requirements.txt):

%pip install -r requirements.txt        

Restart the Kernel:

You will need to restart the kernel after installing the requirements

Explore Dataset

import pandas as pd

df = pd.read_json("alpaca_data.json")

df         

Let's reduce the size to only 1k to run locally in less time with fewer resources.

dataset_df_1k = df[:1000]        

dataset_df_1k.to_json('alpaca_data_1k.json', orient='records')        

This command converts the dataset_df_1k DataFrame into a JSON file named "alpaca_data_1k.json", where each row in the DataFrame is a separate JSON object.

Training:

• The finetune.py script applies PEFT to LLaMA models, handling prompt construction and tokenization.
• It allows adjustment of several hyperparameters like batch size, learning rate, and LoRA-specific settings.
• We will be using the open_llama_3b_v2 model https://huggingface.co/openlm-research/open_llama_3b_v2

!python finetune.py \
    --base_model 'openlm-research/open_llama_3b_v2' \
    --data_path './alpaca_data_1k.json' \
    --output_dir './lora-alpaca-1k' \
    --batch_size 16 \
    --micro_batch_size 16 \
    --num_epochs 2 \
    --learning_rate 1e-4 \
    --cutoff_len 512 \
    --val_set_size 900 \
    --lora_r 8 \
    --lora_alpha 16 \
    --lora_dropout 0.05 \
    --lora_target_modules '[q_proj,v_proj]' \
    --train_on_inputs \
    --group_by_length        

1. batch_size: This parameter defines the number of training examples used in one iteration of model training. In deep learning, the entire dataset is typically divided into smaller batches, and the model's weights are updated after processing each batch.
2. micro_batch_size: This is often used in scenarios where the batch size is large and the available memory (RAM or VRAM) is limited. The large batch is divided into smaller 'micro-batches'. Each micro-batch is processed sequentially, but the weight update is performed only after the entire batch is processed.
3. num_epochs: An epoch is one complete cycle through the entire training dataset. num_epochs specifies how many times the learning algorithm will work through the entire training dataset.
4. learning_rate: This is a hyperparameter that determines the step size at each iteration while moving toward a minimum of a loss function. Essentially, it controls how much the model's weights should be adjusted with respect to the loss gradient.
5. lora_ parameters*: These are specific to the Low-Rank Adaptation (LoRA) technique used in fine-tuning.lora_r: Specifies the rank in LoRA. A higher rank means more parameters are being adapted, which can lead to more powerful but also more resource-intensive training.lora_alpha: This might refer to a scaling factor in LoRA, affecting the impact of the low-rank matrices.lora_dropout: Dropout rate in LoRA layers. Dropout is a regularization technique where randomly selected neurons are ignored during training, which helps in preventing overfitting.

These parameters collectively determine how the model learns from the data and adapts its weights during the training process. Fine-tuning them can significantly impact the model's performance and training efficiency.

Inference:

• The generate.py script demonstrates how to load the model and LoRA weights for inference, using Gradio for a user interface.

!python generate.py \
    --base_model 'openlm-research/open_llama_3b_v2' \
    --lora_weights 'lora-alpaca-1k' \
    --share_gradio True        

Conclusion: Alpaca-LoRA represents a significant advancement in fine-tuning large language models, offering a balance between performance and resource efficiency. It invites users to experiment and contribute to further improvements in model performance, especially with a focus on better datasets.