Build, train, and deploy a specific model using custom data on Runpod.io.

We'll use the GPT-2 small model (125M parameters) as our base model and fine-tune it on custom data.

# Lab: Fine-tuning and Deploying GPT-2 on Runpod.io

## Objective

In this lab, you will fine-tune a GPT-2 small model on custom data using Runpod.io, and deploy it with a Flask API for text generation.

## Prerequisites

- A Runpod.io account

- Basic understanding of Python and machine learning concepts

## Step 1: Setting Up Runpod.io

1. Log in to your Runpod.io account.

2. Click "Deploy" and select a GPU (recommend at least 16GB VRAM).

3. Choose the "PyTorch Latest" container.

4. Set a pod name (e.g., "GPT2-FineTune-Lab").

5. Deploy the pod and connect to it using the web terminal.

## Step 2: Preparing the Environment

In the web terminal, run:

```bash

pip install transformers datasets torch flask

mkdir gpt2_lab && cd gpt2_lab

```

## Step 3: Prepare Training Data

1. Create a file for your training data:

```bash

nano train.txt

```

2. Add your custom training data. For example:

```

This is a sample text for our AI model.

We're fine-tuning GPT-2 to understand specific patterns.

The model will learn from this custom data.

```

3. Save and exit (Ctrl+X, then Y, then Enter).

## Step 4: Data Preparation Script

Create and run a script to prepare the data:

```bash

nano prepare_data.py

```

Add the following code:

```python

from datasets import Dataset

def load_dataset(file_path):

with open(file_path, 'r') as f:

texts = f.read().split('\n')

return Dataset.from_dict({"text": texts})

# Load and process the dataset

dataset = load_dataset('train.txt')

dataset.save_to_disk('processed_dataset')

print("Dataset prepared and saved to 'processed_dataset' directory.")

```

Run the script:

```bash

python prepare_data.py

```

## Step 5: Model Fine-tuning Script

Create the fine-tuning script:

```bash

nano finetune_gpt2.py

```

Add the following code:

```python

import torch

from transformers import GPT2LMHeadModel, GPT2Tokenizer, TextDataset, DataCollatorForLanguageModeling

from transformers import Trainer, TrainingArguments

from datasets import load_from_disk

# Load pre-trained model and tokenizer

model_name = "gpt2"

model = GPT2LMHeadModel.from_pretrained(model_name)

tokenizer = GPT2Tokenizer.from_pretrained(model_name)

tokenizer.pad_token = tokenizer.eos_token

# Load and tokenize dataset

dataset = load_from_disk('processed_dataset')

def tokenize_function(examples):

return tokenizer(examples["text"], padding="max_length", truncation=True, max_length=128)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

# Set up training arguments

training_args = TrainingArguments(

output_dir="./results",

num_train_epochs=3,

per_device_train_batch_size=4,

save_steps=500,

save_total_limit=2,

)

# Create Trainer instance

trainer = Trainer(

model=model,

args=training_args,

train_dataset=tokenized_datasets,

data_collator=DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False),

)

# Start training

trainer.train()

# Save the fine-tuned model

model.save_pretrained("./fine_tuned_gpt2")

tokenizer.save_pretrained("./fine_tuned_gpt2")

print("Fine-tuned model saved to './fine_tuned_gpt2' directory.")

```

Run the fine-tuning script:

```bash

python finetune_gpt2.py

```

This process may take some time depending on your dataset size and GPU.

## Step 6: Creating a Flask API

Create a Flask application to serve your model:

```bash

nano app.py

```

Add the following code:

```python

from flask import Flask, request, jsonify

from transformers import GPT2LMHeadModel, GPT2Tokenizer

import torch

app = Flask(__name__)

# Load the fine-tuned model and tokenizer

model = GPT2LMHeadModel.from_pretrained("./fine_tuned_gpt2")

tokenizer = GPT2Tokenizer.from_pretrained("./fine_tuned_gpt2")

@app.route('/generate', methods=['POST'])

def generate_text():

data = request.json

prompt = data['prompt']

input_ids = tokenizer.encode(prompt, return_tensors="pt")

with torch.no_grad():

output = model.generate(input_ids, max_length=100, num_return_sequences=1, no_repeat_ngram_size=2)

generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

return jsonify({'generated_text': generated_text})

if name == '__main__':

app.run(host='0.0.0.0', port=5000)

```

## Step 7: Deploying and Testing

1. Run the Flask application:

```bash

python app.py

```

2. In the Runpod dashboard, set up port forwarding:

- Go to your pod's settings

- Under "Network", add a new port forward:

- Internal Port: 5000

- External Port: Leave blank for auto-assignment

3. Note the external URL provided by Runpod for your port forward.

4. Test your API using curl (replace YOUR_RUNPOD_URL with your actual URL):

```bash

curl -X POST https://YOUR_RUNPOD_URL.runpod.net/generate \

-H "Content-Type: application/json" \

-d '{"prompt":"Once upon a time"}'

```

You should receive a JSON response with generated text based on your prompt and fine-tuned model.

Conclusion

You have successfully fine-tuned a GPT-2 model on custom data using Runpod.io and deployed it with a Flask API.

This setup allows you to generate text based on your fine-tuned model through web requests.

Remember to stop your Runpod instance when not in use to manage costs.

---

This Lab provides a complete workflow for fine-tuning and deploying a GPT-2 model on Runpod.io.

It includes data preparation, model fine-tuning, and deployment with a Flask API.

Students can follow these steps to experiment with their own custom datasets and see how fine-tuning affects the model's output.