What does Fine-Tuning OpenAI models mean and how does it help?

Just thought of sharing something to satisfy your curiosity if any with Fine-Tuning of Microsoft Azure OpenAI models.

Microsoft Azure Open AI model used here : "text-curie-001"

• Go to your Microsoft's Subscription Azure AI Studio
• Go to Models -> Base Models
• Select "text-curie-001" and click Deploy. This would be your Non Fine Tuned Model

• Click Create
• You will see the Deployment in the Deployments tab

Now let's ask some questions to this Non Fine Tuned Model. You can use the Completions in the Playground or you can use a python Program to connect to this model and ask a question. Have shown both examples below.

Now though this might a right answer, based on how the model has been trained.

However, being an iMochaite, I would like the answer to be related to our Startup iMocha which is helping enterprises to build a skills-first, data-driven ecosystem encompassing hiring, upskilling, and managing talent – at scale, for any job role, any industry.

So let's train this model on iMocha facts.

• Create a .jsonl file with prompts and completions. Example below

{"prompt": "What is iMocha", "completion": "iMocha is a skills intelligence and assessment platform that enables talent teams to make smarter talent decisions."}
{"prompt": "iMocha", "completion": "iMocha is a SAAS HRTech into Skills Assessment and Skills Intelligence"}
{"prompt": "Who are the founders of iMocha", "completion": "Amit Mishra and Sujit Karpe are the founders of iMocha"}
{"prompt": "Who is the Head of Engineering at iMocha", "completion": "Vishal Madan is the Head of Engineering at iMocha"}
{"prompt": "Tell me more about iMocha", "completion": "iMocha is a SAAS HRTech into Skills Assessment and Skills Intelligence. It was founded in 2015 by Amit Mishra and Sujit Karpe. It is a top tier partner of Microsoft and is a TM Forum Member"}
{"prompt": "Name few clients of iMocha", "completion": "iMocha caters to the Talent Acquisition needs of Hexaware, Capgemini, United Nations, Navy Federal and companies like Upgrad, HPE use it for their Talent Development needs"}
{"prompt": "Tell me more about iMocha", "completion": "iMocha is Helping enterprises to build a skills-first, data-driven ecosystem encompassing hiring, upskilling, and managing talent – at scale, for any job role, any industry"}
{"prompt": "What can you do with iMocha's Live Interview", "completion": "Using iMocha’s solution, you can conduct automated interviews, live interview, and live coding interview. You can collaborate seamlessly with your hiring team in real-time. Invite up to 3 interviewers to assess a candidate thoroughly and reduce the number of interviews."}        

• Go to Azure AI Studio -> Models -> Base Models
• Select the same base model "text-curie-001". However this time instead of clicking Deploy, click "Create a Custom Model"

• Once you click on "Start Training job", you might have to wait for around 30 mins or so which includes queuing and the actual training.
• Once the training is completed, you will see the new Custom Models in the Management -> Models -> Custom Models

• The Custom Model has to be deployed, just like the Base Model (Non Fine Tuned) was deployed.

Once this Custom Fine Tuned Model is deployed, you will see 2 deployments in your Deployments tab, one for the Non Fine Tuned Model and for the new Custom Fine Tuned Model.

You would need to wait till the Status becomes Succeeded before you start using it

Now let's go back to the Playground or use the Python Program that we had created and get the response to the same question "What is iMocha?" and see if the answer differs.

As an iMochaite, though the response is not completely accurate, I am now happy that the response is more closer to what iMocha is into i.e. Talent Acquisition, Management, Development, Intelligence and not the earlier "Coffee Delivery Service".

Please also note that this Fine Tuned model is trained on only 8 facts. In reality we would be training this model on 100+ facts for far better accuracy. However for the essence of Fine Tuning to be captured and to reduce on Training Costs for this Proof of Concept, this limited training is good enough.

Did you notice something?

Though my training data had exactly the same prompt "What is iMocha" with a completion saying "iMocha is a skills intelligence and assessment platform that enables talent teams to make smarter talent decisions.", it actually did not use the same completion.

"{"prompt": "What is iMocha", "completion": "iMocha is a skills intelligence and assessment platform that enables talent teams to make smarter talent decisions."}So what is actually happening during the training?        

So what is happening during the Training?

Large Language Models ( LLMs ) like the OpenAI curie model, work on something called as Embeddings that are nothing but mathematical vectors that store the context of a word per dimension ( topic ) . By some kind of vector mathematics and techniques like cosine similarity, it is decided, how multiple words or phrases come together based on the topic at hand and finally forming the actual response.

Without going into details, with my training, I have placed iMocha?and related words in the right position in the vector space and hence brought it closer to the words like B2B, Talent Management, Talent Intelligence etc., closer than coffee ordering.

Hope this article helped to get some basic curiosity satisfied.

Please note that Training, Deployment and Prompts are billable, so please check the pricing before you try out yourself.