Unlocking the Potential of Fine-Tuning in Generative AI

In a previous post I was sharing my excitement after been selected for the third year in a row to be part of the prestigious Forbes Technology Council team.

Now I want to share my first article in this new "season" and go more deeper including Microsoft perspective: "The Power Of Fine-Tuning In Generative AI"

In the article, I explained the primary two approaches for improving the precision and efficiency of Generative AI technology in AI applications: Prompt Engineering and Fine-tuning.

• Prompt Engineering refers to the practice of formulating precise and effective instructions to get optimal responses from advanced language models like GP3.5 or GPT-4. Essentially, the quality of the answers we will receive depends on the quality of the questions we ask. So, let's avoid stupid questions :)

• Fine-tuning refers to the process of customizing a pre-trained model to our specific data and know-how for advanced tasks. Fine-tuning also allows us to adapt the model to your domain, style, and tone, as well as filter out any unwanted or harmful content.

Is fine-tuning another method for training a Machine Learning Model?

Lately, a customer approached me with this question, and the concise response is affirmative. Both are machine learning methods at our disposal to enhance a model's performance.

With that in mind, it's crucial to understand that training refers to the process of educating a model from scratch on a large dataset to learn the underlying patterns and relationships between the input and output data. Fine-tuning, on the other hand, is the process of taking an existing pre-trained model and adapting it to a new task by training it on a smaller dataset that is specific to that task.

In other words, fine-tuning allows us to leverage the knowledge that the pre-trained model has already learn to achieve higher quality results, reduce the latency and cost of your requests.

As a customer using Azure OpenAI services, am I able to perform fine-tuning?

Certainly, fine-tuning is possible on Azure. To accomplish this, we should use Azure OpenAI Studio or Azure Machine Learning Service. Both offer a fine-tuning workflow for constructing personalized models from existing pre-trained models. This process involves using a distinct set of prompts, resulting in improved performance across a broader spectrum of tasks.

Based on my experience, organizations don't start with Generative AI by doing fine-tuning. Typically, is a four-phase journey.

1. The first phase is to use the available APIs to learn and get familiar with the technology, in the case of Microsoft, the Azure OpenAI Service.
2. Next, organizations progress to a phase where they harness the potential of supported chat models using their own data. This phase holds significant importance as it enables them to analyze and learn from their data with enhanced precision and speed. This approach typically involves tapping into specific sources, thereby enriching responses with the most current information drawn from the organization's designated data source. For instance, by combining OpenAI with Azure Cognitive Search, organizations can retrieve pertinent information from their data sources, guided by user input and the provided conversation history. This resulting information can serve as a prompt for Azure OpenAI, producing the natural and enjoyable Generative AI responses we all appreciate.
3. The logical third step involves fine-tuning a foundational model, which becomes necessary when an organization has particular use cases or requirements demanding tailored behavior or enhancements in model performance. This can include domain-specific knowledge, data specificity, and the need to mitigate bias, among other factors. Nonetheless, there are customers, predominantly in the banking and government sectors, who engage in fine-tuning as they seek increased control over the model's output to guarantee alignment with their particular objectives and policies.

To create a fine-tuned model in Azure OpenAI Studio, we will need to select a base model (i.e., ada, babbage, curie), determine the training approach, the training data formatted in JSONL, validation data sets (JSONL File where each line contains a prompt-completion pair), and optionally configure advanced options for our fine-tuning job (i.e., number of epochs *, learning rate, batch size, early stopping, or checkpointing). You can find more details and examples on how to configure these advanced options in this tutorial or this article.

You also can train Azure OpenAI models using Azure Machine Learning Service which offer a larger collection of models for fine tuning. Using Azure ML, we can train OpenAI based models such as GPT-3.5-Turbo, Babbage-002, and Davinci-002 as well open source models such as Llama-2-70b-chat and Databricks-dolly-v2-12b.

However, it's essential to remember that fine-tuning requires a substantial amount of labeled data and expertise in machine learning. It's not always necessary, and for many general use cases, the pre-trained models provided by OpenAI can be highly effective without further customization. It's crucial to carefully assess our specific needs and consult with Microsoft Data & AI Specialists before deciding to fine-tune an Azure OpenAI Service model.

Is it possible to develop my own version of Copilot?

By now, you must be impressed by the array of Copilots available at Microsoft, each serving as AI assistants to assist with various natural language tasks. Here's an overview of some of them:

1. Bing Chat Enterprise: Building upon the foundation of Microsoft Copilot, Bing Chat Enterprise prioritizes commercial data protection. It ensures that our business data remains secure, preventing access or usage by Microsoft or any other parties. It can be obtained as a standalone product or as part of Microsoft 365 E3 and E5 packages.
2. GitHub Copilot: This AI pair programmer is designed to enhance our coding productivity. It provides suggestions for entire lines of code or even complete functions directly within our code editor. Moreover, it can generate code based on natural language descriptions or comments, making coding more efficient.
3. Microsoft Sales Copilot: Tailored for sales professionals, Copilot in Viva Sales is an AI assistant that enhances sales effectiveness. It offers insights, recommendations, and reminders based on our sales data and activities. Additionally, it can assist in crafting personalized emails, proposals, and presentations for our prospects and customers.
4. Microsoft Security Copilot: This specialized AI assistant is dedicated to bolstering your organization's cybersecurity. It monitors our security posture, alerts us to potential threats, and provides guidance on remediation steps. Furthermore, it can assist in automating security tasks and policies to enhance our organization's security defenses.
5. Recruiter 2024 and LinkedIn Learning’s AI-powered coaching experience recently announced by LinkedIn with significant enhancements in their hiring and learning products.

But what do in cases where the existing Copilots do not align with your organization's specific requirements or policies? Well, in this case the solution lies in Azure OpenAI Copilot Stack which is the four stages of the mentioned journey.

To close

In conclusion, as you embark on your journey with Generative AI and consider when to use fine-tuning, remember that it's a powerful tool that can help tailor AI models to your specific needs and objectives. However, it's not a one-size-fits-all approach and should be carefully evaluated in the context of your organization's requirements. Choose the path that aligns best with your goals and consult with 微软 Data & AI Specialists when necessary. The possibilities with Generative AI, including Azure OpenAI Copilot Stack, are vast, and the journey promises to be an exciting one filled with innovation and customization.

* the number of times the model goes through the entire training data set during the fine-tuning process