Scaling AI use cases, ChatGPT on your data

ChatGPT is a powerful language model that can generate responses based on natural language inputs. It is a highly effective tool for generating content for various use cases. One of the key benefits of ChatGPT is that it can be fine-tuned on your own data, which can lead to significant improvements in the quality and relevance of generated outputs.

But now you don't even need to fine-tune the model to use your own data!

Azure OpenAI on your data enables OpenAI's GPT-3 Turbo and GPT-4 language models to generate responses based on your data. You can access Azure OpenAI on your data through a REST API or the web-based interface in Azure OpenAI Studio. This tool helps create a solution that connects your data, resulting in an enhanced chat experience.

One of the most notable features is the ability to retrieve and use relevant data to improve the output of the language model. This is made possible in collaboration with Azure Cognitive Search. Based on user input and conversation history, Azure OpenAI on your data identifies the most appropriate data source to retrieve data from, then augments and resubmits that data as a prompt to the OpenAI model. The retrieved information is added to the original prompt, and the model processes it like any other prompt.

After the prompt has been submitted to the model along with the retrieved data, the model uses this information to generate a relevant response. It's worth mentioning that while the retrieved data is appended to the prompt, the resulting input is processed by the OpenAI model according to its original prompt.

The best and easiest way of set this up in my opinion is using the "Add your data" in the Azure AI Studio.

To get started, you need to already have been approved for Azure OpenAI access and have an Azure OpenAI Service resource with either the gpt-35-turbo or the gpt-4 models deployed.

Before starting to set this up, create a storage account and add the data that you want do use. Enable managed identity on the Azure OpenAI resource and give that managed identity the Storage Blob Data Contributor role on the storage account. The supported formats of the data is:

.txt

.md

.html

Microsoft Word files

Microsoft PowerPoint files

PDF

You can choose if you want the indexer to run once, hourly or daily. You can also decide to use Vector search using Ada embedding models, available in select regions.

When clicking next you can decide of the different search options in the below table.

After this has been done you can decide if you want the responses to be only from the data that you provide or not.

Now you are ready to start chatting with you data!

As an example, I needed to replace the razorblades on my lawnmower robot, and when trying to find the info it was quite hard. So I added the owners manual as my data and asked it how I should replace them.

Next, you can deploy the model either to Power Virtual Agents or a web app. I think the web app is the simplest way. After deploying you can change the system message, front end or what ever you want by cloning down the repo that the app uses and changing what you want. Then build it and update you app with Azure CLI

You get the references and citations in the answers.

To make the app more enterprise, add the services in the below architecture and utilize the service as explained in this previous article. Add on monitoring as this article explains, and you are set!