Trying Out Azure OpenAI - Diving into the Experience

A few days ago, I signed up my organization, Haefele Software, for the Azure OpenAI service, specifically the implementation involving OpenAI's cool GPT-powered chat functionality. With a clear goal in mind, I jumped into the adventure of using historical AFL match results dating all the way back to 1897. The idea was to set up a controlled chat environment using just this dataset to see how practical it is to integrate custom data into the advanced GPT-3.5 model.?

As I await my turn to access the exciting GPT-4 model (currently on a waitlist due to its sky-high demand), I wanted to share my experience with those curious about trying out this technology. Getting started turned out to be refreshingly smooth, highlighting the user-friendly nature of the system.

Preparing the data

I found an entire compilation of AFL game outcomes, neatly organized with key details including Date, Round, Teams, Venue, and Results:

This data trove started as a text file that required some tidying up to make it compatible with my proof of concept.

My initial task was to convert it into a more accessible CSV format. Next, I used this CSV to import the dataset into Azure Table Storage.

However, as it stood, this data wasn't particularly meaningful to an AI language model like GPT-3. This prompted me to add an enrichment step during the import process, where I added a narrative for each row to generate a natural language summary of the game. Here’s an example of what I came up with:

On Sunday, 1 May 2011, in Round 6, Essendon played against Gold Coast at Essendon's home stadium of Docklands. Essendon won and Gold Coast lost with a score difference of 139.
The breakdown of points was as follows: 
Essendon(home) Goals: 31
Essendon(home) Behinds: 11
Essendon(home) Total: 197

Gold Coast(away) Goals: 8
Gold Coast(away) Behinds: 10
Gold Coast(away) Total: 58        

This transformation turned the raw data into a narrative, enabling the AI model to comprehend and interact with the information more effectively.

Preparing the data for Azure Open AI

Within Azure Open AI, there are different ways to handle external data sources. In my case, I chose to index my dataset using Azure Cognitive Search. This involved setting up an Azure Cognitive Search instance and indexing my dataset.

This choice made it easier for me to integrate my data into Azure, giving the AI model a convenient way to access and understand the data. By utilizing Azure Cognitive Search, I established a connection between my dataset and the capabilities of Azure Open AI, creating a smoother pathway for data-driven insights.

Once the data was indexed, it was ready for Azure Open AI.

Creating an Azure Open AI instance

After going through the sign-up process and gaining access to Azure Open AI, I moved on to the next step of creating my own instance. This was done directly through the Azure portal.

Once I had the necessary access, I logged into the Azure portal and set up my personalized instance of Azure Open AI. This instance served as my dedicated workspace to utilize OpenAI's tools within the Azure environment.

The process of creating the instance was pretty straightforward. This step connected the dots between my access to OpenAI and the practical application of its features.

By selecting "Go to Azure OpenAI Studio," I could immediately begin the process of configuring my conversational AI powered by GPT-3.5 using the dataset I had prepared.

Selecting a Model Deployment

In the process of preparing my chat AI for action, a pivotal step involved choosing the right GPT model for the task at hand. This decision held significant implications for the capabilities and performance of the eventual conversational AI system.

By specifying the GPT model, I essentially determined the underlying framework and sophistication of the AI's language processing abilities. This ranged from the model's language comprehension depth to its creative generation prowess. The selection process involved considering factors such as model size, complexity, and the nature of the interactions I intended to enable.

Ultimately, this step was akin to choosing the "brain" of my chat AI—shaping its language understanding, responsiveness, and potential to engage users effectively. Right now, the only available models to me are the "GPT-3.5 turbo" and "GPT-3.5-turbo-16k"

GPT-3.5 turbo:

This model variant, known as "GPT-3.5 turbo," is designed to strike a balance between generating high-quality responses and efficiency. It's optimized for most use cases where a prompt generates a response of up to 4096 tokens in length. Tokens can vary in length, from single characters to whole words, and longer tokens, like paragraphs, can take up more of the available token limit. This model suits various conversational and language-related tasks, providing coherent and contextually relevant answers while maintaining a reasonable response time.

GPT-3.5-turbo-16k:

In contrast, "GPT-3.5-turbo-16k" is tailored for situations requiring longer responses. It's optimized for generating responses with a maximum length of up to 16,384 tokens. This extended token limit allows for more comprehensive and detailed outputs, making it suitable for tasks that require in-depth explanations, longer narratives, or detailed content generation. However, it's important to note that working with longer tokens can increase response times due to the computational resources required for processing and generating such extensive text.

Integrating My Indexed Dataset into the Chat Playground

Using your own data within the chat environment is a feature currently in its preview stage. In my case, I opted to utilize the indexed dataset I had prepared earlier. This involved a straightforward process of connecting my chosen data source with the selected GPT model deployment.

By leveraging my indexed dataset, I enriched the chat playground's interactions with domain-specific knowledge and context.

The preliminary nature of this feature suggests exciting possibilities for tailored AI interactions, enabling users to create unique and engaging conversational experiences.

When utilizing GPT, it's recommended to employ text data as the source. Hence, I opted for the summary field I had crafted earlier.

Testing in the Playground

With the groundwork laid, the next step involves testing the chat AI right within Azure AI Studio. This handy phase lets me experiment and make adjustments before rolling it out.

By immersing the chat AI in this controlled environment, I can have virtual conversations to see how it responds, adapts, and behaves. It's like a trial run, helping me spot any tweaks needed to ensure a smooth experience when fully launched.

This testing phase is like a safety net, allowing me to fine-tune and enhance the AI's performance. It ensures that users get a polished, efficient, and user-friendly AI when it's officially up and running.

Launching an Azure Web Application

And now, the simple yet impactful step: deploying the system to an Azure Web Application. This straightforward process allows me to have a fully functional version of GPT-3.5 up and running within my Azure environment, directly drawing from the data I've prepared.

By deploying to an Azure Web Application, I'm essentially taking what I've crafted and making it accessible to users through the web. This streamlined deployment ensures that the power of GPT-3.5 is harnessed within the Azure ecosystem, with the added benefit of utilizing my own curated dataset. The result is a seamlessly integrated AI experience right within the familiar environment of Azure.