Azure Open AI w/ User Auditing Using Cosmos DB

I have always been fascinated by the potential of artificial intelligence to create engaging and natural conversations. That's why I was excited to try out the Azure Open AI sample chat app. It's a web-based application that allows you to chat with a virtual assistant powered by Azure Open AI. In this article, I will share my experience of using the app, and how I leveraged Azure Cosmos DB to improve user auditing and analytics.

Chatting with Azure Open AI

The Azure Open AI sample chat app is a simple and intuitive interface that lets you type or speak your queries and get responses from the virtual assistant. You can also choose from different scenarios, such as booking a flight, ordering a pizza, or getting medical advice. The app is VERY basic, yet it allows you to leverage the magic of a large language model (LLM) with minimal provisioning time.

This application is awesome, but a lot of my customers have been asking for more enterprise-type features even while they are testing Azure Open AI. Namely, customers have asked "How do I know who is using it?" and "Which users use it more frequently or who are 'power users'?" Recently, the sample application was updated to include auditing of user and completion prompts. That means that both chatbot input and outputs are now logged inside our premiere NoSQL database using the lowest cost Serverless tier of the service!

Auditing

In my GitHub repository, I have forked the sample chat application and modified it slightly based on the feedback from my strategic enterprise customers I mentioned earlier. The specific reason I didn't use the "out-of-the-box" sample chat app found here is that the Cosmos DB auditing didn't include username auditing.

I augmented the code based on feedback from the internal software developers here at Microsoft to track utilization based on usernames going forward. Look at the code here on my GitHub repository.

Deploying the Enhanced Solution

To deploy the solution, I'd suggest you let the Azure AI Studio handle deploying the starter sample application as-is and then replacing the deployed code with the code from my repository using Visual Studio Code.

1. Deploy the starter application while also selecting the Cosmos DB tickbox during the deployment process. Here's a quick video outlining this process.
2. Replace the sample application after it is deployed (~10 min.) with the code from my GitHub repository.

Here's a screen shot of my cloned repository, right-click --> Deploy to Web App and target the web app that Azure AI Studio deployed to overwrite the standard sample app with the enhanced application from my GitHub:

Results

Here's an example audit log showing username auditing from my Azure Cosmos DB backend:

Conclusion

By deploying the code from my repository, you will include the out of the box auditing plus username monitoring. I hope this helps you on your next Azure Open AI adventure!