Import an Azure OpenAI API as a REST API into Azure API Management

What is Azure OpenAI?

According to the official MS documentation, the Azure OpenAI Service provides REST API access to OpenAI's powerful language models including GPT-4o, GPT-4 Turbo with Vision, GPT-4, GPT-3.5-Turbo, and Embeddings model series. These models can be easily adapted to your specific task including but not limited to content generation, summarization, image understanding, semantic search, and natural language to code translation.

Preparation

To follow along with the example in this article, you will need to have:

• An Azure API Management instance
• Permissions to give access to the Azure OpenAI resource from the API Management instance
• An Azure OpenAI resource with a model deployment
• An Azure Application Insights instance

Import the API using Azure OpenAI Service option

The easiest way to import an Azure OpenAI API is to use the Azure OpenAI Service option in the API Management.

Using this approach, most of the configuration will be done automatically by the API Management. This includes the operations for each endpoint, a system-assigned managed identity with the appropriate permissions to access the Azure OpenAI resource, a backend resource and a policy that directs API requests to the OpenAI service endpoint, and a policy that can authenticate to the Azure OpenAI resource using the instance's managed identity.

To configure this, go to the Azure portal and open an already existing API Management instance.

Click on the 'APIs' section, and then select the 'Azure OpenAI Service' card.

Next, choose the Azure OpenAI instance you want to import, choose the API version, enter a display name, a name and optionally a description.

Check the 'Improve SDK compatibility' option. What this will do is add /openai to the end of the URL. This is important because some SDKs expect the URL to end with /openai.

Optionally, you can choose to add this API to some of your existing products.

On the policies tab you can see the option to add the following policies:

• Manage token consumption
• Track token usage

By enabling manage token consumption, we can protect the API from overconsumption and control the cost. By enabling the track token usage, we can log consumed total, completion and prompt tokens.

The configuration for the manage token consumption policy allows you to specify the 'Tokens per minute (TPM)', and 'Limit by' option which can be either 'Subscription' or 'IP address'.

The configuration for the track token usage policy allows you to specify the Application Insights instance that will be used for logging, and the dimensions which will be used to group the token usage.

Finally, click on the 'Create' button.

Testing the imported Azure OpenAI API

After the import is completed, we can go ahead and check if the API is working as expected.

Examine the imported API and notice that we have everything we configured previously.

Open Postman and make an HTTP POST request to the /chat/completions endpoint.

https://iliev-copilot-apim.azure-api.net/ilievtech-azure-openai/openai/deployments/{deployment-id}/chat/completions?api-version={api-version}        

Make sure to change the {deployment-id} and the {api-version} with your own values.

And this is great! We can see that we get a 200 OK status code back.

Let's try going above the token limit. Call the endpoint multiple times each after the other. Notice that it will return 429 Too Many Requests status code.

And this is what we expect, because we configured the policies in the previous section.

That's it. It's pretty simple to import and consume an Azure OpenAI service through Azure API Management.

Thanks for sticking to the end of another article from 'Iliev Talks Tech'.

Next steps

• Azure AI Search - Data Deletion and Change Detection Policies
• Generate Smart-cropped Thumbnails with Azure AI Vision
• Perform sentiment analysis with Azure AI Language