Consume a batch of messages in an Azure Function from an Azure Service Bus

What is an Azure Function?

According to the official MS Docs (Azure Functions Overview | Microsoft Docs) an Azure Function is a serverless solution that allows you to write less code, maintain less infrastructure, and save on costs.??

Some of the benefits of using Azure Functions are:?

• They are scalable?
• They are fast to execute?
• They are serverless?

Azure Function Triggers?

There are multiple triggers that you can choose from for an Azure Function like HTTP, Timer, Event Grid etc. For our Azure Function trigger we will be using a Service Bus Topic Trigger.?

If you want to find out more about triggers and bindings, I suggest you read the official MS Docs (Triggers and bindings in Azure Functions | Microsoft Docs).

Project Structure?

Our project will consist of a Console App which will be used to send messages to our Service Bus topic and an Azure Function which will be used to process the messages.?

Creating an Azure Service Bus?

First let’s create the Service Bus that we’ll use in this example.??

Go to the Azure Portal and in the search bar type 'Service Bus'.

After that, click on the '+ Create' button.

Fill in the information needed like in the following image:

Note: to be able to create a topic we need at least a Standard Pricing tier.?

Click on the 'Create' button.?

Wait for the deployment to be successful.

Next we need to create a topic. Go to the Service Bus Namespace we created and click on 'Topics' and then on the '+ Topic' button.?

Fill in the details like in the following image and click on 'Create':

Finally let’s create a subscription to our topic.?

Select the topic we created previously, click on 'Subscriptions' and then click the '+ Subscription' button.

Fill in the details like in the following image and click on 'Create':

Perfect. We successfully create a topic and a subscription to that topic.?

Next let’s see how we can create the Azure Function that we will be using along with this topic.?

Creating an Azure Function?

Add a new Azure Functions project like in the following image:

Select the Service Bus Topic trigger and fill in the details for connection string setting name, topic name and subscription name.??

Then, finish creating the Azure Functions project.

Explaining the scenario

Let’s start by examining our console application which contains the following code:

using Azure.Messaging.ServiceBus;


var connectionString = "<NAMESPACE CONNECTION STRING>";
var topicName = "bulk-message";
var nmMessages = 3;


ServiceBusClient client;
ServiceBusSender sender;


client = new ServiceBusClient(connectionString);
sender = client.CreateSender(topicName);


using ServiceBusMessageBatch messageBatch = await sender.CreateMessageBatchAsync();


for (int i = 1; i <= nmMessages; i++)
{
? ? if (!messageBatch.TryAddMessage(new ServiceBusMessage($"Message {i}")))
? ? {
? ? ? ? throw new Exception($"The message {i} is too large to fit in the batch.");
? ? }
}


try
{
? ? await sender.SendMessagesAsync(messageBatch);
? ? Console.WriteLine($"A batch of {nmMessages} messages has been published to the topic: {topicName}.");
}
finally
{
? ? await sender.DisposeAsync();
? ? await client.DisposeAsync();
}


Console.WriteLine("Press any key exit...");
Console.ReadKey();        

From the code we can see that first we need to set the topic name to which we will send the messages and also the connection string. You can get the connection string if you go to the Service Bus Namespace you created and click on Shared access policies.

Then we create the clients that we will use for sending and processing messages.?

We have a variable named nmMessages which we use to set the number of messages we will send to our topic.?

We add the specified number of messages to the batch and send the batch of messages to the Service Bus topic.?

Finally, we call the DisposeAsync() on the client types to ensure that network resources and other unmanaged objects are properly cleaned up.?

Execute the Console app and take a look at the subscription. We can clearly see that currently we have 3 active messages, which we sent just now.

Now we need to receive these 3 messages in our Azure Function. Add the following code to the Function1.cs file:

using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;


namespace BatchMessaging
{
? ? public class Function1
? ? {
? ? ? ? private readonly ILogger<Function1> _logger;


? ? ? ? public Function1(ILogger<Function1> log)
? ? ? ? {
? ? ? ? ? ? _logger = log;
? ? ? ? }


? ? ? ? [FunctionName("ProcessBulkMessages")]
? ? ? ? public void Run([ServiceBusTrigger("bulk-message", "ConsoleSubscription", Connection = "connectionstring")] string[] mySbMsg)
? ? ? ? {
? ? ? ? ? ? foreach (var item in mySbMsg)
? ? ? ? ? ? {
? ? ? ? ? ? ? ? Console.WriteLine(item);
? ? ? ? ? ? ? ? _logger.LogInformation($"C# ServiceBus topic trigger function processed message: {item}");
? ? ? ? ? ? }
? ? ? ? }
? ? }
}        

We can see that instead of a string mySbMsg we are waiting for an array of messages string[] mySbMsg.??

Inside the Run method we go through all the received messages, write each message to the console and log information that the message has been processed successfully.?

To complete the batch processing setup, we need to add the following code to the host.json file:

{ 
  "version": "2.0",
? "extensions": {
? ? "serviceBus": {
? ? ? "prefetchCount": 100,
? ? ? "messageHandlerOptions": {
? ? ? ? "autoComplete": true,
? ? ? ? "maxConcurrentCalls": 32,
? ? ? ? "maxAutoRenewDuration": "00:05:00"
? ? ? },
? ? ? "sessionHandlerOptions": {
? ? ? ? "autoComplete": false,
? ? ? ? "messageWaitTimeout": "00:00:30",
? ? ? ? "maxAutoRenewDuration": "00:55:00",
? ? ? ? "maxConcurrentSessions": 16
? ? ? },
? ? ? "batchOptions": {
? ? ? ? "maxMessageCount": 3,
? ? ? ? "operationTimeout": "00:01:00",
? ? ? ? "autoComplete": true
? ? ? }
? ? },
? ? "logging": {
? ? ? "applicationInsights": {
? ? ? ? "samplingSettings": {
? ? ? ? ? "isEnabled": true,
? ? ? ? ? "excludedTypes": "Request"
? ? ? ? }
? ? ? }
? ? }
? }
}        

Note that we have specified the batchOptions -> maxMessageCount property to 3. So, for example, if someone sends us a batch of 5 messages, our function will first execute with 3 messages and then with the remaining 2 messages.?

Now run the function locally and examine the result:

Our Azure Function receives the planned batch of messages, in this case the 3 messages we sent from our Console app. The output of our Azure Function is shown in the image below:

Perfect. We can see that our Azure Function executed correctly and received the batch of messages.

Conclusion?

We have successfully created a Service Bus topic and sent messages to the same. Then we created an Azure Function that received the batch of messages and processed them together.?

Thanks for sticking to the end of another article from "Iliev Talks Tech". #ilievtalkstech?

The full, more detailed implementation of this example can be found on my GitHub repository on the following link:?

DimitarIliev/batch-message-processing (github.com)