Configuring Event Grids correctly in Logic App Standard

Event grids is a very useful resource in Azure which helps us handle and deliver events from various Azure resources. It supports a lot of Azure services out of the box and helps us avoid unnecessary polling. Its one of the clean and standard ways to implement a publish-subscribe mechanisms and sometimes has been used creatively as well to replicate Biztalk.

But it comes with its own pit falls, first and foremost, how we configure Event grids correctly. There is a lot of documentation about it, and it seems a bit scattered everywhere. I could not find even a simple example of how to make Event grids work correctly with Logic apps standard which led me to so much of trial and errors and finally I got it right :) Here are the 3 steps which I did. This is a simple scenario, where i need to listen to events emitted by blobs uploaded in an Azure storage account (V2 version and not v1 because V1 does not support events) and then do some processing in a workflow in Logic app standard. This scenario can be extended to the resource of your choice (If it supports Event grids :) ). Also I have used Managed identity as this is the preferred chocie for authentication in Azure.

• IF you are working on the Azure portal, you need to add 2 RBAC assignments to the intended storage account where you want to listen to events. Storage Blob Data Contributor EventGrid Contributor. You might be puzzled that you do not see the EventGrid Contributor role in the portal on the storage account. I think thats a bug in MS, because i see that role under the resource group where the storage account lies. So i simply used this scope instead of the storage account. IF you are deploying through the pipeline, you just need the EventGrid Contributor role. The reason, you need the Storage blob Data Contributor role while working in the portal is because when you create an event grid trigger in your workflow, it needs to list the storage accounts under Resource Name (marked in red) in the trigger.

I also selected managed identity while creating the eventgrid-connection. How i created the event grid connection with managed identity enabled via bicep is present at the end of this blog.

If you do not add the RBAC permissions above, you will get the below error in the trigger history of the workflow (not run history).

"The client '3xxxx5' with object id '3xxx5' does not have authorization to perform action 'Microsoft.EventGrid/eventSubscriptions/write' over scope '/subscriptions/9xxx8/resourceGroups/<resource-group-name>/providers/Microsoft.Storage/storageAccounts/<storage-account-name>/providers/Microsoft.EventGrid/eventSubscriptions/LogicApp8efeb9cc-b04b-45b4-a747-681360228795' or the scope is invalid. If access was recently granted, please refresh your credentials."        

• After this trigger, i add some steps and my workflow looks like this:

Here is what i do in the logic app:

1.) After the event is received, I parse the json event with the event grid schema. You cna get the schema from here Input to Parse JSON by using the example in MS document. Inputs to Parse_JSON is:

@triggerBody()        

2.) I check in the condition after, whether the event is a subscription Validation event or not. This is an important step. Condition is:

"equals": [
          "@body('Parse_JSON')[0]?['eventType']",
          "Microsoft.EventGrid.SubscriptionValidationEvent"
        ]
        

The reason you need to do this, is to validate that your workflow is the correct subscriber to the events in the storage account. This is a security mechanism, to avoid rookie subscribers to your storage account. When you save the workflow, a subscription will be created in the storage account automatically with some default naming convention, but it will be in 'Creating' state. Till you do not validate this event, it will be in this state. To validate the event, I have used the Response action: Response_for_validation.

{
  "type": "Response",
  "kind": "Http",
  "inputs": {
    "statusCode": 200,
    "body": "@item()?['data']?['validationCode']"
  }
}        

You need to send a 200 response code, to validate the subscription. After this is done, the provisioning state of the subscription will be 'Succeeded'.

You can see the subscription which was automatically created after successful validation under Storage account -> Events along with ehat event types you selected while configuring the trigger.

If you do not validate the event, the subscription will be deleted automatically. And you will be really puzzled why you are not getting events. You will not be notified in any way, and will be scratching your head like i did :)

Please note: It tries to send 4 events one after the another for validation, so you will see 4 successful runs of the workflow. I believe, you need to validate all of them.

• Now try uploading a blob in a container in storage account. The event should be picked up by the trigger. As it is not a subscription validation event, it goes under False condition. This event will have details like the operation done on the blob, the url and other stuff which I believe is sufficient for you to download and process the blob successfully.

Note that we have the same principle here as we have for validation. We still have to send the response code 200 in the False condition because Event grid is atleast once delivery which needs to be acknowledged. If you do not acknowledge the event, it will retry the event in an exponential schedule as mentioned in the MS document.

This is how i ackowledge the event using Response for processing step.

{
  "type": "Response",
  "kind": "Http",
  "inputs": {
    "statusCode": 200,
    "body": "@triggerOutputs()?['queries']?['code']"
  }
}        

Its also important that you send the response before you start the processing, to acknowledge event first and hence preventing any duplicates. If your processing takes more time than the first retry schedule which happens after 10 seconds, then the event will be retried again which can lead to duplicates like it did for my case :).

I tried to test this by putting a Terminate condition before the Response for Processing so as to not acknowledge the event and indeed i got the same event fired multiple times.

First was the actual event, second event happened roughly 10 seconds later, third happened after 30 seconds, fourth happened after a minute and the fifith happened after 5 mins, This followed the retry policy mentioned in the MS document

So its very important that you acknowledge the event.

Another tip, while deploying the workflow with the event grid trigger, please use parameters instead of app settings in the trigger code. This event grid trigger is a Shared trigger and not a Built in trigger, so it will not understand app settings as it does not run in process. We do not have an built in trigger yet for azure event grid. Here is the code for the event grid trigger:

"triggers": {
            "When_a_resource_event_occurs": {
                "type": "ApiConnectionWebhook",
                "inputs": {
                    "host": {
                        "connection": {
                            "referenceName": "azureeventgrid"
                        }
                    },
                    "body": {
                        "properties": {
                            "topic": "/subscriptions/@{encodeURIComponent(parameters('WORKFLOWS_SUBSCRIPTION_ID'))}/resourceGroups/@{encodeURIComponent(parameters('WORKFLOWS_RESOURCE_GROUP_NAME'))}/providers/Microsoft.Storage/storageAccounts/@{encodeURIComponent(parameters('storageAccountName'))}",
                            "destination": {
                                "endpointType": "webhook",
                                "properties": {
                                    "endpointUrl": "@{listCallbackUrl()}"
                                }
                            },
                            "filter": {
                                "includedEventTypes": [
                                    "Microsoft.Storage.BlobCreated"
                                ],
                                "subjectBeginsWith": "/blobServices/default/containers/<container-name>/blobs/"
                            }
                        }
                    },
                    "path": "/subscriptions/@{encodeURIComponent(parameters('WORKFLOWS_SUBSCRIPTION_ID'))}/providers/@{encodeURIComponent('Microsoft.Storage.StorageAccounts')}/resource/eventSubscriptions",
                    "queries": {
                        "x-ms-api-version": "2017-09-15-preview"
                    }
                }
            }
        }        

You also need to create the event grid connection (azureeventgrid in the above example) and if you are using managed identity like I did, please use the following code:

parameterValueType: 'Alternative' is very important here as this tells the event grid connection to be a managed identity.

resource azureEventGridConnection 'Microsoft.Web/connections@2016-06-01' = {
  name: 'azureeventgrid' 
  location: location
  kind: 'V2'
  properties: {
    displayName: 'azureeventgrid' 
    statuses: [
      {
        status: 'Ready'
      }
    ]
    customParameterValues: {}
    parameterValueType: 'Alternative'
    api: {
      name: 'azureeventgrid' 
      
      id: '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Web/locations/westeurope/managedApis/azureeventgrid'
      type: 'Microsoft.Web/locations/managedApis'
    } 
  }
}

resource azureEventGridConnectionAccessPolicy 'Microsoft.Web/connections/accessPolicies@2016-06-01' = {
  parent: azureEventGridConnection
  name: 'azureventgrid'
  location: location
  properties: {
    principal: {
      type: 'ActiveDirectory'
      identity: {
        tenantId: logicAppName_resource.identity.tenantId
        objectId: logicAppName_resource.identity.principalId
      }
    }
  }
}        

I believe these steps will help you in configuring the event grid correctly :)