How to track the user experience of your Bot with Azure, Power BI and Microsoft Teams

Application templates for Microsoft Teams are very popular and I got the following question from a customer that deployed the FAQ Plus bot to their internal users. For those who wonder what FAQ Plus app is, take a look at the https://aka.ms/teamsapptemplate, it provides a nice out of the box FAQ app, based on the Azure QnAMaker and Azure Bot Framework, and enhanced with Microsoft Teams collaboration features as message extension, chat and meeting rooms. These app templates are open-sourced, so you can basically do whatever you want to customize them to your own use-case and imagination.

Here was question : "How can we collect users-feedbacks and track them over-time?"

Well, if you are familiar with Microsoft Power Virtual Agent, you may have noticed this built-in feature where the user is prompted to provide a feedback on his experience, ranging from Terrible to Excellent - Power Virtual Agent also provides a dashboard where you can see how this score evolves over time. Here is how to build a similar experience on your own.

First, start with any of the Microsoft Bot code samples provided on GitHub at https://github.com/Microsoft/BotBuilder-Samples - I use to work with JS but the Adaptive Card are in JSON format and the design of the solution are language agnostic. Here is the proposed design:

First, you need to identify when you'll push the "AskRating" card to your users - You want to collect feedbacks but not to annoy users with the same question repeated over and over again. The "end of a conversastion" is not always clear to identify but my advise would be to keep track of which user replied to your request for evaluation and keep track of when you received the last eval or when you pushed the rate card to this specific user for the last time.

If you're building your bot app for Microsoft Teams or to interact with Microsoft services, you can use the O365 Fluent UI framework styles and graphical resources provided by Microsoft - Please read carefully the assets licence agreement if you do so and make sure your use case complies with these conditions. Here are the icons I selected from https://uifabricicons.azurewebsites.net/ to build the AskRating card

Then let's create the Adaptive Card that will collect the user feedback - This card will present a message with the above list of emoji and get a JSON object as an output of the user interacting with the card - To facilitate the design of the card, I used the Adaptive Cards designer available at https://adaptivecards.io/designer/ - This is also where you'll find all the SDK documentation to author and send different types of cards - Here is the code snippet you can copy/paste in the designer - Make sure you replace the URL parameters with the locations of your icons

{
    "type": "AdaptiveCard",
    "$schema": "https://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.3",
    "body": [
        {
            "type": "TextBlock",
            "text": "Great! Please rate your experience.",
            "size": "Medium"
        },
        {
            "type": "ImageSet",
            "imageSize": "Small",
            "images": [
                {
                    "type": "Image",
                    "url": "${$root.URL_1}",
                    "size": "Small",
                    "selectAction": {
                        "type": "Action.Submit",
                        "data": {
                            "score": 0,
                            "text": "Terrible"
                        }
                    }
                },
                {
                    "type": "Image",
                    "url": "${$root.URL_2}",
                    "size": "Small",
                    "selectAction": {
                        "type": "Action.Submit",
                        "data": {
                            "score": 1,
                            "text": "Poor"
                        }
                    }
                },
                {
                    "type": "Image",
                    "url": "${$root.URL_3}",
                    "size": "Small",
                    "selectAction": {
                        "type": "Action.Submit",
                        "data": {
                            "score": 2,
                            "text": "Fair"
                        }
                    }
                },
                {
                    "type": "Image",
                    "url": "${$root.URL_4}",
                    "size": "Small",
                    "selectAction": {
                        "type": "Action.Submit",
                        "data": {
                            "score": 3,
                            "text": "Good"
                        }
                    }
                },
                {
                    "type": "Image",
                    "url": "${$root.URL_5}",
                    "size": "Small",
                    "selectAction": {
                        "type": "Action.Submit",
                        "data": {
                            "score": 4,
                            "text": "Excellent"
                        }
                    }
                }
            ]
        }
    ]

}

With this definition, when the user clicks on one of the icons, a message containing the JSON payload defined in the "data" property above will be sent back to your bot application as a message - Here is the type of activity message you'll receive as part of the turncontext :

{
   ...
   type: 'message',
   value: { score: 2, text: 'Fair' },
   from: {
       id: '29290bf7-8e0d-47b5-a82e-88d623139bbc',
       name: 'Thomas',
       role: 'user'
   },
   conversation: { id: '3b1ccbb0-f1bc-11ea-a34d-f972df47629f|livechat' },
   ...
}

What you'll want to do then is to save this information into a persistant storage service as Azure Table - Here again, Microsoft provides SDK to integrate your application with this service and all is well documented on this page - https://github.com/Azure/azure-storage-node#table-storage - Just create a new async function in the ActivityHandler class of your bot that takes a turncontext object (cf. exemple above) as an input and write the data you want to use for your analytics dashboard into the Azure Table - Here is an exemple

 class MyBot extends ActivityHandler {
    constructor() {
        super();
        // Constructor part
    }


    // Defition of the function that writes to the Azure Table
    static async WriteAnalyticsToAzureTable(turnContext) {
        var entGen = azure.TableUtilities.entityGenerator;
        var entity = {
            PartitionKey: entGen.String('analytics'),
            RowKey: entGen.String('random_string'),
            score: turnContext.activity.value.score,
            text: turnContext.activity.value.text,
            uuid: turnContext.activity.from.id,
            uname: turnContext.activity.from.name,
            cid: turnContext.activity.conversation.id
          };
          tableService.insertEntity(AzureTableName, entity, (error, result, response) => {
            if (!error) {
              // result contains the ETag for the new entity
            }
          });

    }; 
};  

You can check that everything works fine directly from the Microsoft Azure Storage Explorer client or online via the Azure portal.

The last part is the Power BI dashbard - Download the desktop client and connect to your data source in Azure - Azure Table is a native connector in Power BI, just provide the required info : account name and account key - Once the connection is established, go into "Transform data" to expend the content of your Azure Table and then press "Close & Apply" - At this point, you should have your Azure Table data loaded into Power BI

You can now build you own Power BI analytics dashboard that tracks your user feedbacks and satisfaction over time ! You cal also publish this dashboard on Power BI and make it available to your colleagues via Microsoft Teams - There are only 2 steps to achieve that :

1.Press "Publish" and select your Microsoft Teams group

2. From Microsoft Teams, add a new tab, select Power BI and locate your Power BI dashboard from the list. 

And here you are !

Don't hesitate to contact me if you have questions.