Qualitative dashboarding

This is the fourth and last brief article on the basis of an evaluation of a European programme to improve systems of protection for victims of #humantrafficking. (Programme supported by Porticus , Evaluation done with Avance Impact ).

Developing dynamic #dashboards has been one of my favourite things for years. Too many monitoring efforts are like a vacuum cleaner: data gets sucked up and never gets out. Written reports are one awesome way to make data available to users, but the average span of reading attention is fast reducing (and for evaluation reports it was already low). Dashboards are another way to make data available. The user starts with a single screen and takes her or his own tour through what the data have to tell, and depending on the architecture of the dashboard this could lead to meaningful and credible data-based stories of change (and yes, there are very important steps to take in order to turn data into stories....).

So far, I had been doing such things in #Excel (I still love Excel), but recently, I had the privilege of being part of a team effort to develop a data architecture (as part of an overall monitoring system) including a series of dashboards for a large programme on child protection (not this evaluation, but another piece of work with Avance Impact ). And we ended up using #PowerBI (linked to "single source of truth" Excel files in SharePoint). Credits for TADA Solutions for kickstarting our PowerBI skills

But this is all for #quantitative stuff, right? Nope. For quant data there are a lot more manipulations and aggregations available of course, but I think there is great value in dashboarding for qualitative data (and it is likely there are some semi-quantitative things one ends up doing when pulling those data into a dashboard).

In the recent evaluation of the European programme to strengthen systems of support for victims of #humantrafficking we used the key elements of #OutcomeHarvesting and ended up with a database with 35 systemic changes, of which we retained 28 after assessments. This is a database with mostly qualitative information. But as it turns out, it is very helpful to use a #PowerBI dashboard to play around with these data and browse through these stories in many ways. The most interesting parts are not so much the total numbers of changes for each and every category, but the options to click through to completer stories of change or see additional information on hovering, clicking, or drilling through.

I am aware that there is a lot of monitoring and evaluation software that impact-driven organisations use, including more generic software that is adapted for tracking impacts. But so far, I have not been impressed by the stories of actual users about the possibilities for reports, overviews or dynamic dashboards (except for financial data), even though such software packages may be a very good option to collect and store data. And being an evaluator, I often need to include additional types of data and fields that are not included in such software. For that reason, I am enthusiastic about the options of dashboarding for both quantitative and qualitative data about change. How I wish, we would have had this during the #PADev project.

The work on this dashboard is not included in the evaluation report. I did most of the dashboard development after the evaluation was largely over. But it will be used for a follow-up study we will be doing on these (and possibly additional) systemic changes that this programme has contributed to.

See the screenshots to get an impression. The basic idea is: overview (1) > select (2) > get smaller list (3) > select > get details for one change (4).

Screenshot 1: overview of the dashboard. On the left a set of filters to select from the total set of 35 changes. In the 4 charts the numbers by dimension of the system, actor in the system, pathway of change that is being used in the change (6 key pathways were defined), and the 3 assessments done for each change (evidence, significance, contribution). In this overview, the user can click on each part of the charts to cross-filter.

Screenshot 2: the user has clicked on the dimension "capacities" (one of the dimensions of the system). You see the subset of changes that affected this dimension (14 total) reflected in the other charts as well (e.g. 10 of those 14 changes affected service providers). Making a selection also activates the button that serves as a drill-through (top-right "Go to list")

Screenshot 3: The user has clicked the "Go to list" button to drill through to the selected set of 14 changes (namely those that affect the system dimension 'capacities'), and gets a list of those changes. The score is an aggregate of the 3 assessments done for each change. Within this list, the user has clicked on a particular change (3rd from top) and this enables the button "Go to details" that is linked to a further drill-through action, namely one that brings the user to a page with details about the selected change. The arrow top-left allows the user to go back to the overview and make other selections.

Screenshot 4: user has clicked on "go to details". This page provides all the details of this change. This is as yet underdeveloped, but it can include full fledged narratives of the change, and / or quotes from stakeholders, and / or pictures or other media, and / or references to websites of the organisations involved, etcetera. From here the user can go back to the list or to the overview. It is also possible to go to the details page at once (without drilling-through from a particular selection) and select a single change to look at its details.

The possibilities are many more (e.g. showing on a map). My main reason of writing about this, is that the PowerBI universe is almost completely filled with examples from business (sales, profits, products), and almost only with quantitative stuff. And this trial convinced me that I can use PowerBI dashboards very well to make qualitative stuff accessible. I know there is specific software that can visualise narratives and plot numbers of narratives against certain characteristics. That is great of course. But I think PowerBI can do most of this as well, and in ways that leave the user a lot more flexibility.

You can find the evaluation report in this post or at the Avance website (https://www.avance-impact.nl/nl/publicaties/) (although you would not find anything on the use of qualitative dashboarding there).