How do you measure the impact of your data analytics on society?

I have performed many economic impacts analyses. These require large amounts of data from public and private sources. Rarely does the data include interviews. It is important to convey the sources and limitations of the data used in any analysis. To the extent that data used in the analysis is private, stakeholders may not replicate results. In addition, not all stakeholders have access to the software tools used to crunch the data and produce results, so an open discussion of methodology is required. These issues are likely well known to economists.

Limited (clean) data availability and acceptance of the measurement approach/ mechanism by stakeholders involved are 2 key challenges associated. "Keep it simple, stupid" - leverage this philosophy while addressing both of these. Come up with a pragmatic way to measure through 1 or 2 simply KPIs (e.g. $ generated or saved, ...) that matters and are believable to accept. Not always we are able to measure properly, but as long as we have a transparent and effectively indicative impact assessment that moves the needle (business KPIs and human acceptance), that's quite good enough.

I think in addition to what to measure, prioritizing quantitative and qualitative metrics is essential. While numbers and data points can provide a clear picture of specific impacts, the stories and experiences of the people impacted by your analysis can provide deeper insight. For example, while you may measure economic benefits in terms of increased income or job creation, it is equally important to understand the lived experiences of those who may be displaced or affected by these economic changes. I think surveys, interviews and case studies also can be valuable tools in this field.

In addition to the mentioned methods, it is very important to use a continuous feedback loop in the measurement process. This means regularly reviewing and updating your measurement tools based on the evolving landscape of data analytics and its social implications. Using data visualization tools can also be helpful in understanding complex data sets and identifying patterns or trends. Additionally, working with third-party organizations or experts can provide an unbiased perspective on the impact of your analysis. They can conduct independent audits or assessments and ensure that measurements are both accurate and unbiased. Finally, always consider the long-term effects.

It is important to recognize that the social impact of data analytics is not always immediate and can evolve over time. Therefore, periodic re-evaluation after project completion is very important. For example, a year or even several years after the end of the project, impact re-examination can provide insights into long-term impacts and unintended consequences that may not have been apparent in the immediate aftermath. This iterative approach to measurement ensures that you capture both the immediate and evolving impacts of your analytics on society.

It is beneficial to engage third-party experts or consultants who can provide an unbiased perspective on the impact assessment. Their outside perspective can help identify blind spots or biases that may be overlooked by those directly involved in the project. Consider the participation of representatives of marginalized or underrepresented groups in society. Their insights can ensure that analyzes do not inadvertently perpetuate existing inequalities or biases.

Staying up to date with the latest trends, technologies and methodologies in the field of data analysis is essential. By attending workshops, conferences, and webinars, you can gain fresh perspectives and insights that can be incorporated into your work. It is so important to improve culture of continuous learning in the team and encourage members to come up with new ideas and challenge existing ideas and consider setting up a feedback loop with a diverse group of users to ensure that analytics solutions meet a wider range of societal needs.