Disseminate till you drop!

The urgency of communicating your evaluation results well under COVID-19

By Hur Hassnain and Marco Lorenzoni

Hats off to you! You are one of those evaluation managers or evaluators who have managed to have your evaluation done despite the challenges posed by the COVID-19 pandemic. But now comes the crucial question, what next?

The global evaluation community has been forced to quickly adapt to the changing external context caused by the pandemic. The need to flatten the curve of infection and the related measures (travel restrictions, lockdown…) means that many evaluation activities were either put on hold or cancelled. Some evaluation teams overcame these challenges by adapting their evaluation approaches to include remote methodologies such as ICT or phone-based solutions. Others postponed the activities, to be resumed once security conditions will allow this. Can we draw any benefits to our evaluation practices from this strange year 2020? Absolutely! Here are a few.

The chance to reflect

If there is one thing this year has brought, it is the opportunity to reflect on what we are doing in both our personal and professional lives – something that is too often neglected due to competing priorities. What are our priorities? What are we spending our time on? Is it necessary? Who is it benefitting? Are there ways to do things better? Making the time to reflect on such questions is an important process which allows us to learn, adapt and improve how we do things. In terms of evaluation, it is good practice to reflect on what information we are collecting and why. Many organisations are constantly collecting data for one purpose or another without consider – is it really necessary to gather additional data? To what extent existing data can be used for evaluation purposes, and what are the most efficient ways to integrate them with missing evidence? Are we using our time in the most effective way?

 Stronger collaboration within the evaluation sector

Global lockdown measures and restricted travel has had a big impact on the collection of primary data. It has spurred on the need to improve collective knowledge on how to collect data remotely and how to use secondary data more effectively. Organizations who had experience working in unsafe or hard to reach contexts had a particular advantage in terms of having awareness prior to the COVID pandemic about how to work remotely. Many online meetings, events and webinars have been convened to share expertise and best practices on remote data collection methods as well as how to do this in a safe and ethical way. These online gatherings have helped to reinforce a sense of community within the evaluation sector and have highlighted the importance of collaboration as well as knowledge and skills sharing. The lessons from the 2019’s DEVCO/ESS initiative ‘Evaluation in Hard-to-Reach Areas’ are still valid; the final lessons-learnt paper is available here. 

 An opportunity to close the feedback loops and disseminate results

The reduction in face to face data collection activities has resulted in more time to spend on other tasks. Some of this time can be spent on revisiting dissemination plans. The importance of ‘closing the evaluation feedback loop’ and ensuring that the learning results from evaluation activities is used effectively is not a new idea. In fact, it has long been recognized as an important stage in the evaluation cycle. However, this critical last step is too often overlooked due to budget/time constraints.

The global pandemic has shown that disseminating evaluation results is now even more important than ever because there is a need to enhance global knowledge and save resources.

 Figure 1: "One to One (or a defined many) communication” by Wesley Fryer (licensed under CC BY-SA 2.0)

Some basics about Evaluation Dissemination

Dissemination of learning and evidence has a twofold dimension, contributing to both internal and external learning processes:

• Internal learning allows organisations to understand what has/hasn’t worked and why; it enables them to improve practice e.g. amending intervention logics/theories of change, implementation plans etc.
• External learning is equally important since it enables stakeholders (local communities, organisations working in similar fields, governments, policymakers etc.) to also understand what has/hasn’t worked and why. Evaluation-based evidence contributes to enhancing global knowledge and initiatives which can be used to further development practices on a much wider scale e.g. systematic reviews and synthesis studies. Sharing more helps to reduce duplication and allows resources to be spent more wisely to reduce evidence gaps. External learning goes hands in hands with transparency and accountability: by disseminating independent conclusions on results of their cooperation work, EU institutions contribute to comply with a duty towards taxpayers, partners and beneficiaries.  

So, how can we improve our evaluation dissemination practice? Below are some steps to be taken to ensure that evaluations’ results are disseminated effectively.

1) Think about your audience – internal and external – who will benefit most from your findings?

The type of audience(s) you want to reach will help to define the type of messages you want to convey, their style and the method of dissemination.

While the identification of your internal audience may be relatively straightforward (in the case of DEVCO this may include your hierarchy and the political section in your Delegation, thematic units at the HQ, geo coordination units, relevant Delegations, fellow colleagues in your Unit or Delegation, the Evaluation and Results Unit, the ESS…), the identification of your external audiences may require a little bit more of reflection.

Start with a question: which stakeholders are the most likely to be interested in learning about the expected and unintended effects of the intervention you evaluated? This will help in identifying your external audience.

Invest time in identifying the institutions operating in the context where the evaluation has been conducted – they will undoubtedly benefit from learning about the evaluation findings. These institutions may include but are not limited to: local and national think tanks, academic institutions, civil society organizations, specialized media, UN bodies and the national statistical offices.

2) Explore different methods for disseminating to external audiences

Okay, you have identified your external audience, and now it is time to consider what communication format(s) will provide the most impact on your target audience? Is one format enough or should you invest time/resources to produce various formats? Where can you get guidance on that?

There are numerous ways to showcase the most significant findings from your evaluation – and the time you devote to select the most appropriate one(s) to your evaluation is well spent! Rather than sticking to traditional, tried and tested report formats and events at the national level to sharing results to a selected audience, how about thinking outside the box and learning from other institutions’ practices? 

Did you know that the most used methods to disseminate evaluation results are the following ones?

Figure 2: Creative communication of evaluation results - DEVCO/ESS study on evaluation dissemination practices

This resulted from a study on evaluation dissemination practices of some of the European Commission and 12 development cooperation institutions and NGOs[1] conducted by DEVCO/ESS in the first semester of 2020. Following this initiative, we opened up a public space in Capacity4Dev on ‘Disseminating evaluation’, which contains a wealth of useful resources such as an interactive report (take the time to browse it, it is a real goldmine providing links to different evaluation dissemination products) and five separate practical guidelines on creating videos, infographics, briefs, blogs and podcasts.

We would like this space to evolve over time and we invite everyone to share with us links to evaluation dissemination products that are worth sharing widely: also, in this field, learning from each other is key. If you have such examples, drop us an email[2] and we will happily publish them in this space.

3) Share internally

Invest time in making sure that those within your organization who could benefit from the evaluation findings are made aware of them by sharing your evaluation reports or summaries. Learning from experience and sharing the lessons is fundamental to improving practice and ensuring that necessary changes are made to address problems and avoid repeating mistakes. Within larger organisations, in particular, it is useful to think about how the learning is being recorded – perhaps there are informal and not-yet connected knowledge management structures/ databases which can be more widely accessed to ensure that knowledge is not lost as a result of staff turnover?

On top of the different methods mentioned above, think about the following ways to disseminate the results of your evaluation internally and decide which are most suitable:

• Email/Newsletters
• Online webinars/presentations
• Sharing blogs/posts via staff intranets/Communities of practice/knowledge networks
• Recording findings – knowledge management

4) Share about sharing!

Share the fact that you are sharing by encouraging a sharing culture within knowledge networks and different platforms so that others can follow the same. Share your dissemination products with the ESS so that we can help you further distributing widely (okay, we are getting a bit repetitive here, but as ancient Romans used to say “Repetita juvant[3]”.)

Disclaimer: Views expressed here are those of the authors and do not necessarily reflect the opinion or position of DG DEVCO.

Please note that this blog post was first published by the European Commission's Capacity4Dev Website under #EvalCrisis blogs.

 [1] Such as the Asian Development Bank, African Development Bank, International Fund for Agricultural Development, World Food Programme, Global Environment Facility, Green Climate Fund, UN Women, UNICEF, UNFPA, Y Care International, InsightShare and Oxfam.

[2] You can reach us at [email protected].

[3] Repeating does well.