How do you design and implement a participatory and inclusive evaluation process?

Neste ponto, entendo ser importante a análise cuidadosa do propósito que se dará aos dados coletados. Pode parecer óbvio, mas é comum observar uma grande quantidade de informa??es sendo gerada, porém subutilizada. Isso resulta em projetos com grande volume de dados que, infelizmente, n?o contribuem significativamente, pois n?o est?o alinhados com os objetivos e o escopo da avalia??o. Neste contexto, é comum observar a cria??o de fases prolongadas de coleta de dados, pois, muitas vezes n?o tem um foco definido, apenas coletam todas as informa??es possíveis.

Clearly articulate the purpose of the evaluation and the specific goals you aim to achieve. Define the scope, including the key areas, activities, and outcomes to be evaluated.

Identify and engage stakeholders who have an interest in or are affected by the evaluation. This includes program beneficiaries, staff, partners, funders, and other relevant groups.

Mapear - Clarificar a Contribui??o - Promover o Engajamento: No processo de avalia??o, é fundamental mapear os grupos ou indivíduos envolvidos, esclarecer a contribui??o que deles se espera e, a partir disso, fomentar o envolvimento ativo. A participa??o do público na avalia??o vai além do simples entendimento de porque ela é importante. Entendo ser importante que cada parte envolvida compreenda como a relevancia da sua contribui??o foi identificada. Na minha vis?o, quando as pessoas compreendem genuinamente o valor de sua participa??o e visualizam os resultados esperados do trabalho em andamento, a tendência é um engajamento mais efetivo.

I think this step is crucial. The nonprofit world is way ahead of the social entrepreneurship/impact investing world here. While it is quite common to collect primary data from people who are actually affected by the company/project/product ("beneficiaries") in the classic non-profit M&E approaches, it is absolutely not standard in the impact measurement and reporting approaches in the impact investing space. There's lots to learn and transfer here. Giving affected people a voice by including them in the evaluation process is not only most effective for learning, but also the morally right thing to do.

Indeed, collecting and analyzing evaluation data is crucial. However, integrating transdisciplinary methods elevates this process. In such an approach, stakeholders actively join the analysis team, fostering a genuinely participative model. Their firsthand interpretation enriches results and builds ownership. Employing mixed methods and triangulation not only bolsters rigor but also offers a holistic view of project performance. This aligns with transparency, collaboration, and inclusivity, enhancing evaluation's effectiveness.

Assemble an evaluation team that reflects diversity in terms of expertise, background, and perspectives. Include representatives from relevant stakeholder groups in the evaluation team.

While I totally agree with the description, in practice we see that many social entrepreneurs are struggling with this. Planning a mixed-methods research design, creating a (scientifically valid and reliable) survey, collecting the data, processing/cleaning/validating it, preparing it for monitoring and reporting, etc. requires capacity and expertise a typical social enterprise simply does not have. Having said this, I believe that there is a lot of potential in using technology to reduce the effort and cost involved in getting high-quality impact data. For example, software solutions that automate survey creation and data processing as well as using large language models to find the right indicators, validate data and understand results.

Fomentar la propiedad y el apoyo de la comunidad. Cuando las comunidades participan en la evaluación, es más probable que se sientan propietarias de los resultados y apoyen las recomendaciones para mejorar. Esto puede ayudar a garantizar la sostenibilidad a largo plazo de nuestros programas y servicios. Comenzar temprano y con frecuencia. Involucrar a la comunidad desde el principio del proceso de evaluación y continuar la comunicación con ellos a lo largo del camino. Sea transparente y abierto. Compartir los objetivos de la evaluación, los métodos y los resultados con la comunidad. Respetar las diferentes perspectivas. Valore la diversidad de opiniones y experiencias dentro de la comunidad.

Before you create your evaluation method, determine if you'll be looking for more quantitative or qualitative feedback. Both have their benefits, and understanding which combination works best for your needs will help you generate the most informative and clear feedback possible.

Involve stakeholders in the planning phase. Conduct workshops, focus groups, or surveys to gather input on evaluation questions, indicators, and methodologies. Work collaboratively with stakeholders to co-create evaluation criteria. Ensure that the criteria align with the organization's mission, values, and the expectations of diverse stakeholder groups.

Impact communication is most powerful when qualitative and quantitative data is combined! Evidence-based storytelling helps to get stakeholders (emotionally) involved and interested while also making sure that the communicated stories are based in real data and not just exceptions or even made up.

Engage stakeholders in interpreting results, making decisions, and applying insights for continuous improvement and informed action, ensuring the evaluation's impact is realized and integrated into future strategies.

Using data not just for reporting, but for monitoring and management is important. However, it requires good data analysis and visualization in order to disseminate interesting findings from the raw data. A common best practice is to collect data before (baseline) and several times after an "intervention", which makes it easier to understand the impact over time.

A piece of advice I've always loved about feedback is that if people tell you that something is wrong with your product and then tell you how to fix it, then something is definitely wrong with your product, but their suggested ways of fixing it are also often wrong. Consult your peers and the results of your evaluations in order to craft informed solutions, and test these solutions accordingly.

Designing and implementing a participatory and inclusive evaluation process involves engaging stakeholders at various stages, ensuring diverse perspectives are considered, and promoting collaboration.

While the evaluation process can never be "perfect", do your best to gather a wide variety of diverse feedback. It is okay to make mistakes, as long as you learn and change for the future.