Your philanthropic impact data is being questioned. How do you convince stakeholders of its validity?

This is both a provenance and communication issue. For provenance, focus on data accuracy, bias, integrity, and other standard checks in your design, monitoring, evaluation & learning (DMEL) processes. On communication, take the opportunity to ensure alignment with donors on data collection goals and methods. Are you answering the same questions and are you working towards similar impact, or did something get lost in translation? If anomalies arise in your findings, promptly inform donors and stakeholders. Use this opportunity to correct both the issue as well as DMEL systems.

It's crucial to address the concerns with transparency, credibility, and a commitment to continuous improvement: 1. Explain the methodology used to collect and analyze the data. This includes detailing how data was sourced, the tools and metrics used for measurement, and the timeline of data collection. Highlight any third-party audits or validations of your data to add credibility. 2. Showcase Transparency: Offer to provide raw data or access to reports that detail the findings. Transparency can help reassure stakeholders that your data is accurate and has been gathered responsibly. 3. Compare your data with industry benchmarks or similar organizations to demonstrate that your results are in line with expected outcomes.

“Explain to stakeholders how these processes work” - starts with UNDERSTANDING how these processes work from start to finish in your organization, and planning accordingly. If you rely on other colleagues to pull data and provide information, plan well ahead! Set mini deadlines to allow time for conflicting priorities, working schedules, thorough review and any necessary changes/clarifications. Develop a good relationship with the internal teammates you rely on data for - those strong relations go a long way!

The below are the 3 main points to ensure data accuracy: Robust Data Collection and Management: Standardized Metrics: Use established metrics and methodologies aligned with industry standards to ensure consistency and comparability. Data Validation: Implement rigorous data validation processes, including cross-checking with multiple sources, regular audits, and third-party verification if feasible. Transparent Methodology: Clearly document and communicate the data collection methodology, including sources, definitions, and limitations, to enable stakeholders to understand the process and assess its reliability.

Ensuring funder satisfaction is critical. To address data collection and presentation, we must understand the funder’s definition of success and process expectations. With this information in-hand, we would then provide detailed information on methodologies, including: - Data collection techniques (surveys, interviews, focus groups) and engagement methods (online, paper, events). - Data collection instruments – share details to highlight survey design. - Data analysis – explain how results were obtained, ensuring trust through transparency and qualified assessment. - Stakeholder workshops – explain methodologies and gather feedback, addressing concerns directly. Our goal is to show our commitment to transparency, alignment & shared values.

What should matter to you and your supporters is not just numbers, but impact. If, for example, your mission is to educate low-income children, it's not just the number of children served, but how many went on to graduate from high school compared to the overall statistics of how many kids never graduated. Sttements from the graduates and/or their parents/guardians are also invaluable.

Balancing qualitative (stories, client opinions) and quantitative (statistics) is vital to producing compelling impact narratives. Understand what impact is: the top-line, long-term change your donors are making happen (remember it is not your impact, it is theirs). Keep the narrative focused on how the data demonstrates the charity is achieving its Vision. Data on outputs and outcomes is helpful in supporting but it is not as compelling as you think it is.

Feedback Mechanisms: Establish feedback mechanisms to gather input and address concerns from stakeholders, fostering a sense of ownership and collaboration. Participatory Evaluation: Involve stakeholders in the evaluation process, allowing them to contribute their perspectives and insights, which can enhance the credibility and relevance of the data. Open Dialogue: Foster open and honest dialogue with stakeholders, acknowledging limitations and uncertainties in the data, while emphasizing the commitment to continuous improvement and learning. The above are the 2 salient points in engaging stakeholders.

If you have a suitable cause for which current or former clients can provide input on approaches/feedback on Impact Reports, do it. Donors may occasionally have unrealistic expectations about what data is possible to collect or what constitutes success or failure but will rarely question those with lived experience of the programmes. Demonstrate your accountability and transparency to your clients. It tells a donor a lot about how you will report to them. Do not be afraid to acknowledge shortcomings too, for the same reason. No organisation is perfect and we can only get better through partnerships with donors that trust us. Learn together.

Some ways I have found to be very helpful are Clear Communication and Storytelling: Visualizations: Utilize data visualizations, such as graphs, charts, and infographics, to present complex data in a clear and accessible manner, highlighting key trends and insights. Impact Narratives: Supplement data with compelling stories and case studies that illustrate the real-world impact of the philanthropic activities, creating an emotional connection with stakeholders. Regular Reporting: Provide regular and transparent reports on the impact data, including progress updates, challenges, and lessons learned, to build trust and demonstrate accountability.

Monitoring and Evaluation are of limited value without effective Learning. You need to dedicate ample time/resources to interpreting data and challenging whether the charity's activities support your ambitions. Where resources or expertise around MEL are limited, approach other organisations and share methodologies/results to see what can be learned. Best practice is almost never achieved in isolation.

When a stakeholder questions the validity of a philanthropic impact report, it points to misalignment. To avoid misalignment, here are 3 questions to ask BEFORE any kind of data collection and impact reporting: 1. What is the intended philanthropic impact that you and your key stakeholders are aligned on? 2. There are many ways to measure and communicate philanthropic impact. Why are you choosing a particular methodology and set of indicators? 3. Have you communicated the strengths and limitations of your choices to your key stakeholders? If there is agreement on number 1, clarity on 2, and the answer to 3 is 'yes', you significantly minimize the risk that your impact report's validity will be questioned.

One thing I’ve found helpful is seeking External Validation: 1. Expert Review: Seek external review and validation of the data and methodology from recognized experts or independent organizations to provide an objective assessment of its rigor and validity. 2.Benchmarking: Compare the impact data with relevant benchmarks or industry standards to demonstrate the effectiveness of the philanthropic activities relative to peers. 3. Peer Learning: Participate in peer learning networks and communities of practice to share experiences, challenges, and best practices in measuring and communicating impact.

It's worth considering who is questioning the impact data. Given how charities, foundations and major donors have been pulled into 'culture wars' by individuals or groups with a negative agenda, there might be occasions when it is not worth devoting time to counter those questioning it. Of course, you would hope that a genuine stakeholder would not take this approach.