The challenges and approaches of reporting direct and indirect numbers in development projects

Reporting is an indispensable aspect of project implementation, with a significant focus on quantitatively assessing indicators outlined in the project's log frame. One such crucial quantitative metric is reach, which denotes the total number of individuals benefiting from a project. Reach holds paramount importance for stakeholders, including funding agencies and governments, seeking accountability regarding the extent of project impact. However, discussions surrounding reach reporting often spark contentious debates within organizational settings, spanning NGOs, governmental bodies, and funding agencies. Stakeholders commonly seek answers to a fundamental question: "How many individuals or lives were positively influenced by the intervention?" Direct and indirect reach elucidate the multifaceted nature of project impact assessment." (Smith & Johnson, 2020; Brown, 2018)

Direct Reach: Refers to the immediate and quantifiable beneficiaries or recipients of project interventions. These are individuals or entities directly targeted by the project activities and are typically measurable in terms of numbers or specific outcomes achieved.

Indirect Reach: Encompasses individuals or entities that are not the primary targets of project interventions but still experience some degree of impact as a result of the project's activities. Indirect beneficiaries may include communities adjacent to the project area or those influenced by the project's broader outcomes or ripple effects. Indirect reach is often more challenging to quantify compared to direct reach and may require qualitative assessments or indirect indicators to measure effectively.

Important to note: It's crucial to clarify the distinction between target and actual reach when discussing direct and indirect impacts. Targets represent the projected number of beneficiaries, encompassing both direct and indirect beneficiaries. In contrast, actual reach quantifies the individuals who have tangibly benefited from the intervention and can be verified.

Counting direct reach is typically straightforward as it entails tallying actual numbers according to project implementation designs (e.g., the number of students in a classroom, households served, patients treated in a hospital, etc.). However, calculating indirect reach poses a greater challenge, requiring a reliable, validated presentation of its value that instills confidence for decision-making purposes.

Common limitations with indirect numbers:

1. Attribution: It can be difficult to attribute indirect impacts solely to the project, especially when multiple interventions or external factors are at play.
2. Data Availability: Gathering data on indirect beneficiaries may be more complex than for direct beneficiaries, requiring additional sources or methods.
3. Measurement Consistency: Indirect impacts may be more qualitative or subjective, leading to inconsistencies in measurement approaches and interpretation.
4. Time Lag: Indirect impacts may manifest over a longer period, making it challenging to capture and attribute them accurately within the project timeline.
5. Contextual Factors: Socio-economic, cultural, and environmental contexts can influence the extent and nature of indirect impacts, complicating their measurement and interpretation.
6. Indirect Chains of Impact: Indirect impacts often involve a chain of events or influences, making it challenging to isolate and quantify the specific contribution of the project.
7. Validity and Reliability: Ensuring the validity and reliability of data used to calculate indirect impacts requires careful consideration of measurement tools and methodologies.

In most organizations, the Theory of Change (TOC) (as well as the project design) is the most common framework used to map out the causal pathways through which interventions lead to desired outcomes, including both direct and indirect impacts. In such cases, organizations adopt the use of multipliers to calculate indirect reach from the direct reach. A multiplier of 3, for example, would mean that for every person reached in a project, 3 more people benefit.

But, how can the reliability and validity of these "multipliers" or indirect reporting be improved? Well these are personal recommendations based on practice and the desire of agencies to get accurate numbers that they can feel confident with.

• Utilize a census method: In cases where the direct reach is a smaller number, it may be suitable to directly inquire from each participant whether they have disseminated the interventions they received to one or more individuals. Subsequently, participants can furnish details such as names, locations, genders, and contact information, where feasible. This approach facilitates the creation of an indirect beneficiary database, significantly enhancing data confidence levels to approximately 90%.
• Use random sampling among the direct project participants: In employing the random sampling technique, monitoring and evaluation (M&E) personnel can utilize a scientifically validated sampling method to randomly select beneficiaries during monitoring reporting. This enables the determination of the percentage of individuals who have disseminated the intervention to others. For example, if a project directly benefits 10,000 individuals, with a confidence level of 95% and a margin of error of 5%, a sample size of 370 can be randomly chosen. Hence in data analysis, if it is determined that 65% of the sample have each reached an average of 2 additional individuals, the calculation for indirect reach would be as follows: (65% 10,000) *2 = 13000 reached indirectly.
• Adopt the use of beneficiary self reporting tool: Another method to capture indirect figures involves providing direct project participants (reach) with a self-reporting mechanism. This tool allows them to share information about individuals they have further assisted through the project's intervention.
• Via impact assessments and qualitative interviews (Focus Group Discussions): This approach resembles random sampling but is distinct in that it is not employed during project monitoring, but rather during project evaluations (annual, midline, or end-of-project evaluations). It proves valuable in situations where project resources are limited and cannot be allocated specifically for monitoring indirect reach through other monitoring techniques.
• Utilize a validated average for ongoing interventions in similar project regions: Ideally, when an organization has established and adopted a multiplier value, such as *6, it is presumed to have undergone rigorous scientific testing and subsequently been embraced as a standard hypothesis within the organization. Consequently, if new or existing interventions are ongoing in the same region, the application of this multiplier for subsequent reporting can be adopted without the necessity of further monitoring techniques. However for new geographical regions, a new value would need to be generated.

In summary, the indirect indicators offer a more nuanced perspective on the impact of the project. Instead of focusing solely on quantifiable outputs, indirect indicators consider broader, often qualitative aspects of change. These may include shifts in attitudes, changes in behavior, or longer-term outcomes that are not immediately measurable. Indirect indicators provide a deeper understanding of the context in which the project operates and the complexities of its effects on individuals and communities. For instance, while a direct number might tell us how many individuals received training, an indirect indicator could reveal whether that training led to changes in practices or attitudes within the community.

How do you define and measure indirect numbers in your context and how do you ensure its validity and reliability?

?

Enjoy the read and share your thoughts

?

Happy Easter