How do you address the attribution and contribution issues in your program evaluation?

Theory-based evaluation offers a strategic approach to tackle attribution and contribution issues in program evaluation. By clearly defining program theory and logic models, evaluators can trace outcomes back to specific program activities, ensuring accurate attribution. Additionally, using mixed-methods approaches allows for a comprehensive assessment of contributions, capturing both quantitative data and qualitative insights. This rigorous methodology enhances the credibility and validity of evaluation findings, providing stakeholders with actionable insights for informed decision-making.

Program theories and logic models are useful tools but they come with their own set of challenges. A detailed program theory or logic model can be time-consuming and needs a lot of input from stakeholders. It can also be tough to account for all the different assumptions, contextual factors, and external influences, which might lead to the program theory or logic model having oversimplified conclusions. Also, sticking to predefined models can limit flexibility in your programming.

Too often, the development of the programmatic theories and/or logic models neglects to include the perspectives of stakeholders and beneficiaries from marginalized populations or underserved communities of color. I have often seen these contributions included as afterthoughts with minimal revision to the original framework. It is imperative to teach our students and mentees that the need to include stakeholders and beneficiaries from every level of the project is not merely a recommendation or optional suggestion.

To start, I would conduct a comprehensive needs assessment to understand baseline conditions and identify contextual factors, establishing a clear understanding of the environment in which the program operates. Using a mixed-methods approach, incorporating both qualitative and quantitative data, will allow to capture a nuanced understanding of the program's impact. Contribution analysis and process tracing further help explore the program’s plausible contribution. For example, when evaluating a community health initiative aimed at reducing childhood obesity, I mapped out the causal pathways in the logic model and systematically gathered evidence to confirm or refute each step.

I often use process tracing to rule out possible causes of observed results. For example, while doing monitoring and evaluation (M&E) for a job training program, I mapped out each participant's journey from initial enrollment to employment. By tracing this process, I pinpointed where external factors like local job market conditions or individual support systems played a role. I combined this with qualitative methods, like participant interviews, to uncover additional influences. This detailed analysis helps differentiate between what the program directly caused and what external factors influenced. This provided a clearer picture of the project's true impact.