How do you align your logic model and theory of change with your evaluation questions and methods?

Start by clearly defining your logic model and theory of change. Understand the intended outcomes, activities, outputs, and inputs that are part of your program or initiative . Secondly based on your logic model and theory of change, develop evaluation questions that will help you assess the effectiveness of your program in achieving its intended outcomes. These questions should be specific, measurable, achievable, relevant, and time-bound (SMART).

Logic models and theory of change are powerful tools for planning, implementing, and evaluating programs and initiatives. A logic model visually represents the sequence of activities, inputs, outputs, and outcomes of a program, showing the logical connections between them. On the other hand, a theory of change outlines the underlying assumptions, pathways, and causal relationships that drive program success. Together, they provide a roadmap for understanding how interventions lead to desired outcomes, helping organizations make informed decisions and achieve meaningful impact.

Great point Simon - I agree wholeheartedly- wonderful summary of the theory of change- love the story telling aspect of Evaluation - please keep them coming! ??

In developing or revising the ToC, it's helpful to take a step back, or let's say, a step upward, to see the broader purpose the program contributes to. For instance, a road infrastructure program might have been initiated to address an increase in the country's accident-related deaths. Understanding the program's initial pathway is important for drafting the theory of change and its visual representation (a logic model, a results framework), depicting this journey. Once this is done, the next step is to come up with questions that clarify things like how the program's chosen path contributed to its bigger goal, the various ways it followed to achieve this, and how considerate it was of the surrounding factors, to name a few.

From a proposal standpoint, this alignment shows you have deeply considered the importance and realism of your intended outcomes, and how they could be monitored by funders. From a programmatic standpoint, such high-level evaluations allow you to "check in" on progress without unduly burdening on-the-ground personnel who are busy day to day. The strategic insights measured by logic-model level monitoring can serve as early warnings, telling programmatic staff where to look for more actionable data. E.g., your logic model tells you, "by day 365 we should see a 10% increase in attended births." You only see 4%. You should check data on numbers of mothers enrolled. If those data are positive, consider your communication methods with enrollees.

Firstly, evaluation questions serve as signposts along the way. They ensure you ask the right questions at each process stage when aligned with the Logic Model. E.g., if one step involves selecting the right seeds, the evaluation question might be, "Did the chosen seeds lead to healthier crops?". Similarly, the ToC provides a deeper understanding of the underlying assumptions and beliefs driving your actions. By aligning it with evaluation methods, you check whether your beliefs hold; like periodically stopping to ensure you're still on the right path. E.g., if your ToC states that providing proper nutrients to the crops will lead to a better harvest, your evaluation method might involve soil testing and analysis to verify this assumption.

Alignment between logic models, theories of change, evaluation questions, and methods ensures that the evaluation is accurate and meaningful, providing insights into the program's functioning and impact. Aligning these elements helps in: Clarifying Expectations: It becomes clear what the program intends to achieve and how success is defined. Focused Evaluation: Alignment guides the selection of appropriate evaluation methods, ensuring that data collection is targeted and relevant to the program's objectives. Holistic Understanding: It facilitates a holistic understanding of the program by connecting the dots between inputs, activities, outputs, and outcomes, fostering a more nuanced evaluation.

I specialize in the Progressive Outcomes Scale Logic Models (POSLM) methodology. Using this method evaluation questions are organically embedded into the evaluation design by assessing the challenges incurred as each stage is reached and as actionable and measurable indicators are accomplished.

I specialize in the Progressive Outcomes Scale Logic Models (POSLM) methodology. Using this method evaluation questions are organically embedded into the evaluation design by assessing the challenges incurred as each stage is reached and as actionable and measurable indicators are accomplished.

A useful bridge between the theory of change on one hand and your data gathering on the other, is to spend time considering; what does success look at at each point in the theory of change, and how would you know that this has been achieved? This helps to focus on what data you will need to gather or select from what you already have. It will also help you to ask questions of your data during analysis, and therefore it steers the sensemaking. A key advantage of aligning a theory of change with data collection is that the former gives you a framework for analysis, which makes the process of analysis and writing so much more streamlined and straightforward.

I agree that there is no single approach to this task. Michael Quinn Patton, in one of his insightful videos, identified 101 different evaluation approaches . In fact, each evaluation question might be seen as a study in its own. Sometimes, a single data collection tool may serve to address multiple questions, and often, different approaches may be necessary for each question. While several factors affect the selection of the appropriate approach, the scope and complexity of our evaluation define its boundaries and guide our decision on the most suitable approaches.

Consider context before, during, and after creating your questions. Do data baselines (e.g., census data) exist and are they accurate? Can such data be collected? In digital form or on paper? Can they be transmitted? How? Is it ethical to collect the data you want, in the culture impacted? Once the program starts, are you getting the answers you need to show progress toward logic model-specified outcomes? Given the context, should you rephrase the question, or change data collection methods?

Identify Key Components: Start by identifying the key components in your logic model and theory of change. Formulate Evaluation Questions: Develop evaluation questions that directly correspond to the identified components. Select Appropriate Methods: Choose evaluation methods that align with your evaluation questions. Develop Indicators: Define specific indicators that will measure the success or progress of each component in your logic model and theory of change. Collect Data: Implement the chosen methods to collect data on the identified indicators. Analyze and Interpret Results: Analyze the collected data in the context of your logic model and theory of change.

Remember that theories of change are not always realised. While it is really important to align data collation or collection with the theory, so that you can test your theory of change, it is good to leave room for the unanticipated and the unexpected, by stepping back, asking more open questions, or inviting different points of view at different stages of the programme so outcomes and the routes towards them can be better interrogated and understood.

Flexibility: Evaluation plans should allow for flexibility. If unexpected factors arise during the evaluation, being able to adapt the methods or adjust the logic model may be necessary. Stakeholder Involvement: Engage stakeholders in the alignment process. Their input can provide valuable perspectives on the logic model, theory of change, and evaluation questions. Continuous Learning: Treat the evaluation as a learning opportunity. Ethical Considerations: Be mindful of ethical considerations, ensuring that the evaluation respects the rights and well-being of participants and adheres to ethical standards in research and evaluation.

Aligning logic models and theories of change with evaluation questions/methods ensures coherent assessment in renewable energy projects. For subsea wind turbines, evaluate efficiency gains through a logic model emphasizing technological advancements. Offshore wind projects may focus on theory of change, linking capacity building to long-term sustainability. Grid code compliance studies require evaluation questions probing adherence and methods like data analysis. Manufacturing projects benefit from aligning with a logic model emphasizing process optimization, with evaluation methods measuring cost-effectiveness. Integrating these elements ensures comprehensive evaluation in renewable energy initiatives.

Consider using evaluation methodologies which are inclusive of community voice in the evaluation design and not solely reliant upon evaluation methodologies developed by academia and/or white dominant culture. Inclusive evaluations offer a much better opportunity to evaluate the effectiveness of programs serving marginalized communities.