How can you use mixed-methods to improve program evaluation?

1 What are mixed-methods?

Mixed-methods are a type of research design that integrates both quantitative and qualitative data and analysis in a single study or evaluation. Quantitative data are numerical and can be measured, counted, or compared using statistical methods. Qualitative data are textual or visual and can be interpreted, categorized, or explained using thematic or narrative methods. Mixed-methods can use different types of data collection techniques, such as surveys, interviews, observations, documents, or focus groups, depending on the purpose and scope of the evaluation.

2 Why use mixed-methods?

Mixed-methods can offer several advantages for program evaluation, such as providing a comprehensive and contextualized understanding of the program. It can also enhance the validity and reliability of the evaluation findings by cross-validating results from different sources and methods. Additionally, mixed-methods can address the limitations or gaps of one method with the strengths or insights of another method. Finally, it can generate more actionable and relevant recommendations by explaining the reasons and mechanisms behind the results, as well as involving stakeholders in the data collection and interpretation.

3 How to design mixed-methods?

Designing mixed-methods evaluations requires tailoring to the specific program context, goals, and questions. Generally, one should define the evaluation purpose, questions, and criteria; identify the main stakeholders; choose the appropriate design; select data collection methods and tools; analyze the data; and report and disseminate the findings. To begin, define the evaluation purpose and stakeholders, then choose between a concurrent or sequential design. Afterward, select valid, reliable, and ethical data collection methods and tools that best suit your evaluation questions. Analyze the data with appropriate techniques for each type of data and integrate the results using different strategies. Finally, report and disseminate the findings using different formats to reach your intended audiences.

4 What are the benefits of mixed-methods?

Mixed-methods can bring many benefits to your program evaluation, such as increasing the depth and breadth of findings, improving the quality and rigor of the evaluation, and enhancing the usefulness and applicability of the evaluation. This is done by providing more evidence and explanations for results, validating and verifying findings, generating relevant and actionable recommendations, and engaging stakeholders in the data collection and interpretation process. These elements can reduce biases and errors that may arise from using a single method while also increasing ownership and commitment to program improvement.

5 What are the challenges of mixed-methods?

Using mixed-methods for program evaluation can pose some challenges, such as requiring more time, resources, and skills to plan and manage multiple data collection and analysis methods. Additionally, it demands more flexibility and adaptability to adjust or modify the evaluation design, methods, or tools. Moreover, it can create ethical and practical dilemmas by needing to balance or reconcile the different expectations, interests, and values of the stakeholders. It's also important to ensure the confidentiality, anonymity, and consent of the participants.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?