How do you evaluate the effectiveness and efficiency of your mixed mode data collection strategy?

2 Choose your modes and samples

The next step is to decide which modes and samples you will use and how you will combine them. There are different types of mixed mode designs, such as sequential, concurrent, or adaptive, depending on the order, timing, and flexibility of the modes. You also need to consider the characteristics and preferences of your target population, the availability and quality of sampling frames, the coverage and response rates of each mode, and the budget and timeline of your project. You should choose the modes and samples that best suit your objectives, indicators, and constraints.

3 Test and optimize your questionnaire

Before you launch your data collection, you should test and optimize your questionnaire for each mode and sample. You can use various techniques, such as cognitive interviewing, pilot testing, or split-ballot experiments, to identify and address any issues or biases that may arise from the mode of administration, the wording or format of the questions, the response options or scales, or the survey layout or navigation. You should aim to minimize the mode effects and maximize the comparability and usability of your questionnaire across different modes and samples.

4 Monitor and adjust your data collection

Once you start collecting data, you should monitor and adjust your data collection process according to the feedback and results that you receive. You should track and analyze the response rates, the completion rates, the response times, the missing or inconsistent data, the satisfaction or complaints of the respondents, and the costs and efficiency of each mode and sample. You should also compare and contrast the data quality and outcomes across different modes and samples, and look for any patterns or discrepancies that may indicate mode effects or other sources of error or bias. You should be ready to adapt your data collection strategy if needed, such as by switching or adding modes, modifying or clarifying questions, or adjusting incentives or reminders.

5 Weight and integrate your data

After you finish collecting data, you should weight and integrate your data to account for the different modes and samples that you used. You should use appropriate weighting techniques, such as propensity score adjustment, calibration, or raking, to correct for any differences or imbalances in the sample composition, the response probabilities, or the measurement errors across different modes and samples. You should also use appropriate integration techniques, such as pooling, meta-analysis, or latent class analysis, to combine or compare the data from different modes and samples, and to assess the robustness and sensitivity of your findings.

6 Report and communicate your results

The final step is to report and communicate your results in a clear and transparent way. You should document and explain your mixed mode data collection strategy, the rationale and assumptions behind it, the challenges and limitations that you faced, and the solutions and adjustments that you applied. You should also present and interpret your results in a way that acknowledges and accounts for the potential mode effects and other sources of uncertainty or variation in your data. You should highlight the strengths and weaknesses of your mixed mode data collection strategy, and provide recommendations for future research or practice.

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?