The Praise of the Recontact Rate: A Note on Respondent Quality and Replication Studies
Data quality has remained a pressing concern in market research for nearly a decade. The proliferation of traffic sources permitting one-time participant engagements has lessened the emphasis on cultivating long-standing value for respondents. This shift toward a more "gig economy" model of researcher-participant interactions concurrently heightens the need for stricter quality oversight. Last year, key industry associations such as ESOMAR, the Insights Association, and MRA came together to spearhead a global initiative to tackle the growing data reliability and integrity issues. Brand-led coalition CASE4Quality is also playing a crucial role in this effort.
One fundamental obstacle in this pursuit is the elusive nature of defining and measuring data quality in market research. Beyond industry norms and standardized operational processes (such as ISO 20252), quality is a multi-layered concept with varying implications depending on the context in which it is evaluated.
In the following note, I will discuss the importance of measuring the 'recontact rate'—the percentage of respondents who can be successfully re-engaged over time—as a key metric for assessing data quality and respondent reliability.?
The Evolution of Sampling Methodologies
Two prominent trends have shaped the current sampling landscape: the increased use of blended data sources and the rise of "river sampling," as an alternative to traditional online panels.
Initially, data blending emerged as a practical solution for aggregating panel capacities to execute complex projects. Over time, it also evolved into a strategic approach aimed at diversifying suppliers and mitigating risks. Today, while blending is accepted and offers potential methodological benefits, fully realizing these advantages requires a more sophisticated understanding of how diverse sources can be integrated based on respondent motivations rather than mere business convenience, - a point that still needs further research on research and deeper understanding.?
Simultaneously, the growing reliance on traffic sources has shifted the research industry toward a gig economy model. The adoption of APIs and automation has streamlined the sampling process, enhancing efficiency at the cost of respondent engagement. Poor user experiences stemming from excessive redirects, and inadequate compensation contribute to respondent fatigue and declining participation rates among high-quality contributors. Unlike traditional access panels, which foster long-term engagement, river sampling prioritizes transactional one-off participation. This "race to the bottom" has led to a proliferation of fraudulent respondents and diminished overall data reliability. As quality respondents exit the system, the proportion of bad actors increases, exacerbating the problem. While this method allows for greater flexibility and cost efficiency, it also presents significant challenges in maintaining data consistency and respondent reliability.
In this context, fraudsters have become increasingly sophisticated in gaming market research surveys. Tools such as Generative AI, emulators, and VPNs allow fraudulent participants—including bots and click farms—to convincingly mimic real respondents. Unlike traditional fraud, where patterns such as excessive licking or gibberish responses were easily detected, AI-assisted fraud can generate well-written, logical responses that pass many conventional quality checks. The reliance on short-term engagement models demands heightened scrutiny to ensure data reliability.
Introducing Respondent Quality
At this point, a crucial distinction must be made between good data and good insights. While high-quality insights often stem from reliable data, this is not always true: bad insights can emerge from good data due to flawed analysis or interpretation. Conversely, poor-quality data inevitably leads to unreliable insights, making it imperative to scrutinize the foundational elements of research methodologies, - the classic garbage in, garbage out.
Sustained interaction allows for a deeper analysis of respondent behavior, making it easier to identify patterns of inattentiveness, dishonesty, or disengagement.
The causes of bad data are multifaceted, often arising from methodological flaws. Poor survey design is a common culprit—when questions are ambiguous, overly complex, or repetitive, respondents may lose interest, leading to rushed answers. Likewise, in a frustrating respondent experience that offers inadequate incentives, participants may not invest the effort required for thoughtful responses. Even when the questions are clear and engagement is high, sampling issues can distort findings. If the selected group does not accurately reflect the target population, the insights drawn will be skewed.
To navigate these evolving challenges, research practitioners must critically evaluate their data collection methodologies. A fundamental question to ask is: Would you personally be willing to participate in your surveys? If the answer is no, it may be time to reconsider the approach.
Ensuring high data quality also involves evaluating respondent quality. The integrity of research findings heavily depends on the individuals providing the data. Respondent quality can be divided into three key criteria: truthfulness, thoughtfulness, and assertiveness.
While brief engagement methods can provide initial insights into respondent quality, long-term panel research offers a more effective approach to assessing and maintaining high-quality data. By fostering ongoing engagement, researchers can better evaluate respondents’ consistency and reliability. This sustained interaction allows for a deeper analysis of respondent behavior, making it easier to identify patterns of inattentiveness, dishonesty, or disengagement. Ultimately, investing in long-term engagement strategies enhances the ability to measure and ensure the quality of respondents, leading to more accurate and actionable insights.
The Recontact Rate as a Foundational Measure of Respondent Quality
A key concept in evaluating data quality is the recontact rate, which serves as an empirical indicator of respondent reliability. Recontact rate reflects the proportion of participants who can be successfully re-engaged at different time intervals—such as three, six, or twelve months—demonstrating their continued availability and willingness to participate in research.
This metric aligns closely with the principles of scientific inquiry, where validity and reliability are foundational. In scientific research, results must be replicable to establish credibility. Similarly, a high recontact rate suggests the capability to reproduce a prior study to determine if the same results can be achieved. Replication is the cornerstone of science, it strengthens the overall body of knowledge and quality, - it is worth mentioning, that it is equally important to periodically refresh respondent panels to mitigate biases that can emerge over time; long-term participants may develop response patterns, familiarity effects, or other biases that could compromise the objectivity of findings.
By balancing recontact efforts with strategic panel renewal, researchers can ensure data remains reliable. Establishing best practices around recontact rates and respondent engagement will be crucial in ensuring that the market research industry produces dependable and meaningful insights in the years to come.