The impact of lengthy screeners on respondent engagement
In a recent member survey 26,425 members told us their pet peeves with surveys and overwhelmingly we were told how frustrating it was to be constantly screened out of surveys.?
While targeting is not always appropriate, those in research should consider the impact of screen outs on respondent engagement, panel churn, data quality and online panel sample scarcity.
Read on to see what panel members had to say:
Key takeaways:
Panelists?
If they constantly screen out of surveys, panelists are going to stop doing surveys or try anything to qualify. A reduction in long screening sections should improve the panelist experience and in turn increase data confidence.?
领英推荐
Researchers
Some may think there is a limitless supply of good respondents and take high quality data for granted but it takes a lot of resources, time, energy and effort to maintain a quality panel, we need to try and take care of them.? If we are not listening to our respondents, we risk losing their engagement, motivation and honesty in completing our surveys.
Consider targeting respondents where appropriate, screening out respondents as early as possible and paying respondents for useful data you collect before screening them out.
Panel Providers
Profile your members deeply and regularly update data that has been collected as their circumstances might change.
Incentivize members for long screener sections and consider? limiting screeners to a maximum amount of questions/time.
About Octopus Group
Octopus Group has grown to be one of Australia’s largest and most trusted online survey panels. It is dedicated to the proper incentivization, education, support and management of the respondent experience and has resulted in a reputation of continued growth and data quality.
Talk to us about the profiling we have on our survey members and what we are doing to ensure the highest standard of data quality in the industry.
?Nobody loves surveys as much as I do ? Data Fairy ?No buzzwords allowed?? Quirk's Award & Insight250 Winner
2 年One challenge we have is that we sometimes use the screener section of the questionnaire to size the market so it includes questions that are not for screening purposes per say. Splitting up into two studies would not be efficient especially on a low IR studies. These screeners tend to be longer and we have no way of compensating the panelists who screen out as there is only one redirect for completes. If there is a solution to this conundrum, I'd love to know!
#Cultivate Better Workplaces
2 年Very important topic (as far as MR topics go). I wonder how many suppliers indeed push back on long screeners and sub-standard surveys, or whether it is just easier to take the revenue. Until there is a concerted effort to act as guardian against poor survey experiences, participation may one day rely entirely on bots, click-farms and 12 year olds saving for a new scooter. Which suppliers are willing to step up?
Managing Director @ PureSpectrum | Leadership, Strategy, Growth
2 年There are some good home truths here Octopus Group. Profile point mapping is definitely the preferred option however how do you keep an accurate ongoing profile of your respondents? We do not live in the days where 200+ data points per panelist is viable (it never was). Outside of key demographics which should be mandatory you need to approach profiling in a dynamic and efficient manner so you continually gather / confirm the information BUT do not overwhelm and exhaust the panelist experience. It is a balancing act and one that many either fail to do properly (or do not try at all). Unless you are working with the same clients over and over there are always going to be data points you do not know in advance. This is where pre-screening, post survey questions and screeners come into their own, that being said you are 100% correct that these should be limited and kept to the absolute essentials to avoid fatigue and dis-engagement. Marketplaces and 'panel companies' need to own this space as much as possible and also marshal their clients to make sure that conversion rates are at an acceptable level.
Human-centred researcher | Rapt | Co-founder & Phase One Insights
2 年I totally agree. As an industry we need to get much better at treating survey participants as people, not commodities. People do us a huge favour by giving us their time and attention, and answering surveys for relatively little reward. We should repay their generosity through careful survey design, engaging and relevant questions, and a general respect for their time.