Three Keys to effective qualification questions

Contrary to much of public opinion, a well crafted screening section in a survey is not as easy as it looks. (I seriously had a client once ask me to lower the price because all I did, according to them, was “flip a switch to turn the survey on." And that is an actual quote.) 

I see so many mistakes made in the screener, all of which will lead to one of two negative outcomes:

1. Letting people into the survey that don’t really match your target audience, or
2. The opposite. Disqualifying even people who should qualify, and thus increasing the difficulty and cost of finding qualified respondents.

Whole classes could be given on writing effective screening sections. But here are three keys to get you started. 

1 Avoid Yes/No questions

Some pitchers have tells that telegraph their pitch. And batters take advantage of that. Using a Yes/No question telegraphs the qualifications. Those that are fraudulent, they know exactly how to answer to qualify. But even those who are not, may discern what you want, and human nature and the need to please, may lead them to fudge a little in answering the question. The easiest way to avoid a Yes/No question is to turn it around into making the object of the question into one answer and then including the other, the “wrong” choices in the answer list. For example, change:

 This year have you learned to bake bread? Yes or No

To:

This year, what new things have you learned to do?

• Make soup
• Bake bread
• Make sausage
• None of these

2 Randomize question answers

Nothing makes it more obvious, except maybe a yes/no question, what answer you are looking for than making the correct qualification option the first in line. The easy solution--randomize your answer options. Of course, the exception to that rule is when the answer options should be in some sort of order--like age breaks. But how often do you randomize the gender options? Most people don’t think to do that--which not only is bad practice, it also often perpetuates gender inequalities, albeit in a very subtle way.

3 Use the respondent’s language

One question that breaks this rule that I have seen way too many times is to ask a respondent, “Have you participated in a focus group in the last 6 months?” This is a common screening question for a qualitative recruit. And guess what--nine out of ten respondents can’t tell a focus group from a survey. Why? Because they don’t speak market research. I can’t tell you how many times I have heard someone, including my own clients, refer to a focus group as a survey. If research end-users can’t keep it straight, how can we expect respondents to? There are any number of ways we can mess up when we don’t use consumer language. The problem is that they are answering a question without really understanding it. And that does not bode well.

There is more you can do to make your screeners more effective, but do these three things and you will be well ahead fo the game.

__________________________________

I have been involved in the market research field, as a corporate-side research manager, a bulletin board platform co-founder, and then a research agency owner for over 35 years now. I have learned a thing or two about doing research better. My intent with these articles is to share practical and thought provoking ideas to elevate your research game, whether you are the person conducting the research or the person using it to make better decisions. (Or if you wear both hats!) 

Twitter: @TedKendall1