Make Sure the Right People Answer Your Survey

One of the biggest problems I have seen in DIY surveys, second maybe to leading questions, is not appropriately qualifying respondents. Frankly, I find this to be one of life’s ironies because so often when I have conducted a survey that came up with some unexpected insights, those who were uncomfortable with the insights would be the first to point out that we had “obviously surveyed the wrong people.” 

That is the go-to excuse for learning that awareness of your brand is not dominating the market like your sales people have told you. (As an example. And one that I heard just last week in a workshop on a brand equity study for a B2B company.) That’s why qualification questions should lead your survey.

The first part of any good survey should be a set of qualifications--questions, that screen out those who do not fit the survey’s target audience. Sometimes this section is referred to as the screener--because of the screening ability. (Duh.)

Before you start to create the actual questions, in my experience, the best thing is to determine what the screening criteria should be. Just start writing the questions, and you are bound to go overboard and to also miss the mark in one way or another. 

An easy place to start is with specific demographics. Are there minimum age and maximum age limits? Is gender a targeting criteria? How about employment, or employment in a specific industry? 

I am a big fan of focusing the behavioral components of a qualified respondent. For example, if you are surveying soft drink consumers, making sure someone has purchased soft drinks, possibly a minimum number or specific flavors, in the past month or three months. And make sure they are the ones making choices about which to buy. Often, criteria will revolve around specific behaviors. For example, if I am doing a concept test for a hiking boot, I will want to make sure the survey respondents are hikers.

Criteria can be used to create subgroups as well. Maybe with the boots concept, I want to understand the the reaction of hardcore hikers as well as more casual hikers to the concept. I might define those by how many hikes they make per month or quarter and then assign them to one of the subgroups of interest based on their answers.

Be careful to not overscreen. A common trap I have seen, even among sophisticated marketers, is to so narrowly define a target audience that it represents a severely small niche--one that could in no way be profitable to serve or even effectively target. And often narrowing the target audience is done to make the results favorable to the brand--like screening out respondents who rate a brand below 8 on a 10-scale. (You might be laughing--who in the world would do that? Yeah, been there, seen it done.)

Setting the criteria first is essentially setting up your screening strategy. Once you have done that, it is a relatively straightforward task to write the actual screening questions. That is my topic for tomorrow. And the day after, we will tackle the pros and cons of the infamous security screens. 

__________________________________

I have been involved in the market research field, as a corporate-side research manager, a bulletin board platform co-founder, and then a research agency owner for over 35 years now. I have learned a thing or two about doing research better. My intent with these articles is to share practical and thought provoking ideas to elevate your research game, whether you are the person conducting the research or the person using it to make better decisions. (Or if you wear both hats!) 

Twitter: @TedKendall1