Screening Question Best Practices

Our business is built on being the follow-through team for researchers who need to get answers from a target audience. This role goes beyond data collection alone, even though data collection services are primarily what we provide. A key area of expertise for us is in consulting on a provided questionnaire to be sure that it will achieve the intended purpose of the research. The success or failure of a research project is often determined by the level of preparedness when starting data collection.

In any given week it is common for me to personally review 10 questionnaires. Often times, there were already multiple decision makers who reviewed it before I ever saw it. Their focus is usually on making sure the questions being asked will collect the right data they need for key insights. The success or failure of our project team is strongly impacted by it. Where do I focus my effort?

First, I go right to the introduction and screening criteria. How are we going to represent ourselves when we first interact with a respondent? What questions will be answered to determine qualification for taking the survey?

Once the core research questions are asked, there is often a transition out of the survey and into a section for collecting basic demographics and, if necessary, incentive fulfillment information. To get answers from people, you have to be very good at managing this transition. Any errors on these seemingly simple questions lead to a poor respondent experience, poor data quality, and difficulty in doing analysis.

Below are three of the best practices we implement in reviewing these screening and demographic questions. For a complete list of the 9 best practices we use, plus a free template to help write a few of the most common screening questions, go here. https://hubs.ly/y0XK3j0

1. Use Simple Language - Jargon and complicated wording creep into these questions all the time. Do we really need to ask, "What is your estimated annual household before taxes in 2015 including any wages, alimony..." OR could we ask "Which of the following ranges include your estimated household income for this year?"
2. Ensure people answering "other" really are "other" - Ask yourself this question. If you were the respondent, would you know which category you fell into, or would you possibly self identify as "Other"? The "Other" or "Don't Know" option in a screening question probably terminates. Make sure this will only end up excluding the people you want it to. A common example of a question where this is an issue is in B2B research asking about how involved someone is in the decision making process for what you are doing research on.
3. Allow respondents to refuse to answer questions - Forcing a response and not allowing someone to refuse to answer a question creates a real issue for data quality. For example, my argument is that it is far better to have an unspecified group for age than an inaccurate one because those that did not want to provide their age simply chose something so they could move on.

I welcome feedback on the content here and what others look for when reviewing questionnaires. We shared more best practices and a question template in our latest free guide. https://hubs.ly/y0XK3j0