If I Had More Time, I Would Write an Easier Survey

January is a time of new beginnings, where people rush to the gym and the health foods aisle, set up spreadsheets to save money, and entertain others with tales of how they will be better partners, colleagues, and friends.

In the spirit of January and resolutions, it is also a good time to revisit the age-old refrain for market researchers: make online surveys easier to take.

Market research insights are riddled with examples of grand vision but poor subsequent execution (much like New Year’s resolutions, actually). And often, as the title of this post would suggest, it is a result of not having a full understanding of the time it takes to design an engaging survey, one that produces high quality data and takes minimal “effort” for a survey taker to complete.

This “effort” can take the form of survey length, and here, the data cannot be ignored. According to YouGov’s latest GRIT report, over half of research clients believe that the maximum length of a survey should be 10 minutes or less. And yet, that same report reveals that over 60% of survey providers estimate their average survey length to be 15 minutes or more.

This “effort” can also take the form of mental resources required to process and accurately respond to a question. And here, data isn’t even required for many of us to conjure up a recent survey experience involving endless amounts of rows and columns… with unseemly chunks of text… with lists of 10 or more items that must be ranked completely.

"If you claim not to have time to write an easier survey, then you might think twice about what else you spent that time on that masqueraded around as being more valuable."

You would imagine how relieved you might feel as a survey taker if a researcher were to present you with a more effortless survey. But lest we forget, the exercise of doing this benefits the researchers just as much. A survey that requires less effort will be met with less respondent fatigue, more accurate and in-depth responses, and lower drop-out rates… all of which can be summarized as more easily obtained, higher quality, and just plain better data.

Looking back at GRIT, only 40% of clients were at least very satisfied with their research partners, with key weaknesses cited as being “quality of data” and “quantity of data.” They might as well have said “everything that really matters.” A vast majority of researchers and clients alike believe that quality is the most important consideration when it comes to designing and executing a study.

And therein lies the point: if you want to spend your time wisely with a survey, prioritize a large chunk toward making questions engaging and easy for a respondent to answer. If you claim not to have time to write an easier survey, then you might think twice about what else you spent that time on that masqueraded around as more valuable.

I would not be very sporting if I left you with the grand vision but no tips on execution. So here are some ways that my colleagues at LinkedIn and I would recommend trimming the fat and reducing the effort required to take a survey:

• Think about whether your survey is mobile-optimized! Again GRIT tells us that well over 50% of all surveys are not mobile optimized. And yet, according to a white paper from Decipher, the ratio of survey takers that start on mobile is swiftly growing and approaches 30%. Thankfully, thinking consciously about making a survey mobile-optimized also channels the goal of reducing effort (e.g. by removing the need for excess scrolling just to see the whole screen).
• Be conscious of the fact that some information may be largely difficult for participants to recall; avoid these scenarios or give them some help.
• Skip out on the consistent use of large grid questions. If necessary, split grids into multiple, shorter ones, or show only one row of the grid at a time.
• Avoid long paragraphs of text – split out introductions / transition statements from questions onto their own pages, bold or underline important words, and include a visual instead of text to facilitate comprehension.
• Instead of asking people to rank a long list of words/phrases, ask for the top 3 only. (You will also thank yourself for doing this when you analyze the data and find you have more focus.)
• When in doubt, form an analysis plan before fielding the survey, and nix questions that are not central to that analysis plan.
• Overall, keep survey length less than 10 minutes, and ideally less than 8 minutes (we must keep those mobile survey takers in mind!).

 
We can do it, market researchers! This new year, let’s take the time to give back to our generous research participants. Besides all aforementioned great things, we may also welcome that warm, fuzzy feeling that comes along with doing something right.

--
I am indebted to Dan Kent-Smith and Karen Bills for their thoughtful advice on previous drafts of this post.