Is your survey following all the best practices?

P.S: I encourage you to go through the entire blog post, but should your reading time today be even limited than you expected, then please scroll down to the summary section at the bottom. A bonus sample survey on taking a COVID-19 pulse check of your friends (Board, staff, donors, etc.), along with a strategy to analyze it, is linked both in summary and in the LinkedIn comment section of the post. ?Every minute you have for reading should empower your perspective for the minutes you are not reading.

Online surveys (or any survey for that matter) are one of those age-old topics that have an immense amount of research available through books and the internet. Have you ever thought – why is that the case? Or, why do we still have surveys in the world that are not following these best practices? For starters, this is because this research method does not have any hard and fast rule in the back-end (something that mathematics formulas will never allow), every survey turns out to be different and most often not following the best practices. So, the question is, what could this blog post tell you that you may not know already? When doing a survey, it is essential to focus on how you can connect the objective of your survey with the analysis you produce out of it. Here are a few survey design principles aimed at setting you up with success in deriving insights out of it. 

1. Understand the ‘why’ behind your survey

Your survey should have a purpose, a “why” that will motivate the design. The clearer you are in your objectives behind it, the better would be the questions you ask. Without a clear purpose, surveys tend to deviate in any direction, and you risk losing the short attention of your survey respondents.

2. Ensure the length of your survey is not more than 5-7 mins

Once you have established the objective behind your survey, ensure that the questions you ask do not go into a rabbit hole. Your aim should be making the most efficient use of the little time your survey respondents are willing to give you. Ideally, 5-7 mins or 15-20 questions should be your goal.

3. Keep the wording of your questions simple

The text of your questions directly impacts the quality of data you receive. Your wording should not be what is simple to understand for you but your audience. Look for words that are common for your audience. Careful choice of words reduces the time at your respondents’ end to understand the question or imagine the scary scenario when they do not understand but mark an answer choice/leave blank just to complete your survey.

4. Avoid asking double-barreled questions

Double-barreled questions are when you ask for feedback on two separate things within a single question. For example, “How would you rate the quality of our customer support and the knowledge they have shared in solving your problem?”. This classic example of asking double-barreled questions can confuse your respondents when they have a differing opinion about “overall quality” of your customer support vs. “the knowledge” your customer showed. This confusion can trip the responses you receive, thus, once again, putting in danger the quality of the data you collect.

5. Avoid leading and biased questions

Your choice of words (as described in point #3) is just as critical as how you phrase your questions. Any question that already reflects some positive or negative thoughts can be injecting bias in the minds of your respondents. For example, “Mango ice-cream is the most preferred dessert in Canada. How would you rate the quality of mango muffins, as compared to the mango ice-cream?” Well, first, this is a hypothetical question, and hence “mango muffins” shall gladly remain hypothetical. But, on a serious note, the moment you add a positive/negative tune to your statement, your respondents have a higher chance of believing that than approach your question with a neutral standpoint. This bias can lead to invalidating your analysis later.

6. Be careful about your open-ended questions

Avoid open-ended questions as much as possible. Your surveys are not your interviews or focus groups intended to facilitate conversation. Your survey is the only research method that holds the capabilities of both qualitative and quantitative research. More the number of open-ended questions, more are chances of having scarce responses (not all respondents prefer to answer open questions), and higher are the chances of you not getting the answer of the primary objective of your survey.

7. Choose your question types carefully

This blog post has established by now that your objective of doing a survey is the key. Because that determines what you ask. But just as important is what you ask, it is also important how you ask your questions to your respondents. What that means is to choose your question types appropriately. For example, if you need only one answer, use radio buttons. This point is doubly important because you should select your question types from two standpoints – readability and analysis. Readability means your chosen question type should be easy to read both on web and mobile devices. That’s why you should have many answer options; using a drop-down radio is much better than a simple radio button question. Similarly, any question type that is adding to the scroll bar should be re-considered. Analysis, on the other hand, means your chosen question should have an appropriate analysis method at the back end to extract insight out of it.

8. Make your answer choices well-rounded

Your answer choices reflect your depth in covering all aspects of your question. It is, therefore, a best practice to not just add the required options but also, in some form or shape, variants like “all of the above,” “none of the above,” “others,” etc.

9. Ensure to set up your scales correctly

Your analysis of the survey depends on how you have set up your questions, especially your scale-based questions. Make sure your scales have appropriate weights. As a general best practice, avoid using even-numbered scales because that removes a neutral ground. For example, the following odd-numbered scale is a preferred method:

- Very helpful (weight assigned: 5)

- Helpful (weight assigned: 4)

- Neither helpful nor unhelpful (weight assigned: 3)

- Unhelpful (weight assigned: 2)

- Very unhelpful (weight assigned: 1)

10. Ensure the logical flow of your survey

Imagine going into an interview. Would you talk about your education, then childhood, then a little about your last job, followed by your school again? Hopefully not! In the same way, your survey is representative of your thoughts to your respondents. Maintaining a logical flow not just ensures the comfort of your respondents to understand your intention behind the questions but also build an unspoken form of trust that the party asking me questions knows what they are doing. Making this kind of confidence, based on the quality of your survey, is the most straightforward form of marketing that you can do for your purpose. So, ensure that your surveys have a logical flow from one section to another and not just a bunch of random questions thrown in one single page.

Summary:

1. Understand the ‘why’ behind your survey

2. Ensure the length of your survey is not more than 5-7 mins

3. Keep the wording of your questions simple

4. Avoid asking double-barreled questions

5. Avoid leading and biased questions

6. Be careful about your open-ended questions

7. Make your answer choices well-rounded

8. Use matrices or grids carefully

9. Ensure to set up your scales correctly

10. Ensure the logical flow of your survey

11. BONUS: Here is sample questionnaire along with the analysis strategy to take a pulse check of your organization’s all friends (donors, staff, volunteers, etc.).