How can you analyze questionnaires with missing data?

More than the fact that there is missing data, the most crucial question is: What is the underlying reason for its absence? Here are the possible scenarios: - Poorly designed questionnaire/Human errors [not easily fixable] - Data entry/processing errors [potentially fixable] - Pre-planned Skip logic [no issue to solve] - Non-Response - Ask Why did the respondents do so? Get to the bottom of it as it might impact how you interpret the datapoint e.g. Was the question sensitive in nature? Were the answer options not exhaustive? - Data-cleaning miss [fixable] - Mid-survey dropouts [not fixable]

I am less concerned with missing data per se than whether there is enough data collected in your sample to make statistically valid inferences. That being said, the best way to deal with missing data is NOT to have to deal with it at all, through better questionnaire design. You can reduce the amount of missing data by keeping the survey burden light & making it cognitively easy for survey takers to fill out your survey. When you've done your best & minimized missing data, deal with the missing data that is left by realizing that missing data is a social phenomenon. Focus on whether data are missing for systematic reasons that create biases or whether the effect of the missing data is relatively random and trivial.

Well first, market research wise, I applaud the shrewd exploitation of LinkedIns users to generate free content for the site itself. And, since I'm choosing not to answer this question because I'm not getting paid I'm a great example of the many stories that missing data can tell.

Missingness can act as a confounder and an effect modifier, meaning missingness can have homogenous and heterogenous effects across stratum for any type of analysis. It can make observed associations appear stronger or weaker than their true value. It is unfortunately the practice of many to ignore or delete missing data in their analysis and report results that therefore may lack critical context. Understanding why missingness occurs in your dataset as well as understanding appropriate methods to assess and address missingness separates excellent analysts from poor ones. Likewise, a poor understanding of missing data can lead to a complete waste of an otherwise well collected dataset.

Missing data is not just a methodological problem; it's a social problem. Always look into which social factors may have contributed to the missing data. Are the missing data the result of your data collection methods or variations in those methods? Or are the missing data the result of variations in the demographics of your respondents (e.g., race, gender, age)? As for the basics of counting missing data, I typically use the command sum(is.na(x)) in R & that usually gives me a the quick frequency count of missing data I need.

There are three types of missing data, Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR). MCAR assumes the data is missing independent of observed or unobserved data, i.e there was a fire and some responses were lost. MAR assumes that missingness in the data is explained by other observed variables. If a survery questionnaire about mental health was administered and 25% of the responses to a question assessing anxiety/depression status was left missing, but further analysis shows that another variable gender explains the missingness i.e 100% of the missing data comes from men then the data is MAR. MNAR assumes the missing data cannot be explained by any of the observed variables.

For data Missing Completely at Random, assuming the missingness is less than 5% a complete case analysis is appropriate. However, rarely will you be able to make the assumption that the missing data is MCAR. If the data is Missing At Random there are several options depending on the type of analysis being conducted. A good first step is identifying the magnitude of the effect of the missingness. In a regression analysis this can be by assigning a dummy variable to the missing responses and assessing the significance of the parameter estimate of the newly coded "missingness" factor. Depending on the strength of the effect you may choose to use Pairwise Deletions, Mean Substitutions, or any number of imputation methods to handle missingness.

When dealing with missing data, I typically use the R command rm.na = TRUE, which allows you to make a statistical calculation while ignoring all the missing values. I find that is much more practical in a corporate setting than using complex imputation techniques, because in most cases, (1) imputation rests on unproven assumptions & (2) imputation can be almost impossible to explain non-quantitatively trained stakeholder/leaders. If you are going to use imputation, your imputation method should be an "official" method that is adopted department-wide & is officially documented in its methodology, including any statistical equations for imputing values & any computer code for automating the process of imputation.

I've found that marking a question as required within the settings of creating a survey can eliminate the worry of having missing data.

When it comes to avoiding missing data, I find that increasing sample size is more effective than marking a question as required. You will always have some missing data. The goal is to have a sample size large enough & representative enough so that the missing data does NOT matter. On survey platforms, you can force a respondent to reply to a question, but I think this should be used sparingly, because (1) you force people into giving "non-answers" or socially desirable answers to questions they don't care about & (2) you increase the likelihood they will bail on the entire survey altogether. Attract flies w/ honey not vinegar. The best way to avoid missing data is to design surveys to be short, non-burdensome, and a joy to complete.

1. Be fully transparent when data is missing. Don't try to sweep it under the carpet 2. In your report, explain why the data is missing 3. If the missing data significantly influences the interpretation, take ownership of it. However, if its impact on the interpretation is minimal, be sure to highlight this to reassure the audience 4. Consider showcasing alternative data points that may partially address the gap created by the missing data

I view large, robust, and representative sample sizes as more important than missing data per se. I always want a large response rate, because that minimizes missing data, but if your organization has low response rates on employee surveys, that is a data point in itself. I report sample sizes, response rates, and missing data, but you also have be to careful that some stakeholders have a confirmation bias & they might use missing data to reject a disfavored hypothesis, but they don't take into account the missing data relevant to their favored hypothesis. Whatever you do with missing data, you must do your best to ensure that missing data is not used to reinforce confirmation bias or to be used as an excuse for inaction.