How do you handle missing data and non-response in item response theory analysis?

1 Missing data types

Missing data can occur for various reasons, such as skipping items, omitting answers, or dropping out of the test. Depending on the mechanism and pattern of missingness, missing data can be classified into three types: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). MCAR means that the probability of missingness is independent of both the observed and unobserved data. MAR means that the probability of missingness depends only on the observed data. MNAR means that the probability of missingness depends on the unobserved data, such as the true ability or item difficulty.

2 Missing data methods

Choosing a method for dealing with missing data depends on the type and amount of missingness, as well as the assumptions and goals of the analysis. Complete-case analysis is a simple option, but it can reduce the sample size and introduce bias if the data are MAR or MNAR. Alternatively, imputation can fill in the missing values with plausible estimates based on the available data. Full-information methods such as FIML or multiple-group IRT use all observed data to estimate parameters and standard errors while accounting for the missing data patterns. However, these methods require strong assumptions and complex models.

3 Non-response types

Non-response is a special case of missing data, where the entire response vector for a case is missing. Non-response can occur for different reasons, such as refusal to participate, inability to contact, or attrition. Depending on the mechanism and pattern of non-response, non-response can be classified into two types: unit non-response and item non-response. Unit non-response means that no data are available for a case. Item non-response means that some data are available for a case, but not all.

4 Non-response methods

The choice of method for dealing with non-response depends on the type and amount of non-response, as well as the assumptions and goals of the analysis. Weighting adjusts the weights of the cases to account for the non-response rates and characteristics of the non-respondents. Imputation fills in the missing values for the non-respondents with plausible estimates based on available data. Alternatively, full-information methods use all available data and model the non-response mechanism as part of the analysis. Weighting can reduce bias and increase representativeness, but it can also increase variance and error in estimates. Imputation can preserve sample size and reduce bias, but it can also introduce uncertainty and error in estimates. On the other hand, full-information methods can provide efficient and unbiased estimates, but they require strong assumptions and complex models.

5 Missing data and non-response tools

Various tools and software can help you handle missing data and non-response in IRT analysis, such as R, a free and open-source programming language with many packages and functions for IRT analysis. Additionally, Mplus is a commercial software that allows you to perform advanced statistical modeling, including IRT models and multiple imputation; while Winsteps specializes in Rasch models, which are a subset of IRT models, and can handle missing data and non-response using different methods.

6 Missing data and non-response tips

When handling missing data and non-response in IRT analysis, it is important to assess the type and amount of missingness in your data set and consider the possible causes and consequences. You should then select the most appropriate method for dealing with the missing data and non-response, based on your research questions, assumptions, and available resources. It is also important to compare different methods and check the sensitivity and robustness of your results to different scenarios and assumptions. Finally, you should report and document your procedures and results, as well as acknowledge any limitations or uncertainties of your analysis.