How do you choose the optimal number of components for PCA with missing data?

1 Missing data mechanisms

Before applying PCA, you need to understand the mechanism that generated the missing values in your data set. There are three main types of missing data mechanisms: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). MCAR means that the probability of a value being missing does not depend on any other variable. MAR means that the probability of a value being missing depends on some observed variables, but not on the missing value itself. MNAR means that the probability of a value being missing depends on the missing value itself or some unobserved variables. The type of missing data mechanism affects the validity and performance of PCA.

2 Missing data imputation

One way to deal with missing data is to impute or fill in the missing values with some reasonable estimates. There are many methods for missing data imputation, such as mean, median, mode, regression, k-nearest neighbors, or multiple imputation. The choice of imputation method depends on the type and pattern of missing data, the distribution and correlation of the variables, and the goal of the analysis. Imputation can reduce the bias and variance of PCA, but it can also introduce some error and uncertainty.

3 Missing data analysis

Another way to deal with missing data is to perform PCA directly on the incomplete data set, without imputing the missing values. This can be done by using algorithms that can handle missing data, such as expectation-maximization (EM), singular value decomposition (SVD), or probabilistic PCA (PPCA). These algorithms can estimate the parameters and components of PCA by maximizing the likelihood or minimizing the reconstruction error of the observed data. Missing data analysis can avoid the drawbacks of imputation, but it can also be more computationally intensive and sensitive to the initial values.

4 Number of components selection

Regardless of whether you impute or analyze the missing data, you still need to decide how many components to retain for PCA. There are several criteria for choosing the optimal number of components, such as the proportion of variance explained, the scree plot, the Kaiser rule, the elbow method, or the cross-validation error. The proportion of variance explained measures how much of the total variation in the data set is captured by each component. The scree plot displays the eigenvalues of the components in descending order and shows where they level off. The Kaiser rule suggests retaining only the components with eigenvalues greater than one. The elbow method looks for a point in the scree plot where the slope changes sharply. The cross-validation error evaluates how well the components predict new data. The choice of criterion depends on the purpose and quality of PCA.

5 Practical tips

To choose the optimal number of components for PCA with missing data, you need to take into account several factors, such as the type and amount of missing data, the imputation or analysis method, and the selection criterion. Before applying PCA, it is important to explore and understand your data set and the missing data mechanism. Additionally, you should compare different imputation or analysis methods and evaluate their accuracy and robustness. Furthermore, use multiple criteria and visualize the results to find a reasonable balance between complexity and information. To validate your PCA results, you can use other methods or domain knowledge. Despite being a difficult task, PCA with missing data is useful for statistical modeling. By following these steps and tips, you can choose the optimal number of components for PCA with missing data and enhance your data analysis skills.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?