How do you interpret the results of PCA in terms of the original features?

1 What is PCA?

PCA is a mathematical procedure that finds the directions of maximum variance in a dataset and projects the data onto a lower-dimensional space. The new features, called principal components (PCs), are orthogonal to each other and capture the most variation in the data. The first PC explains the most variance, the second PC explains the second most variance, and so on. The number of PCs is equal to or less than the number of original features.

2 How to perform PCA?

To perform PCA, you must first standardize the data to have zero mean and unit variance. Then, calculate the covariance matrix of the standardized data and find the eigenvalues and eigenvectors of the covariance matrix. After that, sort the eigenvalues in descending order and select the top k eigenvalues and their corresponding eigenvectors, where k is the desired number of PCs. Then, form a matrix P with the selected eigenvectors as columns and multiply the standardized data by P to obtain the PC scores.

3 How to interpret the PCs?

The PCs are linear combinations of the original features, which means that each PC has a coefficient for each original feature. These coefficients, also called loadings, indicate how much each original feature contributes to the PC. The loadings can be positive or negative, which means that the original feature can have a positive or negative correlation with the PC. The magnitude of the loadings reflects the importance of the original feature for the PC. The higher the absolute value of the loading, the more important the feature is.

5 How to choose the number of PCs?

When performing PCA, one of the challenges is deciding how many PCs to keep for the dimensionality reduction. To help with this decision, you can look at the explained variance ratio, which is the proportion of variance explained by each PC. You can plot this as a scree plot and look for the elbow point, where the slope of the curve changes significantly. Additionally, you can consider the cumulative explained variance, which is the sum of the explained variance ratios up to a certain PC. You can choose a threshold, such as 80% or 90%, and keep the PCs that reach that threshold. The Kaiser rule suggests keeping only the PCs that have eigenvalues greater than 1. Lastly, you can examine the loadings and the biplot to see if the PCs make sense in terms of the original features and the research question. This depends on your domain knowledge and the purpose of the analysis.

6 How to use the PCs for further analysis?

Once you have performed PCA and obtained the PC scores, you can use them for further analysis, such as clustering, regression, and classification. Clustering is a technique that groups similar data points together based on their distance or similarity, and you can use the PC scores as the input for clustering algorithms, such as k-means or hierarchical clustering. Regression is a technique that models the relationship between a dependent variable and one or more independent variables, and you can use the PC scores as the independent variables and perform linear or logistic regression. Classification is a technique that assigns a label or category to a data point based on its features, and you can use the PC scores as the features and train a classifier, such as a decision tree or a neural network, to predict the label of a new data point. This can help you avoid multicollinearity, which is a problem that occurs when the independent variables are highly correlated with each other.