How do you choose between different methods of variable selection when dealing with collinearity?

1 What is confounding?

Confounding occurs when a third variable, called a confounder, affects both your treatment and your outcome variables. For example, if you want to test the effect of smoking on lung cancer, age may be a confounder, because older people are more likely to smoke and to develop lung cancer. Confounding can bias your estimate of the causal effect of your treatment, and make it appear stronger, weaker, or even opposite to the true effect.

2 What is collinearity?

Collinearity occurs when two or more variables are highly correlated, meaning that they share a lot of information. For example, if you want to test the effect of education on income, years of schooling and degree level may be collinear, because they both measure the same aspect of education. Collinearity can make your estimate of the effect of each variable unreliable, and increase the variance and uncertainty of your model.

3 How to detect confounding and collinearity?

One way to detect confounding is to use a causal diagram, also known as a directed acyclic graph (DAG), which shows the causal relationships among your variables. A confounder is a variable that has a path to both your treatment and your outcome variables, and is not on the causal path between them. For example, in the smoking-lung cancer example, age is a confounder, because it has a path to both smoking and lung cancer, and is not on the causal path from smoking to lung cancer.

One way to detect collinearity is to use a correlation matrix, which shows the pairwise correlation coefficients among your variables. A high correlation coefficient indicates a high degree of collinearity. For example, in the education-income example, years of schooling and degree level may have a high correlation coefficient, indicating a high degree of collinearity.

4 How to deal with confounding?

One way to deal with confounding is to use a randomised controlled trial (RCT), which randomly assigns subjects to different treatment groups. This ensures that the treatment groups are balanced on all potential confounders, and that any difference in the outcome is due to the treatment. For example, in the smoking-lung cancer example, you could randomly assign subjects to either smoke or not smoke, and compare their lung cancer rates.

Another way to deal with confounding is to use a statistical adjustment, which controls for the confounder in your analysis. This can be done by using a regression model, which estimates the effect of your treatment variable on your outcome variable, while holding the confounder constant. For example, in the smoking-lung cancer example, you could use a logistic regression model, which estimates the odds of lung cancer for smokers and non-smokers, while adjusting for age.

5 How to deal with collinearity?

One way to deal with collinearity is to use a variable selection method, which reduces the number of variables in your model. This can be done by using a criterion-based method, which selects the variables that maximise or minimise a certain criterion, such as the Akaike information criterion (AIC) or the Bayesian information criterion (BIC). For example, in the education-income example, you could use the AIC or the BIC to select the variable that best explains the variation in income.

Another way to deal with collinearity is to use a dimension reduction method, which transforms your variables into a smaller set of new variables that capture the most important information. This can be done by using a principal component analysis (PCA) or a factor analysis (FA), which create new variables that are linear combinations of your original variables. For example, in the education-income example, you could use a PCA or a FA to create a new variable that represents the overall level of education.