???? Discovering Adjusted R-squared: Your Guide to Better Regression Models! ????

Why the Adjusted R-Square get increase only by adding a significant variable to the model? What is Mathematics and Logic Behind it? How does it work?

To get answer all questions we need to go step by step through each single topic.

Let's Understand first R-Square.

R-Square – (The Coefficient of Determination) It explains the variation of the dependent variable which is explained by the independent variables in the model that's why it uses as a measure of goodness of the fit of the model. it ranges from 0 to 1. If the value is closer to 1 we consider the model is well fitted or our model is good. or if it is closer to 0 we consider the model is worse.

The value of R-Square always increases by adding variables in the model whether it's not significant or not contributing significantly to the model.

This is because the additional variables introduce more flexibility to the model, allowing it to capture noise and random fluctuations in the data.

But R-Square Adjusted only increases when we add significant variables to the model.

The logic and mathematics behind why Adjusted R-squared increases when adding significant variables to the model are based on the principles of model complexity and overfitting.

Model Complexity and Overfitting:

While a higher R-squared may seem desirable, it can lead to overfitting. Overfitting occurs when a model fits the noise and random fluctuations in the training data, resulting in poor generalization to new, unseen data. In other words, an overfitted model may perform well on the training data but poorly on new data.

Penalizing Complexity:

Adjusted R-squared addresses the issue of overfitting by penalizing the model for including unnecessary predictors. It takes into account the number of predictors and adjusts the R-squared value accordingly. The formula for Adjusted R-squared is:

The idea and Mathematics is that the penalty term (Adjusted term) (n - 1) / (n - p- 1) increases as the number of predictors (p) increases. If an added variable does not contribute much to explaining the variation in the response, the increase in R-squared is offset by the increase in the penalty term, resulting in a smaller increase (or possibly a decrease) in Adjusted R-squared.

So, the simply R-Squared Adjusted tells us when we add a new variable in the model whether it leads to a significant increase in the R-Square or not. and if R-Square increased significantly means the variable we added is significant to the model.

Significant Variables:

When you add a significant variable to the model, it means that this variable contributes to explaining the variation in the response. Since the variable adds meaningful information, the increase in R-squared is not offset by a large increase in the penalty term, leading to a noticeable increase in Adjusted R-squared.