What do p-values and coefficients tell you in regression analysis?

In regression analysis, p-values and coefficients help understand the relationships between variables. P-values tell you if the effect or relationship between variables is real or just happened by chance. A low p-value (usually less than 0.05) means the relationship is likely real and significant. For example, if you're studying if hours studied affect test scores, a p-value less than 0.05 suggests that studying more hours does indeed affect scores significantly.

In regression analysis, a coefficient is the result of first of analysis of input data potentially indicating a relationship between predictor (independent) and predicted (dependent) variables. A P-value is a result of second analysis of the input data for testing whether the coefficient-indicated relationship may be mere chance, in which case there may be no real relationship, i.e., the relationship may actually be null. Mathematicians call this a test of the “null hypothesis,” which emphasizes that the test has limitations. Accordingly, they do not say that a low P-value shows the coefficient-indicated relationship is true. Instead, it is said that that a low P-value fails to indicate the coefficient-indicated relationship is null.

The regression analysis is a part of inferential statistics, the p-values in regression determine whether the relationships that you observe in your sample also exists in the larger population. The linear regression p-value for each independent variable tests the null hypothesis that the variable ( coefficient Beta) has no correlation with the dependent variable.

In regression, p-values are used to understand relationships between variables. High p-values (above 0.05) that are commonly used indicate that there is no observation of proven results, and are not statistically significant.

For example, if you're studying if hours studied affect test scores, a p-value less than 0.05 suggests that studying more hours does indeed affect scores significantly. Coefficients show the strength and direction of this relationship. In our example, if the coefficient is 2, it means for every additional hour studied, the test score increases by 2 points.

When you want to interpret the coefficients you need to check first the sign of the coefficient as it will tell you whether there is a positive or negative correlation between each independent variable and the dependent variable. A positive coefficient indicates that as the value of the independent variable increases, the mean of the dependent variable also tends to increase

Usually, when a p-value is greater than the significance level, it indicates there is insufficient evidence in your sample to conclude that a non-zero correlation exists, when you want to check the p-values, it is standard practice to use the coefficient p-values to decide whether to include variables in the final model

Practically speaking, it is important to analyze the context of this variable in the defined business problem. Although we have a p-value that exceeds the significance level defined in the analysis, it is necessary to evaluate its impact in the entire context.

The p-value measures the probability of obtaining results at least as extreme as the observed results, assuming the null hypothesis is true. Therefore, if the p-value is low, it is unlikely to observe such extreme results if the null hypothesis were true, so we are inclined to reject the null hypothesis.

The coefficient value signifies how much the mean of the dependent variable changes given a one-unit shift in the independent variable while holding other variables in the model constant, the linear regression coefficients in your statistical output are estimates of the actual population parameters

In testing significance, mathematicians are careful what they say about P-values, e.g., tending to say what they don’t show. They say, in essence, that a low P-value fails to show that the coefficient-indicated relationship is null, not that it’s true. Likewise, they don’t say that a high value shows the relationship is false. This way of speaking may be partly because P-values don’t reflect effects that may arise due to missing predictors, too many predictors and small data sets.

In the analytical context, although they are essential, in addition to p-values, we can analyze the R2 value, the F test, the sample normality test, the analysis of residuals and their behavior, the confidence intervals. This entire set of tools can help solve the business problem.

For selection of predictors to include in a regression analysis model, non-independence or correlation among predictors has been a significant issue in my experience. If the number of potentially available predictors permits, this issue might be addressed by modeling every possible combination of the predictors and comparing relevant statistics of the models. Alternatively, it can be addressed by removing coefficients that are highly correlated with one another. A matrix of correlations among the predictors can also help in predictor selection. However, compare the resulting models since correlated predictors can sometimes improve results.