How can you use polynomial regression effectively?

Selecting Polynomial Degree: 1. Visual Inspection: Plot data with varying degrees to choose a degree that fits well without excessive complexity. 2. Cross-validation: Split data, fit different degrees, and choose the degree with the best validation performance. This is equivalent to experimentation, and checking out which degree works the best. 3. Bias-Variance Trade-off: Balance bias and variance by selecting a suitable degree. 4. Information Criteria: Use AIC, and BIC to penalize complexity. Select the degree with the lowest criterion. 5. Domain Knowledge: Consider prior understanding of the problem to guide the choice of degree. 6. Regularization: It helps in selecting the degree, e.g. using LASSO, or RIDGE regularization.

What I find missing in most of such discussions is the subtle role played by the scatter plots in providing a good visual assessment of linear/curvilinear relationships between the characteristics of interest. We focus on the curve that fits the dots (in the scatter) best and ignore the scattered data points altogether. The plot also enables us to explore potential outliers that can visibly impact the nature of relationship between the variables. Exploratory data analysis must remain the first step in any 'regression' (in this context) ventures.

Polynomial regression is an extension of linear regression, allowing for a curved relationship between the dependent and independent variables. Instead of fitting a straight line, it uses a polynomial function (quadratic, cubic, etc.) to model complex patterns in the data. This technique is valuable when the relationship is non-linear, offering a more accurate representation of the underlying behavior. The degree of the polynomial determines the complexity of the model, influencing the trade-off between model flexibility and overfitting. Polynomial regression is widely applied in various fields like physics, economics, biology, and engineering to capture intricate relationships within the data.

Yes cod is always dicey as its a non decreasing function.Engage the use of Information criterion mainly the bayesian as it's a strictest measure

The assumption of normality is 'mostly' a 'rare' phenomenon in this era where there is no dearth of data. Approximate normality is what we have to look for in the context of model building exercise.

Coefficient of determination must never be seen as the sole trusted metric in model evaluation process. Choice of sample size can have a marked effect on the expected value of the coefficient of determination.

In cases of polynomial regression, regression analysis works well until the data exhibits 2nd or even 3rd order linearity. In these situations, the model might either underfit or overfit, potentially leading to a decrease in model accuracy. This is when gradient boosting or boosted regression analysis can be used to mitigate these issues and improve predictive performance.

Always add error bars to your results. If you can, try to calculate which parameters can change the error and how these changes propagate through the model