What is the weights used in weighted least square regression and Gaussian Weighted least regression ( LOWESS)?

Weighting in Weighted Least Squares (WLS)

In Weighted Least Squares (WLS), the weights are used to correct for heteroscedasticity (unequal variance in residuals). The goal is to give higher importance to data points with lower variance and lower importance to data points with higher variance.

General Weighting Formula in WLS

wi=1/σi^2

where:

• wi = weight for observation i,
• σi^2 = variance of residuals for observation i

Why Use Weights in WLS?

• If residuals have constant variance → OLS works fine.
• If residuals have varying variance → WLS corrects for this by assigning weights inversely proportional to variance.

Common Ways to Assign Weights in WLS

1. Inverse Variance Weighting (Most Common)

wi=1/σi^2

• If residual variance σi^2 is large, weight wi is small.
• If residual variance σi^2 is small, weight wi is large.

? Used when variance changes across observations. ? Example: Used in econometrics, finance, and experimental physics where measurement errors vary.

2. Inverse Absolute Residual Weighting (Robust WLS)

wi=1/∣ei∣

where IeiI is the residual from an initial OLS fit.

? Used when variance is unknown but residuals are a good proxy.

? Example: Applied in robust regression to reduce the impact of outliers.

3. Weighting Based on a Power Function

wi=1/ ∣Xi∣^α

where α is a chosen exponent.

? Used when variance is proportional to a power of the independent variable.

? Example: In time series models where variance increases with time.

Key Takeaways

1. OLS assumes constant variance (homoscedasticity).
2. WLS corrects for heteroscedasticity by giving lower weights to high-variance points.
3. Inverse variance weighting is the most commonly used method.
4. Robust WLS (using absolute residuals) can be used when variance is unknown.

Locally Weighted Regression (LOWESS/LOESS), and Gaussian process models

This weight function is used to assign higher weights to nearby points and lower weights to distant points, making it useful for:

1. Locally Weighted Regression (LOWESS/LOESS): Instead of fitting a global model, this method fits a regression model only using nearby points, where "closeness" is determined by the Gaussian function.
2. Kernel Density Estimation (KDE): Used in probability density estimation to assign weights to points based on their distance.
3. Gaussian Process Regression: Used in machine learning to model complex functions with uncertainty.

IF you are looking for supply chain solutions using machine learning or analytics or statistics modeling, please email us at [email protected] or WhatsApp us @ +91-7993651356

#Supplychain #Supplychainmanagement #Supplychainsolutions #Supplychainanalytics #Inventoryoptimization #Inventorymodeling #Inventory #Logistics #MathnalAnalytics #Inventoryissues #Stockoutissues #Overstockissues #Forecastissues