How do you handle uncertainty and volatility in demand forecasting scenarios?

MAPE is the worst metric to use to judge forecast quality. Its use stems solely from its ease of calculation. Beyond the limitations mentioned in the outline, its interpretation is anything but easy. The percentage form deceives the decisionmaker. The formula is easy, but its interpretation is not. The distinction is lost on most practitioners. The range of MAPE is not 0% to 100%. It can easily exceed 100% and 0% is not achievable. It is impossible to say where the true realistic best MAPE value is for any business, product line, or SKU. Lastly, of all forecast quality metrics MAPE has the worst correlation to business value. It is not an indicator of quality, but rather a broken compass.

To handle uncertainty and volatility in demand forecasting, it is critical to consider various factors such as trend, seasonality, random fluctuations, and cyclical patterns. Numerous statistical techniques, including time series analysis, causal models, and machine learning algorithms can help in this regard. You should also monitor the mean absolute percentage error (MAPE) to measure the accuracy of your forecasts. However, MAPE cannot handle zero or negative values, and it can be biased towards low-volume products. To overcome these issues, more advanced methods like probabilistic and ensemble forecasting can be used.

I suggest to run this experiment about MAPE : simulate a time series of sales will be mostly 0, and from times to times, 1. This is our "true sales". Now, compare with MAPE these values with the following "forecasts" : - always 0 - the average sales per period - a uniform random number generator with values between 0 and 1 - a hypothetic forecast which would predict the sales exactly, but every time one day before the actual sale happens. Guess what ? The best in terms of MAPE is the average or the random one (depending on odds), followed by "0 always", and the worst is the one that predicts perfectly one day before.. which would be perfect for replenishment....

Understanding the limitations in your metrics is key. Blindly believing in your metrics can lead to unforeseen issues in performance.

Instead of relying solely on deterministic models, consider try using probabilistic forecasting models, Monte Carlo Simulation to generate mutiple secanarios which helps in range of possible outcomes and their likelihood

Compared to MAPE, WMAPE only improves the weighting and the division by zero issue. However, it suffers from all the same other issues as MAPE.

Being MAPE inspired, this metric has the flaws. Forecasts that are optimized wrt WMAPE are generally poor ??input signals?? for a replenishment computation engine. A few reasons for this : - no distinction between over forecasting and under forecasting while they have much different consequences - they evaluate forecast accuracy over fixed time steps while we actually need a cumulative forecast over a longer period - they weight based on volume, and not based of forecast value added yielding a mix of ??better?? forecasts, most useless and some really useful

It is disappointing, that on a topic of uncertainty and volatility, NOT ONE metric listed actually measures either of those. All 4 metrics listed measure only forecast error (the mean of the forecast uncertainty distribution). MAD and RMSE may be better than MAPE in measuring error, but they do not measure uncertainty. Two aspects of uncertainty are critical to measure: accuracy and precision. Or as known in the data science domain: calibration and sharpness. A pure metric of accuracy is Total Percentile Error (TPE). Other accuracy metrics are pseudo-spherical score and pinball loss function. Metrics of precision are Mean Inter-Percentile Range (MIPR), Inter-Quartile Range (IQR), and Standard Deviation.

Scenario analysis and sensitivity analysis are like a bandaid on a severed limb. They only address the uncertainty around assumptions, not around outcomes. They address questions of the type: what-if, then what? This is important for long time horizons and big impacts. However, most uncertainty in supply chains is not of this kind. Most uncertainty revolves around everyday variability and volatility. Lost sales, stockouts, expediting, inter-depot transfers, expiration, obsolescence, and waste are all caused by that, and for most companies the much more painful problem to address. The only way to properly model this uncertainty, and stabilize the supply chain is with probabilistic forecasting and planning, and probabilistic metrics.

why not try conformal prediction? This approach stands out because it doesn't tie you down to one model or method (model-agnostic) and adjusts smoothly with new data coming in. Basically, it looks at how your latest forecasts stack up against what happened before (conformity), offering dynamic prediction intervals. Also, conformal prediction and probabilistic forecasting methods are not mutually exclusive.

Para manejar la incertidumbre y la volatilidad en los escenarios de previsión de la demanda, se pueden aplicar herramientas de análisis predictivo, modelos de predicción de demanda, metodologías de gestión de la incertidumbre, generación del plan de demanda y verificación de la fiabilidad de los datos. Estas prácticas ayudan a asegurar la cobertura en todo momento y a alinear la producción a la demanda real del mercado, protegiendo el flujo y segmentando estratégicamente la cadena de suministro.