A New Way of Dealing with Forecast Accuracy for Intermittent Demand

Intermittency in demand forecasting is a well-known and challenging problem for retail sales, inventory and operations planners, especially in today’s global supply chain environment.

Intermittent demand for a product or service appears sporadically with zero values, and outliers in the demand data.?As a result, accuracy of forecasts suffers, and how to measure and monitor performance has become more critical for business planners and managers to understand.

The conventional MAPE (Mean Absolute Percentage Error) measure is inadequate and inappropriate to use with intermittent data because of divisions by zero in the formula lead to undefined quantities.

In a previous article, I introduced a new approach, which can be more useful and practical than a MAPE in dealing with inventory planning , financial budget planning, and forecasting for sales & operations (S&OP). What these applications have in common are the multi-step ahead forecasts, whose accuracy needs to be assessed on a periodic basis. I will call this new approach the Forecast Profile Accuracy (FPA) process.

A Smarter Way to Deal with Intermittency in Demand Forecasting

For the past four to five decades, the conventional approaches in dealing with intermittent data has been based on the assumption that inter-demand intervals and the nonzero demand events are independent. This is the logic behind the Croston method and its various modifications and adaptations. But, when you start looking at real data in an application, you discover that the independence assumption may be more of a mathematical convenience than a realistic occurrence.

My new approach examines the data in two stages. First, we note that each non-zero demand event can be preceded by an inter-demand interval of zero demand. In a regular demand history,?each demand event is preceded by an interval of zero duration. Under this assumption, non-zero Intermittent Demand events ID* are dependent on ‘Lag time Zero Interval' (LZI).?The LZI distribution the spreadsheet example is shown in the left frame for a variable labeled LTI. In the right frame, we show the conditional distribution for the ID* given the three durations. They are not the same hence it would not be advisable to assume that they are dependent.

When forecasting for an inventory safety stock setting, for example, it will be necessary to consider the conditional distribution of the non-zero demand. In the spreadsheet above, I have created three forecasts for the 2017 holdout sample. Forecast F1 is a level na?ve forecast with 304 units per month. For ease of comparisons, I have assumed that annual totals are the same as the holdout sample. In practice, this would not necessarily be true. The same is assumed for Forecasts F2 and F3. so that comparisons of forecast profiles are the primary focus. In practice, the effect of a ‘bias correction’, if multi-step forecast totals differ, should also be considered,

The consideration of the independence assumption is?a fundamental difference between a Croston method and the LZI method.

Step 1. Creating Duration Forecasts

When creating forecasts, we first need to forecast the durations. This should be done by sampling the empirical or assumed distribution. In our spreadsheet, LTI_0 intervals occur more frequently than the other two and the LTI_1 and LTI_2 intervals occur with about the same frequency. Then, based on the LTI in the forecast, a forecast of ID* should be made from the data the LTI depends on. An example of this process is given at my profile and on the Delphus website.??

After a forecasting cycle, is completed and the actuals are known, both the LTI distribution and the conditional ID* distribution need to be updated before the next forecasting cycle, so the empirical distributions should be kept current.

Step 2. Creating the Alphabet Distributions

Depending on your choice of a Structured Inference Base (SIB) model for intermittent demand ID* (or the natural?log transformed intermittent demand Ln (ID*), alphabet profiles need to be coded for the actuals and the forecasts from the (conditional) demand event distributions for each of the durations LTI_0 in column C, LTI_1 in column F, and LTI_2 in column I. Note that the sum of the weights in an alphabet profile add up to one.

For the data spreadsheet we need also the add the alphabet weights associate with the duration distribution. This is done by multiplying the conditional alphabet weights by the weights in the (marginal) duration distribution LTI_0 (= 0.4545), LTI_1 (= 0.2727), and LTI_2 (= 0.2727). These coded alphabet weights can be found in columns D, G, and J, respectively.?Note that the sum of the three weights in the combined alphabet profiles add up to 1.

Step 3. Measuring the Accuracy of Intermittent Demand Profiles

In a previous article on LinkedIn as well as my website blogs, I introduced a measure of performance for intermittent demand forecasting that does not have the undefined values encountered with the MAPE (zero demand values in a denominator).. The accuracy measure is derived from the Kullback-Leibler divergence or dissimilarity measure from Information Theory applications.

where a(i) (i = 1,2, …, m) are the components of the Actuals Alphabet Profile (AAP) and f(i) (i = 1,2, …, m) are the components of the Forecast Alphabet Profile (FAP), shown in the spreadsheet below. In the forecast, the lead-time is the horizon m.

Step 4. Creating a Forecast Profile Accuracy Measure

The alphabet profile for the actuals (AAP) is shown in the top line of the spreadsheet above. The alphabet profiles for the forecasts (FAP) are shown in the second, fourth, and sixth lines in the spreadsheet. The calculations for the D(a | f) divergence measure are shown in the third, fifth, and seventh lines with the D(a | f) results shown in bold. They are 22%, 40%, and 16%. respectively, for forecasts F1, F2, and F3.

It can be shown that D(a | f) is non-negative and equal to zero, if and only if, the FAP = AAP, for every element in the alphabet profile. Hence, F3 is the most accurate profile followed by F1 and F2. This should not be interpreted to mean that F3 has the best model. However, it does suggest that the F3 model has the best multi-step ahead forecast pattern compared to the pattern of the actuals. Hence, we consider D(a | f) accuracy for multiple step-ahead forecasting cycles with fixed lead-times..

Hans Levenbach, PhD is Owner/CEO of Delphus, Inc and Executive Director,?CPDF Professional Development Training and Certification Programs.

Dr. Hans is the author of a forecasting book (Change&Chance Embraced) recently updated with the new LZI method for intermittent demand forecasting in the Supply Chain.

With endorsement from the International Institute of Forecasters (IIF), he created CPDF, the first IIF certification curriculum for the professional development of demand forecasters. and has conducted numerous, hands-on?Professional Development Workshops for Demand Planners and Operations Managers in multi-national supply chain companies worldwide. Hans is an elected Fellow, Past President and former Treasurer, and former member of the Board of Directors of the?International Institute of Forecasters.

The 2021 CPDF Workshop Manual is available for self-study, online workshops, or in-house professional development courses.

He is Owner/Manager of these LinkedIn groups: (1)?Demand Forecaster Training and Certification, Blended Learning, Predictive Visualization, and (2)?New Product Forecasting and Innovation Planning, Cognitive Modeling, Predictive Visualization.

I invite you to join these groups and share your thoughts and practical experiences with demand data quality and demand forecasting performance in the supply chain. Feel free to send me the details of your findings, including the underlying data without identifying proprietary descriptions. If possible, I will attempt an independent analysis and see if we can collaborate on something that will be beneficial to everyone.

?