Why You Keep Missing Your Service Level Targets

In this article I expose a?root cause of why most companies?consistently fail to meet the service levels they?target?and then I provide a means to address it.

We forecast demand and understand that we need to provide buffers to cover for the uncertainty in demand and the error in our forecast.?And we use?generally accepted formulas to?determine safety stocks to set proper buffer levels. Yet, we still miss our?service level targets or reach them only with significant effort and expense expediting. Isn't the purpose and promise of safety stocks to meet our targets without expediting? Shouldn't expediting be necessary only to exceed our targets?

There are many reasons safety stocks could be failing to meet their objective. We could have a wrong formula. We could be missing some key contributing factors to the uncertainties against which we aim to buffer. Very often, we provide the wrong input to our formula. The?most common error is assuming uncertainty or forecast error is symmetric. Whenever you use?absolute errors (MAD, MAPE,...) or squared errors (RMSE, standard deviation, ...) in your safety?stock formula, this is the error you are committing, pun intended. This article demonstrates the cause and effect on service level of the symmetry assumption.

The Traditional Perspective of Forecast and Error

The traditional approach to forecasting is to take historic demand and fit a curve through it, then assume the curve will continue into the future. This serves as a forecast. This works pretty well for time series of fast-moving items or at high levels of aggregation (e.g. across all regions for a single item). But when applied to the level of granularity where forecast accuracy really matters - the level at which we keep inventory and plan shipments - they break down. As an example see the?figure 1 below?showing 3 years of?weekly demand of?a medium-moving item for a single shipping location. It is completely stationary with an average demand value of 20 units per week.

Figure 1: 3 years of weekly demand in green. Fitted forecast in red.

The average value of 20 is the forecast quantity. To determine our safety stock let's assume we use a standard deviation?of the difference between our fitted forecast and the actual demand. In the above sample demand time series that standard deviation is also 20. If we were to put this in a typical safety stock formula, where we have 1 week lead time and target 99% case fill service level we would require 47 units safety stock. Due to the simplicity of this scenario we can simply use the 99% cumulative probability of a normal distribution with mean and standard deviation of 20 each. This yields 67 units as the 0.99 probability level, and after subtracting the 20 units forecast, results in the 47 units safety stock.

Note that any other scenario would make this a more complex case open to many more interpretations. But we take the approach above without loss of generality to the argument against the symmetry assumption.?

Now, let's start exploring where the above?approach breaks down.

First, we graph the historic demand differently. We get rid of the time perspective, which is fine since the time series is completely stationary. Along the horizontal axis we list the weekly demand quantities, and vertically we count how many times those quantities occurred.

Figure 2: count occurrences of each historic demand quantity in green. Average (mean) forecast in red.

We can clearly see from figure 2 that which was not so evident in figure 1: the demand is not symmetrically distributed. This a very typical for medium-moving items at this weekly-location granularity. For slow-moving items the asymmetry is much worse. If we?super-impose the distribution assumed by using a standard deviation in our safety stock formula we get figure 3 below:

Figure 3: actual demand distribution (green) and one assumed for the safety stock calculation (red)

There are two key observations evident in?figure 3. First, we are underestimating the probability near the peak - the red curve is lower than green bars. Second,?we cover for negative demand, which in practice will never happen - returns/cancellations should not be forecasted as negative demand. The first observation is not necessarily bad. But the negative coverage lowers the total probability of positive demand to less than 1, meaning on average we underestimate demand.

For completeness of the demonstration I will show the effect of covering for the negative demand error versus not doing so. A common assumption is to treat the negative part of the distribution to cover the cases of zero demand. This assumption however introduces bias: when all negative values are rounded up to zero the average increases. For the normal distribution the actual mean would be 22 units, or a 10% bias. Even though we forecasted a zero bias mean, when we view the complete distribution assumed in the output used for safety stock calculations a strong bias is evident!

A Correction on the Traditional Perspective

We should correct this bias.?When we use a normal distribution cauterized at zero we need to target a lower mean than the actual to achieve a mean that matches the actual after cauterization. In this case using a mean of 18.35 and a standard deviation of 17.21 gives a non-negative distribution (after cauterization) with an unbiased mean of 20,?a standard deviation that is slightly high (20.5), and a cumulative probability at the peak that matches the actual demand. Note that other values could have been chosen, but the standard deviation after cauterization will always be too high, and improvement at the peak would be traded off against improvement at the tails.

Figure 4: adding a cauterized version to remove bias from the normal distribution when negative demand is assumed to be zero.

This new version of the normal distribution clearly improves on both of the identified issues with the first version. However it is also clear it still does not fit the distribution of the actual demand closely.

An Asymmetric Alternative Perspective

The skewed bell shape suggests this may be a lognormal distribution. Figure 5 shows a lognormal fit:

Figure 5: fitting a lognormal distribution to the actual demand

A lognormal distribution with parameters mu = 2.645 and sigma = 0.83255 seems to fit pretty well. It has no negative or even zero demand, just like the actual, and its peak is roughly the right height. When we show all three distributions together (figure 6 below) we can also see the tail is longer/fatter which seems to match the actual more closely:

Figure 6: 3 different distributions fit to the actual demand

Using the same logic as used for the normal distribution above the safety stock levels can be determined for the two additional distributions. The normal distribution yielded a safety stock level of 47 units to achieve a 99% service level. For the?cauterized normal distribution it results in?39 units, whilst for the lognormal distribution we see a steep increase to 78 units . These?three values are vastly different. At most one can be accurate. But which one? And how bad are the others?

The Results

When running 10,000 simulations using random samples from the historic demand the results are as follows:

Table 1: results of simulation for three distributions

• The traditional approach misses the mark by 2%! For no other reason than its symmetry assumption. All other factors were ignored in this simple case.
• The cauterized version (which addresses the error/bias in the normal distribution) is worse. It must be noted though that other parameter choices?may have yielded better outcome in this specific test. But as I hope to demonstrate?in future articles it comes at expense elsewhere.
• At least one well-chosen asymmetric distribution achieves the service it targets, proving no other interfering factors were at play here.

So, by visual inspection of distribution graphs in this simple stationary case, or in hindsight, we can demonstrate that the symmetric assumption is lacking!

Could this Service Level Shortfall?Be Avoided?

In real life the problem is not nearly as easy to analyze as the simplistic example provided above. With non-stationary demand patterns or patterns under heavy influence of causal factors the distribution graph above does not apply. Even if it would apply visual analysis of such graphs for every item/location does not scale.?Thus, so far?I have demonstrated a problem, but have not provided any solution.

The question is: could we have predicted this in a scalable way??

The answer is Yes! We could use a metric that does not just consider the average expected value, but the entire distribution of the uncertainty. Any metric that considers only the mean (e.g. MAPE, MAD, RMSE, MASE, etc.) would give equal value to each of these three options since they all have the same constant mean of 20 units. MAD = 12 units and?MAPE = 60% for all three. One metric that does consider the entire distribution of the forecast is the Total Percentile Error introduced in an earlier article.

For each of the three distributions the unit-based Total Percentile Error (TPE) based on just the historic demand is:

From these results it is clear the TPE provides information where the MAPE does not. Since the TPE penalizes any error in the distribution - not just the error at the 99 percentile -?the correlation between TPE and severity?of the service level shortfall, while it?does exist,?is not complete. This is evidenced by the TPE of the cauterized normal distribution being less than the TPE of the plain normal, because the latter has a larger error in ranges of the distribution other than the upper?tail.

Conclusion

The traditional approach of providing an average expected value (per item and period) as forecast plus some symmetric indicator of error clearly has fundamental issues. It is the direct cause of targeted service levels not being met. The simplistic example illustrated a small but significant service level shortfall of 2%. In more realistic scenarios this shortfall is usually much larger. There are many reasons for this. For example, when lead times are greater than the 1 week used here. But primarily because in most companies the long tail of slow-moving items consists of more than 90% of the entire portfolio, all of which will suffer worse than the medium-moving item used in this article.

Thankfully, we do have a metric available that allows us to gauge how bad our estimate of the uncertainty is. The Total Percentile Error or TPE tells us how bad the?center forecast value is, but also how bad the tails are. For service level impact we care about the upper tail only. Whenever items have expiration or obsolescence risk we also care about the lower tail, whilst for purchasing, production, and capacity planning we care most about the center forecast. If the TPE is low, chances are the upper tail error which is part of the TPE?is also low. This in turn would be an indicator that we are properly targeting service levels.

?Find all my articles by category here. Also listing outstanding articles by other authors.

*) Minions shown in headline photo are a trademark of Universal Studios. All other content is original by the author and can be freely shared and re-used.