Predictability Means Understanding Your System

In nearly every conversation I have about transforming ways of working, the word “predictability” comes up. People crave predictability to plan effectively and manage expectations. Without it, we are left with feelings of discomfort and uncertainty. How do we solve this problem?

Traditional Methods Fall Short

Traditional methods typically involve gathering experienced individuals to estimate task durations. However, these methods often overlook the variability and complexity of the work and the environment, leading to inaccurate forecasts.

Even modern agile practices that use things like using story points and velocity, often fail to provide reliable predictions. For instance, the lack of correlation between story points and delivery time highlights their limitations. Below is an image created by Thijs Morlion based on analysis of teams using story points for estimation. This image reflects what I have seen in my experience as well - note where smaller stories take longer than larger ones e.g. 1 point stories take longer than 5 point stories!

A Better Way - Advantages of Probabilistic Forecasting

Probabilistic forecasting, based on actual system performance data, offers a more effective solution. Here’s an example from a recent project:

Scenario: A project was 6 months overdue. The team estimated they could complete it by the end of April. As coach I used my experience and held the mirror up to the team, to challenge their thinking and approach.

Data Analysis:

• 78 work items left to deliver.
• Delivery rate: 9 items per week.
• Estimated time to complete: ~9 weeks, extending to mid-May.
• Failure demand (issues due to poor upstream quality): 20%, adding ~16 more items (totaling 94).

Monte Carlo Simulation:

• 10,000 simulations indicated only a 7% chance of completing by the end of April.
• To reach an 85% confidence level, the forecast extended to June 20th.

Discussion:

• (Me) Would you bet ￡1M on a 7% chance??(Leader) Of course not, that would be crazy!
• What odds would you feel comfortable with??They would need to be much higher, at least 75%!
• A 75% chance targets June 5th. How does that sound??It's a lot further out than we had hoped for.
• Yes it is, but that is the reality of your system. Would you rather communicate to your stakeholders something realistic, or something that is highly unlikely to happen??We need to build some trust back, so we need to deliver on this one!
• In that case, why don't we go with 85% chance on June 20th and if it comes in sooner even better??Yeah OK, that makes sense. We need to think about why it takes so long!
• We can look at that next. Avoiding the pressure of an unrealistic date hanging over us will help create an environment to improve things. Oh great, that's a good thing!

Why Probabilistic Forecasting Works

Probabilistic forecasting removes emotional biases and reliance on subjective opinions. It reflects reality through data, leading to more accurate and compelling forecasts.

While forecasts aren’t always precise, aiming to be "less wrong" in a complex, variable world saves time, money, and stress. Running a Monte Carlo analysis took 5 minutes, compared to multiple meetings with many people needed for traditional planning.

?

Conclusion

Probabilistic forecasting doesn’t guarantee you'll always be right, but it helps you be more accurate and realistic. This approach not only enhances predictability but also saves significant time and effort.

If you find this approach compelling, let's connect. I’ve spent years refining these methods, and I’m here to help you leverage my experience to improve predictability in your projects.

?

Next Steps

You’re likely seeking better outcomes and predictability. Follow me for upcoming articles on how to improve predictability in your workflows.