Superforcasting for Project Managers
"Superforcasting: The art and science of prediction" is a great book that uses real research to dive deep on what forecasting is, what differentiates a good forecaster from a bad one, and how we can improve at it. Even though we can't predict the future, who wouldn't benefit from improving your chances to forecast simple or complex events from sport matches, stocks prices, tender results or project costs?
In the book, the authors explain how they utilised groups of volunteers to forecast complex events over a specific timeframe such as "Will Russia attack Ukraine in the next 3 months?" or "Will gold reach $2,000/oz. before July 31st?" and compare them to experts in those fields and other organisations. The results show that many are not better at it than simple chance. However, there is a group of outstanding forecasters that consistently beat the rest by great margin on every field. These are then nicknamed the Superforecasters.
I enjoyed the book and believe there is a set of key elements that can be applied by anyone in life and of course in Project Management. Without further ado, these are my key takeaways:
High Level Comparisons
This is probably the most interesting takeaway for me. In the book, the authors point out that the Superforcasters tend to look at the bigger picture first before diving into details and adjusting their forecast accordingly.
For example, let's assume someone asked you what are the chances that a coworker who has a 2.5 year old girl and a 1 year old boy, lives in Parramatta and is an accountant, owns a pool? Where would you start?
Many people would start by thinking the kids are too young to enjoy a pool, or that Sydney is too expensive to own pools, or that accountants love to swim (whatever!). Instead, Superforcasters would start by investigating how many people own pools in Sydney, which is around 15% according to a study performed in 2019. Only then they will proceed to analyse the details and to modify the initial forecast of 15%.
Would it be the same if we start with the details and then change the forecast as we move outwards? Apparently no! According to the research presented in the book, once the initial assessment is done, we develop an anchoring bias towards it, preventing us from making a substantial change to it.
Be aware of averages:
Nothing exemplifies this better than the story about Bill Gates walking into a bar, raising the average wealth of the group of people by several millions. Are they actually better off now that Gates is in the bar? No (unless he pays for drinks). Learn the difference between mean (average) and median, and to recognise when to use which.
Break bigger questions into smaller questions
To explain this point, the authors used the famous question by Enrico Fermi to his students. Estimate how many piano tuners are in Chicago? Without using of google or any other research tool. To answer this sort of question, Fermi recommends to break it into smaller questions:
If you don't know the answer to those questions, then continue breaking them into even smaller questions such as how many people live in Chicago? or How does Chicago's population would compare to a city I do know the population of?
领英推荐
By answering the smaller questions, Superforecasters achieve a more accurate response to the main question than just a mere guess.
Any estimator can relate to this strategy as they use the same principle to divide a project into activities that can be priced accurately until all activities are priced and the estimate can be finalised.
Forecasting is not about certainty, but probability
Similar to the point raised by Anne Duke on her book "Thinking of bets" which I reviewed previously, the Superforcasters focus on probabilistic outcomes.
The media has recently stated that the election polls don't work anymore, because in recent elections (Trump vs Clinton or Shorten vs Morris) they have been "Wrong", but have they? Election polls describe their results in the same probabilistic fashion than the weather report. If your weather app indicates a chance of 20% or even 10% of rain and it does rain, was the forecast wrong? 10% chance still has a likelihood of that outcome that can't be dismissed.
To actually assess whether a forecast process is correct, we need to gather data from the app regarding every time it predicted 10% chance of rain, and the data should show that it only rained in approximate 10% of those days.
Stick to a short to medium timeframe:
Is it worth trying to accurately forecast an event in the next 5 to10 years? Research indicates that even superforcasters aren't better at estimating events in long timeframes than the chimps throwing darts analogy.
Key Superforecaster attributes
The authors of the book explore different theories to evaluate why the superforcasters are so good at what they do when compared to other volunteer forecasters and even when compared to field experts such as intelligence officers and geopolitical experts. Fortunately for all us, the outcome of their research show that there isn't a superforecasting gene, IQ or degree necessary to achieve this status. Instead all it takes is to develop the following aspects:
Summary
Reading this book on itself won't make you a superforecaster, nor will it teach you how to never be wrong. Instead, it will shed light on what attributes are necessary to master if we want to improve our forecasting capabilities, as well as sharing many eye opening stories and facts.
Hope you enjoyed this article and feel free wo leave me a comment on the post
Art