Are the academics arguing about the wrong thing?
A bit of a bun-fight is starting between two professors of project management, which I came across in this article on the TransportXtra website.
Australia-based researchers Peter Love & Dominic Ahiaga-Dagbui, have taken issue with the work of UK-based professor Bent Flyvbjerg. Flyvbjerg is one of the most well-known academics in the field of capex projects, and particularly public-sector infrastructure and "mega-projects". The argument is about his well-publicised claims that (i), 90% of infrastructure projects overspend, and (ii) that the main cause of overspend is "under-estimating" (in turn due to optimism bias or deliberate exaggeration).
I was aware of Flyvbjerg's work, but hadn't realised how much he had influenced many governments and their governance processes. Many have institutionalised his recommendations about estimating project costs using "reference class forecasting".
The disagreement has been fuelled by a recent paper by Love & Ahiaga-Dagbui, which criticises Flyvbjerg's work. Love & Ahiaga-Dagbui certainly don’t hold back - as you can see from the title of their paper...
Although the paper contains the things we expect of an academic paper - like words I had to look up, small text, and a long list of references, it also reads like a sensational tabloid newspaper, talking about "fake news", and making claims of flawed methods, and the deliberate misrepresenting of data.
Whilst interesting to watch from a distance, it set me thinking about the relevance of their research to those of us actually managing projects?
What struck me in reading various papers this weekend by both Love and Flyvbjerg, was that they both talk about "accurate estimates". They imply there is something bad about not being able to accurately estimate the 'right cost' for a project in advance.
Whilst I understand the need for cost and time estimates, and there is a problem because most projects seem to spend more, and take longer, than estimated, I don’t think the "poor estimate" should be the main focus of attention.
The goal of a project should be to achieve its objectives as quickly and cost-effectively as possible. Meaning execution and performance are much more important. If a project costs 25% more than estimated, is in my view of secondary importance to its absolute cost and duration. Are we building things faster and cheaper than we did a decade ago? And if not, do we know why? If you put any importance on hitting the estimate, you also need to know whether the estimate was virtually impossible, achievable but challenging, or very comfortable?
Statistically, any single-figure estimate is likely to be wrong, because in reality an estimate is a probability distribution, not a single figure. And no amount of big data analytics can predict the future, or tell you where on this distribution your specific project "should" lie. If most of your projects come in on-time and on-budget, you should really understand how they do so. Is it because they have targets than even a chimpanzee could achieve, or are doing something specific that delivers great performance? And if it is the latter, do you make sure everyone does the same thing?
Projects should be managed with the mindset of the world-class athlete, which is about being the best I can be.
Do you think top marathon runners care about the statistical distribution of finishing times in past races (which is how Flyvbjerg suggests projects estimate). Whilst it might be factually correct that 80% of runners in the London Marathon finished in less than 5h45m, do you think the better runners care? Would they set this as their "reasonable estimate". I don't think so - they would want something like 3h30m or less, to be in the top 10%. Or if they were in the top 10% last time, maybe this time they are aiming to be in the top 5%.
Clients and governments should be focusing on how to de deliver more projects for our money, and how do we complete them faster. I don’t think Amazon wastes as much time as a project does trying to guess to within 5% the number of books it will sell in 2019. I bet it just sets up a system to meet whatever the demand is at the time.
The next time the risk analyst says the P80 cost is ￡250M, ask them what the P20 cost is, and consider using this as your stretch target...
And projects could learn from this.
I will be writing a short series of articles on the key principles of fast project execution over the coming weeks. The aim will be to share with you some of the emerging research into exactly what helps projects to speed up. And speed is the first step in reducing costs, improving profits, reducing stress, and increasing productivity.
They will be simple ideas that you can implement on your projects, irrespective of the formal methodology you use. How to put together a good-enough plan, and the disciplines of great execution.
I look forward to sharing them with you, and hearing what you think.
PS - I am not saying estimates have no role, just that they should be a second-order importance. Maximising and continuously improving delivery performance is much more important.
Fast, low-cost performance is more important than hitting your estimate.
A good-enough plan with great execution, beats a brilliant plan, poorly executed.
If you agree, and want to know how to do this, look out for my next posts.
The Author:
Ian Heptinstall is a consultant and coach who helps project teams to plan and deliver faster and lower cost projects. He is co-author with Robert Bolton of The Exacutive Guide to Breakthrough Project Management.
www.ianheptinstall.com | www.BreakthroughProjectManagement.com
Independent Oil & Energy Professional
4 年Quite a lot of comments. Why didn't the researchers clarify their points?
Senior supplier Development Manager - Roads Academy
6 年Well-articulated Ian. "We tend to seek easy, single-factor explanations of success. For most important things, however, success requires avoiding the many separate causes of failure” – Jared Diamond
Energy and Mineral Processing Industry Consultant
6 年Ian, I thought of your post this morning as Canada's Auditor General commented on my local news channel about the miserable execution of the Government of Canada's Phoenix pay system project. While the Auditor General spoke of a general project management and project governance failure, he highlighted one specific decision that was made to stay within budget - a curtailing of the functionality of the pay system that caused no end of problems and compromised the project outcome. I don't think the preoccupation with budget and schedule is a solely the prerogative of academics, rather it is a fundamental flaw in project management where project managers and even project governance bodies are incentivized based upon the near term metrics of budget and schedule and favourable project outcomes are a 'distant' second. In terms of proximity, budget and schedule are here and now. Project decisions are often made to trade risk to maintain budget and schedule. Risk is less proximate and less tangible - assumed risks become someone else's problem down the road, and are difficult to measure and monitor. Assumed risks pile up until the project outcomes are compromised. What's measured gets managed, and we are measuring the wrong things!
Associate Director, Broadleaf
6 年Some earlier comments hint at the need to think more widely than cost alone and with more subtlety than a set and forget cost target. That is, to acknowledge that there may be good reason to do something that will change the final cost. The obsession with a single estimate and contingency is inconsistent with the complexity of real life and implies that cost control is the be all and end all. I know no one really believes that but it’s the way the discussion is generally framed.
Owner, Validation Estimating LLC
6 年Ian, I suggest you re-read the Flyvbjerg/ Love -Ahiaga Dagbui papers. They are not at heart about the accuracy, but the "cause" of overruns.? Flyvbjerg contends great execution (such as you advocate) is futile - just accept the mediocre practice and results of the past, and make that your goal and learn to love it (or better yet, do not run the race-do not build mega). Love-Ahiaga Dagbui on the other hand contend the cause of overruns is measurable and the cause is lousy execution and practices. We need to measure the practices, improve as needed, and plan and estimate accordingly. It agrees with you. And, oh, but the way, if you improve execution and practices, you will be more predictable (accurate) and win races. How lovely, yes?