Reference Class Forecasting
Glen Alleman MSSM
Vietnam Veteran, Applying Systems Engineering Principles, Processes & Practices to Increase the Probability of Program Success for Complex Systems in Aerospace & Defense, Enterprise IT, and Process and Safety Industries
When it is conjectured,?You can't do estimates on software projects. Stop for a second and think about that concept.
Really, "you have no idea whatsoever about how long this will take and how much it will cost?"?I don't want to estimate because it is a bit different and a waste of time. That says you're either too lazy to do the estimating, I need to learn how to do the estimating, or you really want to get going and write software because that's what you do for a living.
Could you take a look at the picture below? Many contributors to the estimating conversation have mentioned?estimating bias?as a source of?bad estimates?(which they want to fix by doing NO estimates)?and Bent Flybjerg. This is how to do?Reference Class Forecasting?for?ANY?project. Now you may not need to do estimates because your project is of a size that you can see the end from the start, it has low enough risk that the variances in cost are not considered a problem, or short in duration so that you can control most of the variables quickly, or your customer doesn't really have that much at risk, so the investment in estimating no matter how small is not considered that useful.
But let's pretend these are not the case. There is some non-trivial amount of money at risk, the?planning horizon?is outside your ability to see the end, your customer has to go ask someone for that money (the CIO, the BOD), or the business actually will depend on staying on or near budget and have your features show up on or near the planned date.?
Here's how to start the process of making a statistically adjusted estimate of time and budget.
Reference Class Forecasting Example
Reference Class Forecasting is a method of predicting the future by looking at similar past situations and their outcomes. This approach to estimating was developed by Daniel Kahneman and Amos Tversky, which earned Kahneman with the Nobel Prize in Economics.
Our long-time friends from California have moved to our neighborhood here in Colorado. Their moving van arrived today.
We brought coffee to them while their old house was being unloaded into the new house. Talking with the moving van owner, he started telling stories about?estimating?the load in pounds. The?agent?makes the first estimate of the weight of the load and issues a quote for the cost of the move. Then the van owner picks up the load and weighs the truck before getting on the road. He told us that the range of precision and accuracy is all over the place, depending on the?agent. Sometimes it's very close. Sometimes it's not.
领英推荐
The estimate's quality depends on the estimator's skill and experience. The?reference class?estimating process is part of that skill and experience.
Suppose we switch to software development. I re-listened to?the Agile for Humans?podcast with Steve McConnell about estimating. Steve's discussion was focused on how to make good estimates. How to put these to work to make business and technical decisions. These themes are based on how to book?Software Estimation: Demystifying the Black Art. This book should?be on the shelf of any credible developer.
After listening again, it was clear those providing a forum for the No Estimates advocates still need to address the fatal flaw of the No Estimates discussion.
There is?NO?principle by which you can decide in the presence of uncertainty without estimating.?
The podcast provided a nice overview of why we should estimate, how to estimate, and what to do with those estimates. We even discussed how to deal with the dysfunctional aspects of management when making estimates. But in the end, we need estimates to provide value in exchange for money credibly.
There can be no consideration for NOT estimating, except on de minimis projects. Just like the moving van owner, he needs an estimate for the weight of the load. From there, he can confirm that the load will fit on the truck. The trailer has a load limit of 33,000 pounds. The mechanics of household goods have a reference class parameter for the size of 33,000 pounds. This comes from empirical?data of moving household goods. Some houses have heavier goods, some have lighter goods. But the estimator?knows how to adjust for that.
It seems those in the software development business who conjecture that estimates are not needed, is the?smell of dysfunction, is evil, and should be stopped, have failed to understand not only the basic need for the estimate when making a decision in the presence of uncertainty but also the basic principles of estimating in the presence of the uncertainties that always exist on projects.
When the idea of No Estimates?is given equal weight to Steve's message, it willfully ignores the decision-making principle. And replacing?that principle?with the practices and processes that attempt to decide without estimates.
This is like the moving van owner saying?I get wild, wrong, and estimates that cause problems with my trailer, tractor, load management, and fee payments for carrying cargo across state lines. So I have an idea. I'll not ask for or use any estimate; I'll start emptying the house in Seal Beach, California, and start planning to move the house to Colorado. What could possibly be wrong with that?
Resources
Java Developer
1 年The only thing that moves faster than the speed of light in this universe is the breathtaking speed that a stakeholder can translate a ballpark estimate into a "do or die", "unyielding line in the sand", commitment. I've seen this time-and-again and it is the number one phenomenon "not laziness" that causes developers to only reluctantly give estimates. And if you feel defeated before you even start - you aren't going to spend much time on the estimate there is no point.
RIBA Client Adviser and architect
1 年It’s a great tool for framing the terrain of the possible costs or measured quantities from the exogenous measure, however qualities will need other measures for the terrain definition.