The Core Problem with Project Success

There are many core Root Causes of program problems.

Here are 4 sources of project performance failures from research in our domain of Software Intensive System of Systems.

Before diving into the details of these, let me address another issue that has come up around project success and estimates. A common chart used to show the poor performance of projects compares?Ideal?project performance with Actual?project performance. Here's the notional replica of that chart.

This chart shows several things.

• The?notion of?Ideal?is just that - notional. All that line says is that this was the baseline Estimate at Completion of the project work. It says nothing about the credibility of that estimate or the possibility?that one or all of the Root Causes above are in play.
• Then the chart shows that many projects cost more or take longer (costing more) in the sample population of projects.?
• The term?Ideal?is a misnomer. There is no ideal in the estimating business—just an estimate.
• The estimate has two primary attributes - accuracy and precision.
• The chart (even the notional charts) usually doesn't say what the accuracy or precision is of the value that makes up the line.

So let's look at the estimating process and the actual project performance.?

• There is no such thing as the?ideal?cost estimate. Estimates are probabilistic. They have a probability distribution function (PDF) around the Mode of the possible values from the estimate. This Mode is the Most Likely value of the estimate. If the PDF is symmetric (as shown above) the upper and lower limits are usually done in some 20/80 bounds. This is typical in or domain. Other domains may vary.
• This says?here's our estimate with an 80% confidence.?
• So now, if the actual cost, schedule, or technical parameter falls inside the?acceptable range?(the?confidence interval), it's considered GREEN. This range of variances addresses the uncertainty in the estimate and the project performance.

But here are three?problems.

1. There is no cause stated for that variance.
2. The?ideal?line can never be?ideal. The straight line is for the estimate of the cost (and schedule), and that estimate is probabilistic. So the line HAS to have a probability distribution around it. The confidence interval on the range of the estimate. The resulting actual cost or schedule may well be within an acceptable range of the estimate.
3. Are the estimates being updated when work is performed or new work is discovered, and are those updates resulting from changing scope? You can't state?that we made our estimate if the scope changes. This is the core?Performance?Measurement?Baseline?stuff we use every week where we work.

As well, since?the ideal?line has no probabilistic attributes in the original paper(s), not shown above - Here's how we think about cost, schedule, and technical performance modeling in the presence of the probabilistic and statistical processes of all project work. ?

So let's be clear.

NO point estimate can be credible.

The?Ideal?line is a point estimate. It's bogus on day one and continues to mislead as more data is captured from projects?claimed?not to match the original estimate. The ideal estimates are worthless without the underlying uncertainties (aleatory and epistemic) in the estimating model. So when the actual numbers come in and don't match the?ideal estimate,?there is NO way to know why.?

Was the estimate wrong (and all point estimates are wrong) or was one or all of Mr. Bliss's root causes the cause of the actual variance?

Another issue with the?Ideal Line?is that there are no confidence intervals around the line. What if?the?actual?cost came?inside?the acceptable range of the?ideal?cost? Then would the project be considered?on cost?and?on schedule? Add to that the coupling between cost, schedule, and technical performance, as shown above.?(This will be a coming post on how to model this with a Design Structure Matrix)

The use of the Ideal is Notional.

That's fine if your project is Notional.

Why does a project or a collection of projects not match the baselined estimate?

That estimate MUST have an accuracy and precision number before being useful to anyone.?

• Essentially that straight line is likely an unquantified?point estimate.?And?ALL?point estimates are WRONG, BOGUS, WORTHLESS. (Yes, I am shouting on the internet).
• Don't ever make decisions in the presence of uncertainty with point estimates.
• Don't ever do an analysis of cost and schedule variances without first understanding the accuracy and precision of the original estimate.
• Don't make suggestions to make changes to the processes without first finding the root cause of why the actual performance is a variance from the planned performance.

?So what's the summary so far:

• All project work is probabilistic, driven by the underlying uncertainty of many processes. These processes are coupled - they have to be for any non-trivial projects. What are the coupling factors? The non-linear couplings? Don't know these, no way to suggest much of anything about the time-phased cost and schedule.
• Knowing the reducible and irreducible uncertainties of the project is the?minimal critical success factor?for project success.
• Don't know these? You doomed the project on day one.

So in the end, any estimate we make in the beginning of the project MUST be updated as the project proceeds. With this?past performance?data, we can make improved estimates of future performance, as shown below. By the way, when the #NoEstimates advocates suggest using past data (empirical data) and don't apply the statistical assessment of that data to produce a confidence interval for the future estimate (a forecast is an estimate of a future outcome), they have only done half the work needed to inform those paying what is the likelihood of the future cost, schedule, or technical performance.

So Now To The Corrective Actions of The Causes of Project Variance

If we take the 4 root causes in the first chart - courtesy of?Mr. Gary Bliss, Director Performance Assessment and Root Cause Analysis (PARCA) let's see what the first approach is to fix these.

Unrealistic performance expectations missing Measures of Effectiveness and Measures of Performance

• Defining the Measures of Performance, the resulting Measures of Effectiveness, and the Technical Performance Measures of the resulting project outcomes is a critical success factor.
• Along with the Key Performance Parameters, these measures define what DONE looks like in units of measure meaningful to the decision-makers.
• Without these measures, those decision makers and those building the products that implement the solution have no way of knowing what DONE looks like.

Unrealistic Cost and Schedule estimates based on inadequate risk-adjusted growth models

• Here's where estimating comes in. All project work is subject to uncertainty. Reducible (Epistemic) uncertainty and Irreducible (Aleatory) uncertainty.?
• Here's how to?Manage in the Presence of Uncertainty.
• Both these cause risk to cost, schedule, and technical outcomes.
• Determining the range of possible values for aleatory and epistemic uncertainties means making estimates from past performance data or parametric models.

Inadequate assessment of risk and unmitigated exposure to these risks without proper handling plans

• This type of risk is held in the Risk Register.
• This means making estimates of the probability of occurrence, probability of impact, probability of the cost to mitigate, the?probability of any residual risk, and the probability of the impact of this residual risk.
• Risk management means making estimates.?
• Risk management is how adults manage projects. No risk management, no adult management. No estimating, no adult management.

Unanticipated Technical issues with no alternative plans and solutions to maintain effectiveness

• Things go wrong. It's called development.
• Where's Plan B? Maybe even Plan C when things go wrong.

When we hear we can't estimate, planning is hard or?maybe not even needed, or we can't forecast the future, let's ask some questions.

1. Do you know what DONE looks like in meaningful units of measure?
2. Do you have a plan to get Done when the customer needs you to, for the cost the customer can afford?
3. Do you have the needed resources to reach Done for the planned cost and schedule?
4. Do you know something about the risk of reaching Done, and do you have plans to mitigate those risks somehow?
5. Do you have some way to measure the physical percent complete toward Done, again in units meaningful to the decision-makers, so you can get feedback (variance) from your work to take corrective actions to keep the project going in the right direction?

The answers should be YES to these Five Immutable Principles of Project Success

If not, you're late, over budget, and have a low probability of success on Day One.

? NRO Cost Group Risk Process, Aerospace Corporation, 2003

? Space Systems Cost Risk Handbook Applying the Best Practices in Cost Risk Analysis to Space System Cost Estimates?