Monte Carlo Simulation for Project Cost and Schedule Risk Management

Estimation is part of project management.?

The project manager's most important estimates are Time, Cost, and Technical Performance.?

And remember, you're High School Economics class - The Value of something cannot be determined alone. You need to know the Cost to acquire that Value and the Time that It will be available for use.

Showing up late and over budget diminishes the value produced from the project.

And unlike the advice of Agile thought leaders, showing up on time, over budget, with missing capabilities diminishes value as well as stated in Misunderstanding Making Decision in the Presence of Uncertainty.

Since it is easier to estimate small tasks, these estimates are often calculated and performed as?point estimates. For example,?a task will take 3 days. Or as an estimate with two-point ranges. A task will take between 2 and 5 days.

When there are several tasks, each with possible dependencies on other tasks there is a problem. You can't add up those durations' duration and ranges. Since each task may or may not be dependent on other tasks, and when one task?is being modeled on the?low?end of its range, another task may be being modeled on its upper range of possible values.

When we hear the term,?Monte Carlo,?we think of the gambling center in Monaco.?The scientific study of probability concerns itself with the occurrence of random events and the characterization of those random happenings. Gambling casinos rely on probability to ensure that they are profitable over the long run.

For this to happen, the odds or chance of the casino winning has to be in its favor. This is where probability comes into play because the theory of probability provides a mathematical way to set the rules for each one of its games to make sure the odds are in their favor. As a simulation technique, Monte Carlo simulation relies on probability. [10] [18]

Monte Carlo simulation, also known as the Monte Carlo method, originated in the 1940s at Los Alamos National Laboratory. Physicists Stanislaw

Ulman, Enrico Fermi, John von Neumann, and Nicholas Metropolis had to perform repeated simulations of their atomic physics models to understand

how they would behave given many uncertain input variable values. As random samples of the input variables were chosen for each simulation run, a statistical description of the model output emerged that provided evidence of how the real-world system would behave. That real-world system was the first atomic bomb. [13]

The repeated random sampling model of the input variables over many simulation runs defines Monte Carlo simulation. The result is an artificial world (model) that is meant to resemble the real world in all relevant aspects closely. [8]

So before we proceed, let's look at a definition of Monter Carlo Simulation in the project domain so we don't have to decipher someone else's definition designed to obscure the actual definition.

A Monte Carlo Simulation is “a problem-solving technique used to approximate the probability of certain outcomes by running multiple trial runs, called simulations, using random variables.” ... It then simulates the completion of remaining work and produces a histogram showing the distribution of possible delivery dates.

The Monte Carlo Method?

Monte Carlo Simulation has four steps, no matter the domain or the problem:

1. Define the distribution of possible inputs for each input random variable.
2. Generate the inputs randomly for those distributions.
3. Perform the deterministic computation using that set of inputs.
4. Aggregate the results in the individual computation into a final result.

This is the simple but powerful process of?Monte Carlo Simulation?that is universally applicable from development projects to nuclear physics (my original domain) to molecular plant biological processes (our son's domain), to financial planning (our broker's domain), to the modeling of human behaviors of special needs clients (our daughter's domain).

This approach allows you to?model?systems in the future using past data OR using models of what the future might be if the system hasn't been done before.

The notion that Monte Carlo Simulation cannot be applied to a single project is simply wrong.

Let's go ahead and look further.

What Is Monte Carlo Simulation of Projects

Monte Carlo Simulation started with?Buffon's Needle?Problem,?which says...

Let a needle of length?L?be thrown at random?onto a horizontal plane ruled with parallel straight lines spaced by a distance?d?from each other, with?d > L. What is the probability?p?that the needle will intersect one of those lines?

Monte Carlo simulation on projects examines all paths through the network of activities or all possible states of the project for the duration, Cost, and risk that create impacts on duration and Cost.

It provides an accurate estimate (within the confidence?intervals) for the overall duration of the project schedule for that work and the impact of risk on that Cost and schedule.

As well it provides?a sensitivity analysis for all the interacting?tasks

Let's look at a notional project, where the tasks are interconnected and dependent - predecessor and successor relationships with each other like the project below.

Each work activity in a discrete model will have an estimated duration - a scalar number, usually measured in days. Since all projects operate in the presence of uncertainty, this?deterministic?duration is likely to have little credibility in actual use.?For traditional projects, a Monte Carlo Simulation creates a list of durations from the?Probability Distribution?of a specific duration for a specific task.?

This probability distribution can be built from?past data?for similar work, like the PDFs shown above, which have different shapes depending on the type of work. Or it can be pre-defined for a shape and an upper and lower range for that shape. In the most straightforward approach, when we know little about the past performance of the work, a?Triangle?distribution provides confidence that isn't too optimistic on the lower bounds and too pessimistic on the upper bounds. Using this past data is called?Reference Class Forecasting?[14].

When there is no past data available, a second approach can be used. This involves?ranking?to ranges of the?most likely?value for the variable. Here's an example of a spacecraft with?ranking ranges?around the?most likely?using a?Triangle?distribution.

In this case, the Business Usually ranges is -5% for a low and +10% for a high around the?most likely?value for the duration, Cost, or some technical performance parameter. The Business, as usual, but with some technical processes, gets a bit wider. Flight software is always an issue where we work, so those ranges are comprehensive even more. Putting all the components together into a working system is fraught with uncertainties, so a wider range is used. Getting the software qualified?is about the same variability?as getting it certified, so the same range is used. But the big problem comes with the spacecraft going to the Thermal Vacuum chamber. That is modeled as -5% to +175%.

These assignments are made initially during the proposal, then updated monthly for the?reality?of the project's performance. The proposals I work on usually require an 80% confidence of an?on or before?schedule and an?at or below?for Cost. Monte Carlo Simulation tools are the heart of this work.

This is a Closed Loop Control Systems for managing the performance of a Software Intensive System of Systems, all developed using Scrum.

The triangular distribution can be used when we do not know the distribution. Still, we have some idea of the minimum value for the variable, the maximum value for the variable, and what you think the most likely value is. The Triangle distribution is an excellent place to start since it models the?log-normal?distribution, which is found for many naturally occurring processes. The uncertainties for the work effort on projects is a naturally?occurring process derived from the Aleatory uncertainty processes of humans working on technical processes.

When you run the Monte Carlo Simulation tool (Risky Project in this case), you get a chart that looks like this. This chart is the result of the MCS for a complete project, that is, for the?end of the project. A similar diagram can be produced for any specific task in the project with the same results.

The chart shows the probability distribution of the completion dates for the project when the durations for all the work on the project are defined as?the most likely?value, the upper and lower limits of those durations, and the PDF for the curve that the Monte Carlo tool is going to produce samples from - this is usually a Triangle curve for me since it gives us?credible?values with the least amount of work.

Let's Look At Some Myths of Monte Carlo Simulation

Here are some common myths, misunderstandings, and willful ignorance of using probability and statistics, which is the basis of Monet Carlo Simulation, when making informed decisions on managing a software project.

Statistics don't apply to single events

In the presence of uncertainty, a single event in the future - like the delivery date or the Cost to deliver that outcome, Monte Carlo Simulation is?THE?tool to develop a?confidence?and?accuracy?model of that future event. All that is needed is to know...

• The?Most Likely?value that event could take on. If you have No idea what that most likely value might be, there are several ways to come up with that answer, starting with?wideband Delphi.?Here is an example of?How To Estimate Almost Any Software Deliverable in 90 Seconds.?
• Then?use a simple?Monte Carlo?tool you can find on the web for Excel.?RiskAmp?is one I like.
• A fully integrated project risk management tool is Intaver's Risky Project
• Then you'll be able to show the?probability of occurrence?for the variable's range of possible values and be able to debunk that statement since Statistics DOES apply to a single event when you can answer the question.

What's the probability of completing this work?on or before?the due date, given that I know something about what the work entails, what the dependencies of the work are, and what ranges of the random variables drive the work?

Suppose you need to know the answers to these broad statistical questions. In that case, your project is doomed before you start, assuming you actually have a due date, a not to exceed budget, and a minimum set of capabilities for that time or budget. If you still need those, we'd call that a?de minimis?project - meaning no one cares when you show up, how much it costs, or what you'll deliver. Nice work if you can get it.

Here's an example from an actual project - the Wright Brothers Army contract for the Wright Flyer.

Here's their schedule (this is the MSFT Project version, but we had access to the Smithsonian archives and recreated this schedule from their notebooks).

They estimated the reducible and irreducible uncertainties from the notebooks to be assured they could meet the contractual dates.