Aleatory and Epistemic Uncertainty in Development Projects

All Projects Operate in the Presence of Uncertainty.

This Uncertainty?is unavoidable. One might argue that Uncertainty is desirable in the Agile paradigm. Otherwise, how can?Emergence?add Value to the deliverables?

Risk and Uncertainty are integral to all decision-making processes for non-trivial work. The process involves a set of actions and outcomes, each associated with a probability or a statistical process.

All risk comes from uncertainty.

In this Uncertainty, the design, Development, and deployment of products or services must rely on estimations, forecasts, and predictions based on an?idealized?understanding of an?unknown?(but knowable)?future desired outcome. Guiding the work efforts toward a future is based on a?product roadmap, regardless of the development method—Agile or Traditional. Without such a?roadmap,?the developers and managers of the Development are just wandering around looking for a solution to an ill-defined problem.

If this future reality is?unknowable,?you have a bigger problem and are headed for failure. Emergence plays a role in all development processes, but Emergence without a goal is called?Research,?not?Development. Development?has specific business goals for?breakeven,?ROI, IRR, Cash Flow,?and other financial performance measures needed to run the business successfully.

So, let's focus on the impacts of Uncertainty in the development paradigm and leave the research alone.

There are two broad types of Uncertainty on all projects, and these two types drive very different responses.

• There is Uncertainty?associated with the natural randomness of the underlying processes of developing products or services.
• There is Uncertainty associated with the?model?of how the real world operates because of insufficient or imperfect knowledge of reality.

These two types have fancy names.

• Aleatory Uncertainty is objective,?stochastic,?or?irreducible.
• Epistemic Uncertainty is subjective, state-of-knowledge, or reducible.

The two types of Uncertainty may be combined and analyzed as a total uncertainty or treated separately. In either case, the principles of probability and statistics apply equally.

Aleatory Uncertainty

Aleatory Uncertainty is created due to?the intrinsic?random nature?of the phenomena occurring during the project's Development or operation, such as work duration, cost variances for labor and materials, and natural variances in the performance of products or services.

This means there is an?Inherent Randomness.?

This data-based Uncertainty is?associated with the inherent?variability?of the basic information?of the real-world development processes. A Stochastic process creates this data. Weather is modeled as an Aleatory uncertainty.

In practice, we can assign a?mean?or a?median?value to this Uncertainty. That's what the weather forecast does. That 40% chance of rain is usually a?mean?value. Where we live, when we hear a 40% chance in Boulder County, we know we have a lower probability?because of our micro-climate. That weather forecast is?over the forecast area?and may vary depending on where you live.

This forecast also includes inaccuracies and imprecisions in the?prescribed forms of the statistical distributions?and all the parameters of the estimates. This is why?forecasting the weather in some parts of the world is a very?sporty?business. In places like Los Angeles, it's easy - as shown in the movie?LA Stories, where Steve Martin is the bored weatherman. With our mountain weather here in Colorado, making a forecast a few days from now is likely a challenge. As they say,?don't like Colorado weather? Wait a few hours, and it'll change.

Any intervention cannot reduce these uncertainties and are just part of the development process. They are?irreducible,?and the only approach to dealing with them is to have?a margin. Schedule margin, cost margin, and performance margin. For weather, the margin is to bring an umbrella when the forecast percent is above some value for your area.

Data-based means that the randomness is in the data generated by statistical processes. For example, the duration of work activity is a statistical process. That duration can take on many values depending?on the underlying?model of the work. We can have a narrow range of values for the duration or a wide range of values for the underlying processes.

Many project phenomena or processes of concern to developers contain randomness. The expected outcomes are unpredictable (to some degree). Such phenomena can be characterized by field or experimental data containing significant variability representing the natural randomness of an underlying phenomenon. The measurements differ from one experiment (or observation) to another, even if conducted or measured under identical conditions.

These experimental results include a range of measured or observed values; within this range, specific values may occur more frequently than others. The variability inherent in this data or information is statistical, and realizing a particular value (or range of values) involves probability.

Epistemic Uncertainty

The term epistêmê in Greek means?knowledge.

Epistemic?Uncertainty reflects our lack of knowledge.

This?lack of knowledge?is a?probabilistic?assessment of some outcome, usually an?event-based?effect.

We assign probabilities to events, probabilities?to the work activities that create the knowledge?needed to assess the Uncertainty, and probabilities of the residual uncertainties after our new knowledge has been acquired.

Some Challenges to Managing in the Presence of Uncertainty

The primary issue with all uncertainties is communicating the accuracy and precision of the risk created by aleatory?and?epistemic?Uncertainty.?

• What is the scope of the Uncertainty?
• What risks does it create to the success of the product or service development effort?
• Is the Uncertainty time-dependent?
• At what level of decomposition of the project is the Uncertainty applicable?
• How do these uncertainties, which create risk, propagate in the project and create other risks due to this interaction?

This is a?Risk Communication?issue. So, let's restate the two forms of Uncertainty.

• Aleatory Uncertainty is the inherent Uncertainty in a nondeterministic (stochastic, random) phenomenon…, which is reflected by modeling the phenomenon in a probabilistic mode. The accumulation of more data or additional information cannot reduce aleatory Uncertainty.
• Epistemic Uncertainty: The Uncertainty attributable to the incomplete knowledge about a phenomenon that affects our ability to model it is reflected in ranges of values for parameters, a range of viable models, the level of model detail, multiple expert interpretations, and statistical confidence. The accumulation of additional information can reduce Uncertainty.

What Does This Mean for Development Projects Working in the Presence of Uncertainty?

If you accept that all project work operates in the presence of Aleatory and Epistemic Uncertainty, then ...

No decisions can be made in the presence of these two types of uncertanties without estimating the impact of your decision on the project.

This is a simple, clear, concise principle of managing in the presence of Uncertainty. Anyone suggesting that decisions can be made without estimating has to ignore this principle willfully, OR the project is de minimus - meaning it's of no consequence to those paying if the project is late, over budget, or the delivered outcomes don't meet the needed performance level for the project to earn its?Value?in exchange for the?Cost?to produce that?Value.

With all risk coming from Uncertainty, both reducible (Epistemic) and Irreducible (Aleatory), here is a Compendium of Resources used to assess risk, their impacts, and handling strategies.

And a collection of briefings and papers on putting these principles to work

• What is Risk??- Unrealistic technical performance expectations create cost and schedule growth, unrealistic cost and schedule estimates, inadequate risk assessments, unanticipated technical issues, and poorly performed and ineffective risk management, all contributing to program technical and programmatic shortfalls.
• Building a Risk Tolerant PMB (Performance Measurement Baseline)?- Using Earned Value to manage the risk of a project terms terms "risk-adjusted physical performance."
• Handling Risk on High-Tech Programs: Without metrics, you're just another guy with an opinion—Stephan Leschka, Hewlett Packard.
• Managing Risk with Deliverables-Based Planning—a four-hour workshop following the guidance of Stephan Leschka of Hewlett Packard, who says, "Without metrics, you're just another guy with an opinion."
• Risk Management Processes?- Defense Threat Reduction Agency, March 27, 2012.
• Risk Management Guidance—Risk management is a critical success factor of any project or program. This document collects risk management categories that ask the question, "Did you think about this risk and its impact on our probability of success?"
• Uncertainty and Resulting Risk on Project Management—It's popular to discuss risk in straightforward approaches to managing risk, like a step-by-step approach to making a list of the risks, assessing the probability of occurrence, and assessing the impact if the risk were to occur.
• Comments on "Managing Projects Involving External Threats"
• Risk Assessment Template for Software Development or Acquisition Projects
• Managing Risk and Opportunity?- 1st in a Series.
• Managing Schedule and Cost Risk?- 2nd in a Series.
• The Risk Tolerant Master Schedule?- 3rd in a Series.
• Risk Management and Agile Software Development?- handling aleatory and epistemic uncertainties.
• Risk Management?- in?Gower Handbook of Project Performance for Agile, Waterfall and Everything in Between, 1st Edition 2017.
• Building a Risk Tolerant Schedule - Technical and Programmatic disruption in project plans don't need to impact Cost, performance, or schedule metrics negatively
• 5 Proven Approaches for Mitigating Project Failure?- Principles are needed before Practices and Processes can be successfully applied.
• Risk Management is How Adults Manage Projects
• Risk Management in Five Easy Pieces with Apologies to Jack?- Managing Cost, Schedule & Technical Performance Risk is the Basis of Good Project Management.
• Five Easy Pieces: The Essential of Managing Programmatic Risk, 10th Annual Rocky Mountain Project Management Symposium, Colorado Springs, Colorado, May 2008.
• Applying Risk Radar to High-Risk Technology Projects: Everyone involved in Development, acquisition, or management talks about?risk.
• Practices of Managing in the Presence of Uncertainties That Create Cost, Schedule, and Technical Technical Risk,?Risk Awareness Week, 11-15 October 2021.
• Programmatic Risk Management: A "Not So Simple" Introduction to the Complex but Critical Process of Building a Credible Schedule
• Assessing Enterprise Project Risk - Probability of Program Success P(s)
• 5 Proven Approaches for Mitigating Project Failure, American Management Association, April 22, 2015.
• Increasing the Probability of Program Success with Continuous Risk Management, Glen Alleman, Thomas J. Coonce, and Rick A. Price, Joint Space Cost Council.
• Increasing the Probability of Program Success with Continuous Risk Management, Glen Alleman, Thomas J. Coonce, and Rick A. Price,?The Measurable News, 2018.04
• Managing Cost, Schedule, and Technical Performance Risk at Y-12?- Cost, Schedule, and Technical Performance Risk Management seamlessly integrates all three.
• Five Easy Pieces of Risk Management, 10th Annual Rocky Mountain Project Management Symposium, Denver, Colorado, April 11, 2008.
• Managing in the Presence of Uncertainty, PMI Mile High Chapter, April 12, 2019.
• Increasing the Probability of Program Success Using Risk+?- a workshop on the principles and practices of Risk+ and increasing the Probability of Program Sucess
• 5 Proven Approaches for Mitigating Project Failure, American Management Association, April 22, 2015.