Aleatory and Epistemic Uncertainty in Software Development Projects

All software development projects operate in the presence of uncertainty.

This uncertainty?is unavoidable. One might actually make the case in the Agile paradigm that uncertainty is desirable. Otherwise, how can?Emergence add value to the deliverables?

In the presence of this uncertainty, the design, development, and deployment of software must rely on estimations, forecasts, and predictions based on an?idealized understanding of what is an?unknown?(but knowable)? future desired outcome. Guiding the work efforts toward a future is based on a Product Roadmap, no matter the development method - be it Agile or Traditional. Without such a?Roadmap,?the developers and managers of the development are just wandering around looking for a solution for an ill-defined problem.

If this future reality is?unknowable, you have a bigger problem and are headed for failure. Emergence plays a role in all development processes, but emergence without a goal is called?Research, not?Development. Development has specific business goals for?breakeven,?ROI, IRR, Cash Flow, and other financial performance measures needed to run the business successfully.

So, let's focus on the impacts of uncertainty in the development paradigm and leave the research alone for now.

There are two broad types of uncertainty on all projects, and on software projects, these two types drive very different responses.

• There is uncertainty?associated with the natural randomness of the underlying processes of writing software.
• Uncertainty is associated with the?model of the real world the software operates in because of insufficient or imperfect knowledge of reality.

These two types have fancy names.

• Aleatory uncertainty.
• Epistemic uncertainty.

The two types of uncertainty may be combined and analyzed as a total uncertainty or treated separately. In either case, the principles of probability and statistics apply equally.

The?Alea in?Aleatory is Latin for?Dice.

This means there is an?Inherent Randomness.?

This data-based uncertainty is?associated with the inherent?variability?of the basic information?of the real-world development processes. These uncertainties?cannot be reduced; they are just part of the development process. They are?irreducible, and the only approach to dealing with them is to have?margin. Schedule margin, cost margin, and performance margin.

Data-Based means that the randomness is in the data generated by statistical processes. For example, the duration of a work activity is a statistical process. That duration can take on many values depending?on the underlying?model of the work. We can have a narrow range of values for the duration. Or a wide range of values, depending on the underlying processes.

Many software project phenomena or processes of concern to developers contain randomness. The expected outcomes are unpredictable (to some degree). Such phenomena can be characterized by field or experimental data containing significant variability representing the natural randomness of an underlying phenomenon. The observed measurements are different from one experiment (or one observation) to another, even if conducted or measured under identical conditions.

There is a range of measured or observed values in these experimental results, and, within this range, certain values may occur more frequently than others. The variability inherent in this data or information is statistical in nature, and the realization of a specific value (or range of values) involves probability.

This is why measures like?velocity are very sporty since?past performance is rarely like future performance in the presence of Aleatory Uncertainties (as well as Epistemic Uncertainties) of actual project work.

Epistêmê in Greek means?Knowledge.

Epistemic?uncertainty reflects our lack of knowledge.

This?lack of knowledge is a?probabilistic?assessment of some outcome, usually an?event-based outcome.

There is a 40% chance of rain in the forecast area for tomorrow, which is an Epistemic uncertainty.

We assign probabilities to events, probabilities?to the work activities that create the knowledge?needed to assess the uncertainty, and probabilities of the residual uncertainties after our new knowledge has been acquired.

In practice, we can assign a?mean or a?median value to this uncertainty. That's what the weather forecast does. That 40% chance of rain is usually a?mean value. Where we live, when we hear a 40% chance in Boulder County, we know we have a lower probability?because of our micro-climate. That weather forecast is?over the forecast area and may be much different depending on where you live in that area.

This forecast also includes inaccuracies and imprecisions in the?prescribed forms of the probability distributions?and all the parameters of the estimates. This is why?forecasting the weather in some parts of the world is a very?sporty business. In places like Los Angeles, it's easy - as shown in the movie?LA Stories, where Steve Martin is the bored weatherman. Here in Colorado, with our mountain weather, making a forecast a few days from now is likely to be a challenge. As they say,?don't like Colorado weather? Wait a few hours, and it'll change.

Some Challenges to Managing in the Presence of Uncertainty

The primary issue with all uncertainties is the?communication?of the accuracy and precision of the risk created by the?aleatory and?epistemic uncertainty.?

• What is the scope of the uncertainty?
• What risks does it create to the success of the software development effort?
• Is the uncertainty time-dependent?
• At what level of decomposition of the project is the uncertainty applicable?

This is a?Risk Communication issue. So, let's restate the two forms of uncertainty.

• Aleatory uncertainty: the uncertainty inherent in a nondeterministic (stochastic, random) phenomenon… is reflected by modeling the?phenomenon in terms of a probabilistic model… Aleatory uncertainty cannot be reduced by accumulating more data or additional information.
• Epistemic uncertainty: the uncertainty attributable to the incomplete knowledge about a phenomenon that affects our ability to model it… is reflected in ranges of values for parameters, a range of viable models, the level of model detail, multiple expert interpretations, and statistical confidence. The accumulation of additional information can reduce uncertainty.

What Does This Mean for Software Development Working in the Presence of Uncertainty?

If you accept that all software development work operates in the presence of Aleatory and Epistemic uncertainty, then ...

No decisions can be made in the presence of these two types of uncertanties without estimating the impact of your decision on the project

This is a simple, clear, concise principle of managing in the presence of uncertainty. Anyone suggesting that decisions can be made without estimating has to ignore this principle willfully, OR the project is de minimus - meaning it's of no consequence to those paying if the project is late, over budget, or the delivered outcomes don't meet the needed performance level for the project to earn its?Value in exchange for the?Cost to produce that?Value.