What is Risk? – A Follow On

Risk is the effect of Uncertainty on objectives. Uncertainty is a state or condition that involves a deficiency of information and leads to inadequate or incomplete knowledge or understanding. In the context of risk management, Uncertainty exists whenever the knowledge or understanding of an event, consequence, or likelihood is inadequate or incomplete

? ISO 31000:2009, ISO 17666:2016 and ISO 11231:2010

Here's "a few hundred links to references and citations about all things risk management.

Risk is Uncertainty that Matters

Risk can be the potential consequence of a specific outcome that affects the system's ability to meet cost, schedule, and technical objectives. Risk has three primary components:?

• The probability?of an activity or event occurring or not occurring is described by a Probability Distribution Function.
• Consequence?or effect resulting from the activity or event occurring or not occurring, described by a Probability Distribution Function.
• Root Cause?(condition and activity) of a future outcome, which, if eliminated, will prevent the occurrence, non?occurrence, or recurrence of the cause of the Risk

For the program manager, three risk categories must be identified and handled:

• Technical?? risks that may prevent the end item from performing as intended or not meeting performance expectations. Measures of Effectiveness, Performance, Technical Performance Measures, and Key Performance Parameters describe the actions of these expectations.
• Programmatic?? risks that affect the cost and schedule measures of the program. The programmatic risks are within the control or influence of the Program Management or Program Executive Office through managerial actions applied to the work activities in the Integrated Master Schedule.
• Business?? risks that originate outside the program office or are not within the control or influence of the Program Manager or Program Executive Office.

Uncertainty comes from the need for more information to describe a current state or to predict future conditions, preferred outcomes, or the actions needed to achieve them. This Uncertainty can originate from the project's naturally (randomly) occurring processes (Aleatory Uncertainty). Or it can arise from the lack of knowledge about the future outcomes of the work on the project (Epistemic Uncertainty).

Risk Can NOT be eliminated; it can only be reduced with specific work activities or handled with?Margin.?Any notion that this is the case needs to be made aware of the principles of managing projects in the presence of uncertainty (Epistemic and Aleatory).

Risk, Their Sources, and Their?Handling Strategies?

Risk identification during early design phases of complex systems is commonly implemented but often fails to identify events and circumstances that challenge project performance. Inefficiencies in cost and schedule estimates are usually held accountable for cost and schedule overruns, but the root cause is often the realization of programmatic risks. A deeper understanding of periodic risk identification trends and biases pervasive during the design and development of the project's deliverables is needed, for it would lead to improved execution of existing identification processes and methods.

Risk management means building a model of the Risk, the impact of the Risk on the program, and a model for handling the Risk. Since it is a risk, corrective or preventive action has yet to occur.?Probabilistic Risk Assessment (PRA) is the basis of these models and provides the Probability of Program Success Probabilities result from Uncertainty and are central to the Risk analysis. Scenarios, model assumptions, with model parameters based on current knowledge of the system's behavior under a given set of conditions.

The source of uncertainty must be identified, characterized, and the impact on program success modeled and understood, so decisions can be made about corrective and preventive actions needed to increase the Probability of Program Success.

Since Risk is the outcome of Uncertainty, distinguishing between the types of Uncertainty in the definition and management of Risk on complex systems is useful when building risk assessment and management models.

• Epistemic Uncertainty ? from the Greek επιστημη (episteme), meaning knowledge of Uncertainty due to a lack of understanding of quantities or processes of the system or the environment. Epistemic Uncertainty is represented in the ranges of values for parameters, a range of workable models, the level of model detail, multiple expert interpretations, and statistical confidence. Epistemic Uncertainty derives from a lack of knowledge about the appropriate Value to use for a quantity assumed to have a fixed value in a particular analysis. The accumulation of information and implementation of actions reduce epistemic Uncertainty to eliminate or reduce the likelihood and impact of Risk. This Uncertainty is modeled as a subjective assessment of the probability of our knowledge and the possibility of an undesirable event.

Incomplete knowledge about some characteristics of the system or its environment are primary sources of Epistemic uncertainty.

• Aleatory Uncertainty ? from the Latin alea (a single die in Latin) is the inherent variation associated with a physical system or the environment. Aleatory Uncertainty arises from inherent randomness, natural stochasticity, and environmental or structural variation across space and time in the properties or behavior of the system under study. 5 Accumulating more data or additional information cannot reduce aleatory Uncertainty. This Uncertainty is modeled as a stochastic process of an inherently random physical model. The projected impact of the Risk produced by Aleatory Uncertainty can be managed through cost, schedule, and technical Margin.

Naturally occurring variations associated with the physical system are primary sources of Aleatory uncertainty.

• Ontological Uncertainty ? is attributable to the complete lack of knowledge of the states of a system. This is sometimes labeled an Unknowable Risk. Ontological Uncertainty cannot be measured directly.

Ontological uncertainty creates risk from Inherent variations and incomplete information that is not knowable.

Epistemic Uncertainty Creates Reducible Risk

This newsletter,?What Is Risk??responds to a tweeter post conjecturing.?

Risk is not there to be mitigated, it's there to be eliminated

This, of course, is not factually true. Risk is the result of Uncertainty, which comes in two kinds for?all projects, for everything.?Aleatory?Uncertainty from the natural?variances in the process and?Epistemic?Uncertainty?from the probabilistic event-based methods that impact the project.?Epistemic?Uncertainty and the Risk it creates can be?reduced?by?handling?operations.?Aleatory?Uncertainty and the Risk it makes can NOT be reduced. Only?a Margin?can be provided to protect the project's cost, schedule, and technical?performance from this Risk.

This brings me to the next tweet, which is equally wrong ...

Estimates lead to buffers. Buffers lead to waste. Waste leads to ruin.

Buffers (Margin) are the ONLY means of?handling?aleatory Uncertainty and the Risk it creates. Let me repeat this; aleatory Uncertainty makes a risk that is NOT reducible. This Risk cannot be reduced (NO risk can be eliminated) since it comes from a naturally occurring process (stationary or non-stationary stochastic process) external to the project.

The only means of?handling?the risk created by aleatory uncertainty is with?margin?- schedule margin, cost margin, technical margin. To suggest that?buffers?(margin) are waste, and those lead to ruin, ignores the basic principle of decision making in the presence of uncertainty.

When aleatory Uncertainty?is present, and it is present in all development work unless you're watching a clock tick, then you need?Margin. With this Margin, you're late, over budget, and have a reduced probability?that your production will only work after you start.

Before looking at how to model and handle the Aleatory uncertainties, let's look at Epistemic Uncertainty and the resulting risks.

The Risk created by Epistemic Uncertainty represents resolvable knowledge, with elements expressed as the probabilistic Uncertainty of a future value related to a loss in a coming period. Awareness of this lack of knowledge provides the opportunity to reduce this Uncertainty through direct corrective or preventive actions.

Epistemic Uncertainty, and the Risk it creates, are modeled by defining the probability that the Risk will occur, the time frame in which that probability is active, the likelihood of an impact or consequence from the Risk when it does happen, and finally, the probability of the residual Risk when the handing of that Risk has been applied.

Epistemic uncertainty statements define and model this event?based risks:

• If?Then ? if we miss our next milestone, then the project will fail to achieve its business value during the next quarter.
• Condition?Concern ? our subcontractor has not provided enough information to status the schedule, and our concern is that the program is slipping, and we do not know it.
• Condition?Event?Consequence ? our status shows some tasks are behind schedule, so we could miss our milestone. If the project fails to achieve its business value next, we need an explicit or implicit Risk handling plan.

For these types of plans. The word handling is used with a particular purpose. "We Handle Risks" in a variety of ways. Mitigation is one of those ways. However, to mitigate the Risk, we must introduce new efforts (work) into the schedule. We are buying down the Risk or retiring the Risk by spending money and consuming time to reduce the probability of the Risk occurring. Or we could be spending money and consuming time to reduce the impact of the Risk when it does happen. In both cases, we are taking action to address the Risk.

There are four kinds of reducible risks on all projects:

• Reducible Cost Risk - is often associated with unknown reducible Technical risks, changes in technical requirements, and their propagation that impact the cost.?
• Reducible Schedule Risk -?Schedule Risk Analysis (SRA)(Monte Carlo Simulation, Method of Moments, for example) is an effective technique to connect the risk information of project activities to the baseline schedule, to provide sensitivity information of individual project activities to assess the potential impact of Uncertainty on the final project duration and cost.
• Reducible Technical Risk -?is the impact on a project, system, or entire infrastructure when the outcomes from engineering development do not work as expected, do not provide the needed technical performance, or create higher than the planned Risk to the performance of the system.?
• Reducible Cost Estimating Risk depends on technical, schedule, and programmatic risks, which must be assessed to provide an accurate picture of the project cost. Cost risk estimating assessment addresses the cost, schedule, and technical risks that impact the cost estimate.?

Suppose we are to have confidence in showing up at the needed time, for the required cost, with the needed capabilities produced by our work - in the presence of Epistemic Uncertainty - we will need to identify these uncertainties and estimate the probability of their risk occurrence. In that case,?the likelihood that the?handling?actions will reduce them and the probability of the residual Risk after the handling is complete.

Managing in the presence of Epistemic Uncertainty and the Risk it creates requires making Estimates.

Risk Management is How Adults Manage Projects?- Tim Lister

Manage Risk, make estimates, be an Adult

Risk does not stand alone; Risk comes from Uncertainty - reducible and irreducible Uncertainty.

You cannot remove the risk without first removing the uncertainty.

Aleatory Uncertainty Creates Irreducible Risk

To review, Risk is the result of Uncertainty, which comes in two kinds for all projects, for everything.?Aleatory?Uncertainty from the naturally occurring variances in the process and?Epistemic?Uncertainty from the probabilistic event-based processes that impact the project. Epistemic Uncertainty and the Risk it creates can be reduced with handling processes.

Aleatory Uncertainty and the Risk it creates comes not from the lack of information but from the naturally occurring processes of the system. For aleatory Uncertainty, we cannot buy more information nor take specific risk reduction actions to reduce the Uncertainty and resulting Risk. The objective of identifying and managing aleatory Uncertainty to be prepared to handle the impacts when Risk is realized.

The method for handling these impacts is to provide?margin?for this type of risk, including cost, schedule, and technical margin.

Schedule Margin should be used to cover the naturally occurring variances in how long it takes to do the work. The cost Margin is held to cover the naturally occurring variances in the price of something we consume in our project. Technical Margin is intended to cover the naturally occurring variation of technical products.

Aleatory Uncertainty and the resulting Risk is modeled with a Probability Distribution Function (PDF) that describes the possible values the process can take and the probability of each Value. The PDF for the possible durations for the work in the project can be determined. We can buy knowledge about aleatory Uncertainty through Reference Class Forecasting and past performance modeling. This new information then allows us to update ? adjust ? our past performance on similar work will provide information about our future performance. But the underlying processes are still random, and our new information created a new aleatory uncertainty PDF.

The first step in handling Irreducible Uncertainty is the creation of a Margin. Schedule margin, Cost margin, and Technical Margin to protect the program from the Risk of irreducible Uncertainty. Margin is defined as the allowance in the budget and projected schedule … to account for uncertainties and risks.

Irreducible Schedule Risk

Projects are over budget and behind schedule, to some extent, because uncertainties need to be accounted for in schedule estimates. Research and practice are now addressing this problem, often by using Monte Carlo methods to simulate the effect of variances in work package costs and durations on total cost and date of completion. However, many such project risk approaches ignore the significant impact of probabilistic correlation on work package cost and duration predictions.

Irreducible schedule risk is handled with Schedule Margin, defined as the added time needed to achieve a significant event with an acceptable probability of success. 8 Significant events are major contractual milestones or deliverables.

With minimal or no margins in schedule, technical, or cost present to deal with unanticipated risks, successful projects are susceptible to cost growth and cost overruns.

This is the purpose of?buffers?to protect the deliverables from the naturally occurring - Aleatory - uncertainties in?All?project work.?

The notion that?Estimates lead to buffers. Buffers lead to waste. Waste leads to ruin,?is a fallacy, and violates principles of managing in the presence of Uncertainty.

Irreducible Cost Risk

Irreducible cost risk is handled with Management Reserve, and Cost Contingency is program cost elements related to program risks and is an integral part of the program's cost estimate. Cost Contingency addresses the Ontological Uncertainties of the program. The Confidence Levels for the Management Reserve and Cost Contingency are based on the program's risk assumptions, complexity, size, and criticality.

The resulting cost number is a random variable when estimating the work cost. Point estimates of cost have little Value in the presence of Uncertainty. The planned unit cost of a deliverable is rarely the actual cost of that item. Covering the variance in the cost of goods may or may not be appropriate for Management Reserve.

Assigning Cost Reserves needs knowledge of where in the Integrated Master Schedule or Product Roadmap, with its Release Plan, these cost reserves are required. The resulting IMS, with schedule margin, provides locations in the schedule where cost reserves are aligned with the planned work and provides the ability to layer cost reserves across the baseline to determine the funding requirements for the program. This allows the program management to choose realistic target dates for program deliverables and the cost reserves ? and schedule margin ? needed to protect those delivery dates.

Irreducible Technical?Risk

If we use the definition A Margin as the difference between the maximum possible Value and the maximum expected Value and separate that from contingency as the difference between the current best estimates and maximum expected estimate, then for the systems under development, the technical resources and the technical performance values carry both Margin and contingency.

• Technical Margin and Contingency serve several purposes:
• Describe the need for and use of resource margins and contingency in system development.
• Define and distinguish between margins and contingency.
• I want to show that, historically, resource estimates grow as designs mature.
• Could you provide a representative margin depletion table showing prudent resource contingency as a function of the project phase?

Any system has a maximum possible, expected, and current best estimate for every technical resource at any point in its development life. Generally, the current best estimate of resource changes as the development team improves the design, that is, as the design matures.

Most technical resources for a system in development carry both Margin and contingency. This is true historically and is independent of why development projects must plan for it to occur.

The goal of Technical Margin (unlike Cost and Schedule Margin) is to reduce the margins (for example, Size Weight and Power) to zero to maximize mission capabilities. The growth and its handling include:

• Expected growth ? contingency accounts for expected growth
• Recognize that mass growth is historically inevitable.
• As systems mature through their development lifecycle, a better understanding of the design emerges from conceptual to actual designs.
• Requirements changes often increase resource use.
• Unplanned growth ? margins account for unexpected growth
• Recognize space system development is challenging
• Projects encounter "unknown unknowns" with the use of new technology that is difficult to gauge and that develop into uncertainties in the design and execution of the program, all the way to manufacturing variations.

The Risk Register Problem

Risk matrices map the "frequency" and "severity" of corresponding risk priority levels. This approach has severe limitations:

1. Poor Resolution. Typical risk matrices can correctly and unambiguously compare only a small fraction (e.g., less than 10%) of randomly selected pairs of hazards. They can assign identical ratings to quantitatively very different risks ("range compression").
2. Errors. Risk matrices can mistakenly assign higher qualitative ratings to quantitatively smaller risks. Risks with negatively correlated frequencies and severities can be "worse than useless," leading to worse-than-random decisions.
3. Suboptimal Resource Allocation. Effective allocation of resources to risk-reducing countermeasures cannot be based on the categories provided by risk matrices.
4. (Ambiguous Inputs and Outputs. Categorizations of severity cannot be made objectively for uncertain consequences. Inputs to risk matrices (e.g., frequency and severity categorizations) and resulting outputs (i.e., risk ratings) require subjective interpretation, and different users may obtain opposite ratings of the same quantitative risks.?

Now that we've discussed the sources of risk - aleatory and epistemic uncertainty - let's address another primary flaw of the risk register when we start with a list of risks and categorize them in some ordinal ranking - with 5 scales ranging from Low, Medium, or High. Or some numeric ranking - 1, 2, 3, 4, or 5.

Those approaches fail to recognize that probabilistic or stochastic processes generate the uncertainties the risks come from.

Since all risk comes from uncertainties, and those uncertainties, come from probabilistic and statistical processes, doing arithmetic with integral equations of those processes is not possible.

You can't multiply or add two integral equations that produce the probability of occurrence and the probability of impact. The risk register can be a starting for the ordinal assessment of risk. These limitations suggest that risk matrices should be used cautiously and only with careful explanations of embedded judgments.

The details of the problem with risk matrices can be found in

• What's Wrong with Risk Matrices? Louis Anthony Cox, in "Risk Analysis, 16 April 2008.
• What's Wrong with Risk Matrices? Decoding a Louis Anthony Cox paper

Wrap Up

Risk does not stand alone. Risk comes from Uncertainty - reducible Uncertainty and irreducible Uncertainty.

You must first remove the Uncertainty before removing the Risk.

For those interested in understanding how Risk is managed on software intensive systems using agile development, here's a bibliography from our several decades of doing just that for Enterprise, embedded control systems, and space and defense systems.