Programmatic Risk Management

Risk Management is How Adults Manage Projects

Risk management is essential for the success of any significant project. [1] Information about key project cost, performance, and schedule attributes is often known once the project is underway. Risks identified early in the project that impact the project later are often termed “known unknowns.” These risks can be mitigated, reduced, or retired with a risk management process. For risks beyond the project team's vision, a properly implemented risk management process can also rapidly quantify the risk's impact and provide sound plans for mitigating its effects.

Risk management is concerned with the outcome of a future event whose impacts are unknown. Two classes of uncertainty produce "knowable" - Aleatory and Epistemic. The 3rd type, Ontological, is unknowable.

Risk management is about dealing with this uncertainty. Outcomes are categorized as favorable or unfavorable. Risk management is the art and science of planning, assessing, handling, and monitoring future events to ensure favorable outcomes. A good risk management process is proactive and fundamentally different than reactive issue management or problem-solving.

This News Letter describes the fundamentals of Risk Management with five simple concepts:

1. Hope is not a strategy – Hoping that something positive happens or something negative will not happen does not lead to success. Preparing for success is the basis of success.
2. All single-point estimates are wrong – Single-point estimates of cost, schedule, and technical performance are no better than 50/50 guesses without knowledge about the variances of the underlying distribution.
3. You are driving in the rearview mirror without integrating Cost, Schedule, and Technical Performance. The effort to produce the product or service and the resulting value can only be made by making these connections.
4. Without a model for risk management, you are driving in the dark with the headlights off – Risk management is not an ad hoc process you can make up as you go. A formal foundation for risk management is needed. Choose one that has worked in high-risk domains – defense, nuclear power, manned spaceflight, or examples.
5. Risk Communication is everything – Identifying risks without communicating them wastes time for all the participants.

Risk management is an essential skill that can be applied to various projects. In an era of downsizing, consolidation, shrinking budgets, increasing technological sophistication, and shorter development times, risk management provides valuable insight to help key project personnel plan for risks. It alerts them to potential risk issues, which can then be analyzed and plans developed, implemented, and monitored to address risks before they surface as issues and adversely affect project cost, performance, and schedule.

Hope is Not a Strategy

Hoping that the project will proceed as planned is not a strategy for success. Project managers who constantly seek ways to eliminate or control risk, variance, and uncertainty engage in a hopeless pursuit.

Managing “in the presence” of risk, variance, and uncertainty is the key to success. Some projects have few uncertainties –only the complexity of tasks and relationships is essential – but several types of uncertainty characterize most projects. Although each uncertainty type is distinct, a single project may encounter some combination of four types: [2]

1. Variation – comes from many small influences and yields a range of values in a particular activity. Attempting to control these variances outside their natural boundaries is a waste of time.
2. Foreseen Uncertainties – are identifiable and understood influences that the team cannot be sure will occur. There needs to be a mitigation plan for these foreseen uncertainties.
3. Unforeseen Uncertainties – can’t be identified during project planning. When these occur, a new plan is needed.
4. Chaos – appears in the presence of “unknown unknowns” and must be addressed through a replanning process.

Plans are strategies for the successful completion of the project. Plans are different than schedules. Schedules show “how” the project will be executed. Plans show “what” accomplishments must be performed and the success criteria for these accomplishments along the way to completion.

The Plan describes the increasing maturity of the project through assessment points. The unit of measure for this maturity must be meaningful to the stakeholders. Something that can be connected to their investment in the project.

When we speak the word “Hope,” it lays the foundation for failure. In the use of Hope, we mean “success is possible but not probable.” When we speak the word “Plan,” it does not assure success, but success is a probable outcome. It is the definition of the probability of success P(s), that is the foundation of the Plan. Having a Plan–A, Plan–B, and possibly a Plan–C exposes risk, assigns mitigations, and measures the probability of success. [3]

The idea of a Plan as a Strategy is critical to making changes in the behavior of the project teams that can lead to “risk-adjusted project management.” Without a Plan, the schedule is simply a list of activities to be performed. The reason for their performance may be understood, but it is unlikely these activities fit in any cohesive Strategy. Strategies have goals, critical success factors, and key performance indicators. Project Strategies – the Master Plan – must also contain goals, critical success factors, and key performance indicators that ensure the project is making physical progress despite uncertainty in cost, schedule, and technical performance.

No Single-Point Estimate of Cost, Schedule, or Technical Performance Can Correct

How long will this take? How much is it going to cost? Will the product or service meet the requirements defined for any specific point in time? What is the confidence in those numbers? These three questions must be answered for the project team to have a credible discussion with the stakeholders about success. A starting point is determining the accuracy needed to provide a credible answer. But that does not address the question – “how can that accuracy be obtained.”

There are many checklists for estimating cost and schedule, with simple guidance on building estimates. Most of this advice needs to be corrected fundamentally. ?The numbers produced by the estimating process do not have their variance defined in any statistically sound manner. By statistically sound, it means that the underlying probability distributions are known. If they are unknown, then some form of estimating taking this unknown into account must be used.

The Project Management Institute (PMI) advices producing three estimates – optimistic, most likely, pessimistic. But these numbers are fraught with error:

• We can’t tell how these numbers were arrived at?
• Are they based on the best engineering judgment?
• Based on historical data?
• What is the variance on the variance of this distribution – the 2nd standard deviation?

With this information, they are of use in estimating risk.

Using point estimates for duration and cost is the first approach in an organization low on the project management maturity scale. Understanding that cost and durations are “random variables” drawn from an underlying distribution of possible value is the starting point for managing uncertainty.

In probability theory, every random variable is attributed to a probability distribution. The probability distribution associated with cost or duration describes the variance of these random variables. A common distribution of probabilistic estimates for cost and schedule is the Triangle Distribution.

The Triangle Distribution in Figure 2 can be used as a subjective description of a population for which only limited sample data exists, especially where the relationship between variables is known but data is scarce. It is based on the knowledge of the minimum and maximum and a “best guess” of the modal value (the Most Likely).

A Monte Carlo simulation of the network of activities and their costs can be performed using the Triangle Distribution for cost and duration. In technical terms, Monte Carlo methods numerically transform and integrate the posterior quantitative risk assessment into a confidence interval. The result is a “confidence” model for the cost and completion times for the project based on the upper and lower bounds of each distribution assigned to the duration and cost.

Integrating Cost, Schedule, and Technical Performance

In many project management methods – cost, schedule, and quality are described as an “Iron Triangle.” Change one, and the other two must change. This is too narrow a view of what's happening on a project. It’s the Technical Performance Measurement that replaces Quality. Quality is one Technical Performance measure.

Cost and Schedule are apparent elements of the project. Technical Performance Measures (TPM) describe the status of the technical achievement of the project at any point in time. ?The planned technical achievement is part of the Performance Measurement Baseline (PMB).

The Technical Performance Measurement System (TPMS) uses the techniques of risk analysis and probability to provide project managers with the early warnings needed to avoid unplanned costs and slippage in schedule. Systems engineering uses technical performance measurements to balance cost, schedule, and performance throughout the project life cycle.?

Connecting Cost, Schedule, and Technical Performance Measures closes the loop on how well a project achieves its technical performance requirements while maintaining its cost and schedule goals. IEEE 1220, EIA 632, and “A Guide to the Project Management Body of Knowledge” all guide TPM planning and measurement and integrate TPM with cost and schedule performance measures (Earned Value). [4]

Technical performance measurements compare actual versus planned technical development and design.? They report the degree to which system requirements are met regarding performance, cost, schedule, and progress in implementing risk retirement. Technical Performance Measures are traceable to the user–defined capabilities. Integrating these three attributes produces a Performance Measurement Baseline that:

• Is a plan driven by product quality rather than work or effort requirements?
• Focuses on technical maturity and quality, in addition to cost and schedule.
• Focuses on progress toward meeting success criteria of technical reviews.
• Enables insightful variance analysis.
• Ensures a lean and cost–effective approach to project planning and controls.
• Enables scalable scope and complexity depending on risk.
• Integrates risk management activities with the performance measurement baseline.
• Integrates risk management outcomes into the Estimate at Completion.

The Cost and Schedule “measures” are straightforward in most cases. Technical Performance Measures (TPM) involve Measures of Effectiveness (MOE) and Performance (MOP).

Measures of Effectiveness (MOE) are the operational mission success factor defined by the customer. These are:

1. Stated from the customer's point of view
2. Focused on the most critical mission performance needs
3. Independent of any particular solution
4. Actual measures at the end of development

Measures of Performance (MOP) characterize physical or functional attributes relating to the system operation:

1. Supplier’s point of view
2. Measured under specified testing or operational conditions
3. Assesses delivered solution performance against critical system level specified requirements
4. Risk indicators that are monitored progressively

Programmatic Risk Must Follow a Well Defined Process

Using an ad hoc risk management process is its self risky. The first place to start looking for risk management processes is where risk management is mandatory – aerospace, defense, and mission-critical projects and projects. These also include ERP and Enterprise IT projects.

Technical performance is a concept absent from the traditional approaches to risk management.? Yet, it is the primary driver of risk in many technology-intensive projects. Cost growth and schedule slippage often occur when unrealistically high-performance levels are required and little flexibility is provided to degrade performance during the project. Quality is often a cause rather than an impact on the project and can generally be broken down into Cost, Performance, and Schedule components.

The framework shown in Figure 5 guides:

• Risk management policy
• Risk management structure
• Risk Management Process Model
• Organizational and behavioral considerations for implementing risk management
• The performance dimension of the consequence of the occurrence
• The performance dimension of Monte Carlo simulation modeling
• A structured approach for developing a risk-handling strategy

Risk Communication

Risk management activities must properly communicate risk to all participants to be effective. Risk is usually a term to be avoided in normal business. Being in the risk management business is not desirable in most businesses – except insurance. It is expected to “avoid” the discussion of risk.

Communicating risk is the first step in managing risk. Listing the risks and making them public is necessary but more is needed. ?Risk communication is the basis of risk mitigation and retirement. A risk management plan and defined mitigations serve only purpose with risk communication.

The Risk Management Plan must address:

• Executive summary – a summary of the project and the risks associated with the activities of the project. Each risk needs an ordinal rank, a planned mitigation is active (a risk approved by the Risk Board), and the mitigations are shown in the schedule with associated costs.
• Project description – a detailed description of the project and the risk associated with each deliverable.
• Risk reduction activities by phase – using some formal risk management process that connects risk, mitigation, and the Integrated Master Schedule (IMS). The efforts for mitigation need to be in the schedule.
• Risk management methodology – using the DoD Risk Management process is a good start. [1] This approach is proven and approved by high-risk, high-reward projects. The process steps are not optional and should be executed for ALL risk processes.

To communicate risk, clear and concise language is needed. There are better choices than English. Ambiguity and interpretation are two issues. Communicating in mathematical terms is also a problem since the symbols and units of measure may need to be clarified.

Figure 5 is from the Active Risk Manager [6] tool that connects risk management with the scheduling system. ARM is a proprietary risk management system that illustrates how risk is retired over time in accordance with a plan. The concept shows explicitly when each risk will be “bought down” or “retired” during the project execution. The Risk Registry and the Integrated Master Schedule must be connected somehow. Without this connection, no Risk Management process can be used to forecast impacts on cost or schedule.

At each project maturity point, current risks, the planned retirements of these risks, and the project's impact must be visible in the schedule. With these connections, project managers can then answer the following questions:

• What happens if this risk is not retired?
• What effort is needed to retire this risk before a specific time?
• If this risk becomes an issue, what is Plan B? How much will Plan B cost? What is the impact of Plan B on the deliverables?
• What cost and schedule reserve is needed to cover all the currently active risks?

Wrap Up

Risk management can be applied to all three elements once cost, schedule, and technical performance are integrated into the Performance Measurement Baseline. With these connections, the project management team can confidently say, “We are doing risk management on this project.”

I would like to remind you to make sure that all five risk management elements are present. ?Leaving one out reduces the effectiveness of the risk management process and increases the risk to the project. Project risk management is a Practice. The theory of Project Risk Management is important, but the Practice is how project risk gets managed.

[1] Risk Management Guide for DoD Acquisition 2003 (Fifth Edition, Version 2.0), www.dau.mil/pubs/gbbks/risk_management.asp

[2] “Risk Management during Requirements,” Tom DeMarco and Tim Lister, IEEE Software, September/October 2003

[3] “Managing Project Uncertainty: From Variation to Chaos,” Arnoud De Meyer, Christoph H. Loch and Michael T. Pich, MIT Sloan Management Review, Winter 2002.

[4] “Probability of Success Operations Guide, Acquisition, Logistics & Technology Enterprise Systems & Services, Office of the Assistant Security of the Army for Acquisition

[5] Performance Based Earned Value, Paul Solomon and Ralph Young, John Wiley & Sons, 2006.

[6] www.strategicthought.com

? “How Much Risk is Too Much Risk,” Tim Lister, Boston SPIN, 20 January 2004.