Why Quantitative Risk Assessments Fail When It Matters Most

Why Quantitative Risk Assessments Fail When It Matters Most

Quantitative risk assessments (QRA) help businesses navigate uncertainty by turning gut feelings into structured, data-driven predictions. They tell you the probability of common risks, from delays to cost overruns.

But there's a critical weakness: they fail when facing truly extreme, unexpected or rare events—known as Black Swans.

Why does this happen?

  • Narrow Problem Framing: Most risk analyses focus only on familiar scenarios. If you don't ask about risks that seem impossible, you'll never model them.
  • Overreliance on Past Data: Black Swans, by definition, are unprecedented. Historical data can’t predict something that has never happened or happens only very, very seldom.
  • Ignoring Extreme Events: Standard probability models underestimate the frequency of massive disruptions. Real-world disasters follow a different pattern—Power Laws, not Bell Curves.
  • Trusting Simulations Too Much: Monte Carlo simulations only consider scenarios you've already defined. They won't alert you to the risks you haven't imagined.

What Should You Do Instead?

  1. Think Beyond Probability: Identify scenarios that can break your business—even if they seem unlikely.
  2. Look Across Industries: Learn from unexpected failures in other sectors.
  3. Test Fragility, Not Just Forecasts: Build systems that withstand shocks, rather than trying to predict every threat.
  4. Use Expert Judgment: Complement quantitative models with insights from experienced professionals who recognize hidden risks.

The greatest danger isn't a risk you can measure—it's the one you ruled out, ignored or simply haven't imagined.

For my paid subscribers, I have a detailed analysis where and why quantitative risk models fail under extreme events.

Sign up here:

https://marcoresilienceengineers.substack.com/

Have a great Weekend!

Marco


P.S.:

I have only 2 Spots left for my “The Ultimate Guide to AI in Security Risk Management” in March.

I’ll show you how you can leverage AI for the whole Security Risk Management Process.

From Identifcation, Quantification to Mitigation. For Cyber and Physical risks.

Sign up here:

https://www.canva.com/design/DAGbVDjA5zE/bNjsj9T_DsjS8weUiTfpYg/view?utm_content=DAGbVDjA5zE&utm_campaign=designshare&utm_medium=link2&utm_source=uniquelinks&utlId=h6c98c5b24b

Thought I'd chime in, here. We utilize both Qualitative and Quantitative methods to assess risk for our manufacturing clients. Here's the rub we encounter all the time. Invariably, we come up with 20 identified risks for a client. About 90 percent of all those risks are Tactical and Operational. Only 1-in-10 will be Strategic. We attempt to quantify all the risks using probabilistic models. Here's what happens. The Tactical and Operational risks have HIGH frequency of occurrence, but pretty low negative impact to the company. AND, we have developed many tactics to mitigate those risks. However, when we calculate the probability of the strategic risk, it comes up very small, YET, the risk to the company can be existential. A dilemma for management. It's not a Black Swan......maybe a Gray Rhino. Soooo, what do we do. We advocate developing a Mitigation Plan to handle the risk, if it ever occurs. It's call The Strategic Risk Paradox, by the RIMS organization. NET-NET......CRO's and all risk pros spend less than 8 percent of their time on Strategic Risks, yet........those risks, according to RIMS studies, represent over 85 percent of all MAJOR financial distress. Lot of work yet to be done.

Why do screwdrivers fail to hammer nails? Quantitative RM "fails" because it's not the best tool for the job. "Narrow Problem Framing" is needed for a quantitative risk analysis with high confidence level. "Overreliance on Past Data" is always bad as expressed by "Overreliance" "Trusting Simulations Too Much" you should trust your model if it has been validated. That trust needs to be established. Using the wrong model and tool is not an issue with the simulation. I'm all for building more resilient systems (systems in the larger context, like organizations, etc.). I'm not so sure about "Fragility." I haven't encountered antifragile systems in my domain of cybersecurity. So, is the quantitative risk assessment failing, or are we failing to apply the right tool for the job?

Marco Felsberger

I help Risk & Resilience Managers build unique knowledge to become a top 1% Resilience Engineer, with innovative but proven Resilience Strategies | Master Risk, Resilience, Antifragility & Complexity

2 天前

Of course, we should still quantify risks in all situations where it adds value. Just beware of the results when it comes to extreme events.

回复

要查看或添加评论,请登录

Marco Felsberger的更多文章