The Need for Systems Thinking

With complex systems, behavior is not determined by cause-effect exclusively. That is, not everything is chains of events with a root cause. Rather, a system's behavior emerges from the structure of feedback loops associated with the system, something understood by the use of systems thinking. The graphic with this article shows a simple structure of feedback loops.

Systems thinking is an approach (a philosophy?) that sees the world and its systems in terms of wholes and relationships, rather than breaking it down, or decomposing it, into parts. It considers how the pieces interact to create the whole. For a system, it requires understanding relationships with the larger world - the environment, humans, other systems.

Security can not be completely analyzed in threat based or threat informed ways, as commonly practiced. Nor can it be completely analyzed via vulnerability assessments. These are approaches rooted in cause->effect chains, more precisely a threat A exploits a vulnerability B to create an effect C, that is, A->B->C.

What is needed to think in terms of feedback loops that extent beyond a system's boundary. Doing so sees the system completely, or at least more completely. For example, Peter Lewycky in 1987 described that explaining safety accidents/incidents can be done in terms of constraints, conditions, and mechanisms. Mechanisms are the chain of events, but to understand them and best prevent incidents in the future, the need is to understand the conditions or lack of conditions that permitted the chain of events. Finally, one needs to understand the constraints or lack of constraints that permitted the conditions. These conditions and constraints, and their lack, are what go beyond a system's boundary.

William "Dollar" Young and Nancy Leveson wrote about this about 11 years ago in "Systems Thinking for Safety and Security". If we take our feedback loops and turn them a few degrees, we see system behavior of the system with feedback loops that amount to closed loop controls (classic system controls, not NIST SP 800-53 use of that term). When we look at the loops, we can see where we need to add loops to achieve safe and secure behavior, we can see where the loops can go wrong or are susceptible to manipulation by malicious actors.

Systems Thinking is a foundational skill of systems engineering - it needs to be one for systems security engineers and really any security engineer. The community needs to get beyond thinking of risk as a function of threat and vulnerability only - not to mention just get beyond it is all about risk management as the means to achieve security. Need systems thinking to achieve assured function as part of intended behaviors and outcomes.

Unless otherwise stated, all views expressed are mine and don’t necessarily reflect those of my employer or MITRE sponsors.