Managing Risk in Complex Systems: we do need a paradigm shift, but it can be evolution not revolution.

Assessing the efficacy of safety critical systems in today’s complicated engineering applications is a challenging responsibility. Traditionally, this has been done by establishing logically the (root?) causal links between specific components and the consequences of their failure. The standard way of doing this is either by inspection, qualitatively (Failure Modes and Effects Analysis, FMEA), or more rigorously by logic, or decision trees, which can be quantified using Boolean algebra (so called Fault Tree analysis). Given these predetermined and fixed relationships, observance of such effects could be assumed to have occurred as a consequence of the appropriate component failure. If we know the reliability of these components, we can then establish a system integrity level for our system. If the safety system itself is a component, then a more extensive set of fault trees is assumed to predict the overall integrity (or safety?) of the whole system.

But today’s systems have tended to become ever more complicated and qualify as at least, complex (Snowden[1]) if not occasionally ?chaotic, with the increasing involvement of artificial intelligence in their control and safety management. If we add to this, the fact that human intelligence is also intimately involved in these issues (often conveniently ignored?), then the now complex sociotechnical systems have yet more (human?) factors to control. The challenge of now analysing formally what goes on in these so called complex sociotechnical systems, is a major impediment to our achieving an adequate understanding of how they behave (how safe they are?) in operation. To do this, it is suggested, (Leveson[2]), that we need a paradigm shift in our approach.

In these more complex systems, there are at least, two new issues to address. Firstly, the simplistic fixed linear cause and effect relationships are no longer reliable predictors of performance, or accurate symptoms for diagnosing “faults” from observed effects. This is primarily as a result of more “complex” behaviours, in which the decidedly nonlinear interdependence of interacting agents with the inherent variabilities in conditions in the real world, means we have to allow for sometimes unexpected and unpredicted “Emergent” outcomes in these systems. Secondly, effects can be seen that have no obvious relationships to the tidy sequential linear logic diagrams used in their design.

But if we want to establish quantitatively, the reliability and hence safety of these systems, we must have some way of legitimately and accurately, modelling these systems and their inevitable interdependencies and interactions with each other and the real world in which they operate.

This note argues that we can now utilise the Functional resonance analysis method (Hollnagel[3]) to include nonlinear relationships and predict emerging behaviours in such systems. These models can then be used as the equivalent dynamic logic trees to traditional FTA and can be quantified using reliabilities as conditional probabilities of success or failure. These are now Dynamic Bayesian Nets, rather than static Boolean gates, allowing for Markovian development of instantiations of the models to predict these emergent effects.

This is very much in line with the traditional engineering approaches to assessing and assuring the safety of systems. The only difference is that we have now evolved the static Boolean Fault Trees into the more advanced Markovian and Bayesian Belief Nets (FTA to DBBN) (Slater[4])

One of the major advantages of the FRAM methodology as an approach to tackling this issue, is its ability systematically, to probe FRAM instantiations, to observe the evolving effects (output variabilities) of variations in the interactions and interdependencies between the functions. To date this has been limited to queries such as what if the input is too late to start the function, or the preconditions are not met, or the resources or controls are not adequate. The results thus focus on the resulting outcomes when the Aspects are present or not (True or False) and the resulting output happens or not, again True or false. (Slater[5])

However, in real life, conditions are rarely black and white, and the processes get completed anyway, with less than perfect “aspects” available. Finding a way to model this reality and to track the effects of true variability in functional interactions, generating Markowian emerging effects in complex systems, has long been a need / must have for most experienced system analysts. We are now in a position to achieve this and meet our objective to achieve and apply an evolution of paradigm from complicated to complex.

[1] Snowden, David J.; Boone, Mary E. (2007).?"A Leader's Framework for Decision Making".?Harvard Business Review.?85?(11): 68–76.?PMID?18159787.

[2] Leveson, N., (2023) A Paradigm Change for Safety and for System Engineering Presentation to the Object Management Group MBSE workshop https://omgwiki.org/MBSE/lib/exe/fetch.php?media=mbse:incose_mbse_iw_2023:0.0.2023-01-28.iw2023_mbse_workshop_plenary_leveson_incose_keynote_talk.pdf

[3] Hollnagel, E., (2012), FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-technical Systems CRC Publications

[4] Slater, D., (2023) Complex System Modelling, Presentation to the Complex System Modelling Meeting: Safer Complex Systems Working Group, Safety Critical Systems Club, DOI: 10.13140/RG.2.2.27522.04806 (PDF) Complex System Modelling 1.1 (researchgate.net)

[5] Slater, D., (2022), FRAM Offshore, Conference: UKFRAMily 2022, Edinburgh,?https://www.researchgate.net/publication/363469351_FRAM_Offshore