The DO’s SSA and its SMS

1.???Introduction

In a Design Organisation, System Safety Engineers assess the design’s foreseeable failure modes and generate System Safety Assessments (SSA), typically as a means of compliance to design codes such as CS25.1309(b).?

With the recent (2022) amendment to EASA Part 21, the Design Organisation also need to do Safety Risk Management (SRM) as part of the Safety Management Element within the Design Management System [for brevity, let’s just refer to it as the Safety Management System (SMS)].

This paper is going to consider three things:

• The purpose of the SSA and the SMS.
• The relationship between the SSA and the SMS, particularly the potential Safety Risk Management (SRM) overlap.
• The interface between the SSA Engineer and the SMS Manager.

2.???The Purpose of the SSA and the SMS

For a new (or modified) system, the System Safety Assessment’s objective is to, inter alia, prove that an inverse relationship exists between the probability of an undesired failure condition[1] and the degree of severity inherent in its effect. ??A “Failure Condition” is defined in EASA’s CM-SA-002 Iss 1 as “ A condition having an effect on the aeroplane and/or its occupants, either direct or consequential, which is caused or contributed to by one or more failures or errors, considering flight phase and relevant adverse operational or environmental conditions, or external events”.?Note that these “failure or errors”:

• could be due to either technical failures or due to human endeavour (such as errors in commission or omission)
• do not apply to Performance and Flight Performance (see CS25 Subpart D instead)
• do not apply the design of Structures (see CS25 Subpart C instead)?

In AMC25.1309 this inverse relationship is illustrated in Fig 1 as replicated below, whereby the “unacceptable” region (in terms of Type Certification) could reveal itself pre- or post-certification:

• Prior to certification: A non-compliance to CS25.1309 discovered during qualification activities.
• Post-certification: A?foreseeable operational situation requiring MEL alleviation, or an unforeseen Occurrence Report requiring AD action.?Both strive to either reduce the Severity and/or the Probability back into the “acceptable” region (i.e. reduce this risk)

Next, let us consider the Safety Management System (as promulgated via ICAO Annex 19).??To understand the purpose of the SMS it might be worth looking at each term in the acronym separately:

• Management:?“The organisation and coordination of the activities of a business in order to achieve defined objectives”??[https://www.businessdictionary.com]
• System:??“A set of detailed methods, procedures and routines created to carry out a specific activity, perform a duty, or solve a problem”?[https://www.businessdictionary.com]
• Safety:?“The state in which risks associated with aviation activities, related to, or in direct support of the operation of aircraft, are reduced and controlled to an acceptable level”?[ICAO SMM edition 4]

ICAO’s SMM edition 4 goes on to advise us that it should be designed to “to continuously improve safety performance through: the identification of hazards, the collection and analysis of safety data and safety information, and the continuous assessment of safety risks”.?This has recently been adopted into EASA Part 21.A.239, where a key pillar/component of an SMS is to “establish, implement and maintain a safety risk management process that includes the identification of aviation safety hazards entailed by its activities, their evaluation and the management of the associated risks, including taking actions to mitigate the risks and verify their effectiveness. ?Whereby:

• “Risk” is often defined as the “predicted probability and severity of the consequences or outcomes of a hazard” [ICAO Doc 9859] and is regularly categorised as shown below (noting that the scope of this article does not extend to exploring the pros and cons of the controversial Risk Matrix):

• The “activity” of a Design Organisation is to design and certify aircraft (and changes to aircraft), not to operate aircraft (except when they do flight trials).?When it comes to how 21.A.239 has incorporated risk management we need to remind ourselves that it is to risk manage this “activity”, not the product/aircraft, as clarified below during SMS rulemaking for Part 21:

So, we can see that the purpose of an SMS is to provide organisations who are working in safety critical industries with a systematic approach to managing safety of its activities in its widest sense .?By that I mean, for Design Organisations, that it goes beyond just xx.1309….it starts off with being compliant with all relevant design codes and regulations (after all, these are all written in blood) and extends to the activities of the design organisation’s innovative endeavours. ??By making sure we do not unwittingly get it wrong, we ultimately also improve product/aircraft safety (as previously discussed here ).

?3.???The relationship between the SSA and the SMS’s SRM

At first glance, the safety criteria (or taxonomy) used in the SSA and the SMS look the same….but they are not.?

The Severity criteria are indeed very similar:?

• For the SMS, the following table[2] is taken from SM-001 :

• For the SSA, the following table is summarised from AMC25.1309 and the ICAO Airworthiness Manual , and I have added the FDAL target allocation (see next para) from ARP4754A for the purposes of showing the perceived similarities with the table above:

However, the probability criteria are very different:

• The SSA uses goal/failure-based criteria (see Chapter 5 here ) to capture the inverse relationship between failure probability and its consequence/effect.?The following is summarised from AMC25.1309:?

• ?The SMS uses risk/accident-based criteria (see Chapter 6 here ).?If we look at SM-001 risk criteria, this requires the use of (a) Table 2 above to classify consequence severity, then (b) use of Table 1 below to qualitatively determine consequence likelihood (the subjective and controversial bit of risk management) before (c) prioritising it in the risk matrix of Table 3 below:

The following picture from ICAO’s SMM edition 2 could be used to neatly illustrate the difference between the criteria used in the SSA and the SMS:

• The SSA considers the worst-case consequence of the failure condition and then sets an internationally acceptable design target to reduce the probability of the pot falling off the ledge.
• The SMS considers the complete sequence of events (beyond just the design of the ledge) and looks into what could be done to reduce the risk to an acceptable level (e.g. warning signs, hard hats, net under the ledge, etc.)? Note: In Part 21 language many (but not all) of these are the ICAs, Manuals and Service Bulletins that the DO generates and maintains (via the 21.A.3A Occurrence Reporting System), but which other organisations may evolve (under their privileges) and need to comply with to manage their through-life risk.

4.???The interface between the SSA Engineer and the SMS Manager

Using the tables above, the SSA engineer can inform the SMS Manager of the potential worst-case Severity of technical failure conditions for the systems being modified/installed on the aircraft.?

However, for the purposes of Risk Management within the wider Management System of your “activities”, the SSA Engineer cannot determine the Likelihood of the end consequence.?For this the SMS Manager needs to use:

• The Occurrence Reporting system [refer 376/2014, 21.A.3A & 21.A.239(c)(6)] to evaluate the effectiveness our internal procedures and competence (including those that generated the SSA).?Occurrence Reports helps the SMS Manager to evaluate the number and size of the holes in the proverbial ?James Reason’s Swiss Cheese model [or the “measurement and monitoring of the organisation’s safety performance” as per 21.A.239(c)4(i)], with the intent to prevent the occurrence (or re-occurrence ) of these organisational failure conditions (i.e. “continuous improvement” of the Design Management System as per 21.A.239(c)4(iii)].
• Using integrated risk management criteria (see para 1.4 here ), ?to risk assess all possible consequences of that organisation hazard (i.e. not just the accident)

Both of these can be visualised in the incomplete, but illustrative, example below: ?

5.???Conclusion

On the face of it, the SSA and the SMS have the same intent and use similar terminology.?But, in fact they are quite different. ?In a Design Organisation using xx.1309 design targets, the:

• SSA is all about the integrity of the technical system.?It examines the robustness of the technical system and whether their failure rates meet acceptable targets
• SMS is all about the integrity of the Management System that created the technical system (see the end of this EASA page , which is replicated below).?It searches for failures and shortcomings in our intellectual endeavours, puts in place mitigations again them and then goes on to monitor the effectiveness of those mitigations.

Note:???For the SSA discussion above, I have used the civil aviation approach as promulgated in design codes such as CS25.1309.??In the absence of such a goal/failure-based approach to the SSA, some military organizations are contacted to the risk/accident-based approach in standards such a MIL-STD-882 and Def Stan 00-56. This often complicates matters: ?Firstly, Design Organisations?are then faced with a struggle to select system architectures which will meet yet-to-be agreed design/FDAL targets when bidding for fixed price contracts.?Secondly, the SMS demanded in these standards have a different intent than the ICAO Annex 19 approach.??In these standards the SMS is actually focussed on how the SSA (or Safety Case) is generated and agreed between all impacted stakeholders?…. and not on the performance of the complete Management System, as discussed by EASA below:

Footnotes

[1] A “Failure Condition” is defined in EASA.s CM-SA-002 Iss 1 as “ A condition having an effect on the aeroplane and/or its occupants, either direct or consequential, which is caused or contributed to by one or more failures or errors, considering flight phase and relevant adverse operational or environmental conditions, or external events”. ?Note that these “failure of errors” could be due to technical failures or due to human endeavour (such as errors in commission or omission)

[2] Note that this Table 2 from SM-001 is very product/aircraft centric and ignores other organisational consequences, so should be extended to address organisational performance consequences too.?For more information, see ICAO’s discussion on Integrated Risk Management in para 1.4 here .

END

Your comments/thoughts/critique/contributions would be appreciated