“STPA vs. ARP4761” or “STPA and ARP4761” – thoughts on how to get the aerospace industry to work with System-Theoretic Process Analysis

ARP4761 – Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment [1] has been widely in use for almost quarter of a century. It forms the basis of safety assessment processes for many aerospace companies, to the extent that it has become the “common language”. Its first revision is on its way; but will it successfully capture the arguments of System-Theoretic Process Analysis (STPA)?

STPA is a relatively new hazard analysis method, based on systems theory, which has its focus on the interactions of systems rather than failures [2]. STPA approaches safety as a control issue rather than a reliability or failure-related one [3]. This is very relevant, if the fatal aviation accidents of the last few decades are considered.

Can STPA replace ARP4761?

STPA is a very promising method that has been created with the right arguments, but the short answer is, until there is a full-scale development programme that uses STPA instead of ARP4761, we don’t know. Replacing ARP4761 with STPA would be very challenging, though. First, there isn’t such a development and certification programme to date at platform level, which has used STPA instead of ARP4761. Furthermore, a complete change of existing structure is a very difficult thing to achieve. Many resources, processes and tools are oriented towards (and experienced in; speaking of resources) working with ARP4761. Having used STPA and being a supporter of it, my proposal would be to integrate STPA into safety analysis and complement ARP4761, rather than to replace it.

The comparison report issued by Leveson et al. [4] is a valuable study that provides the deepest analysis to date that compares the two approaches using the well-known “decelerate aircraft on ground” example [1]. As an expert who has been working with ARP4761 based safety processes since 2003, I believe that some arguments of [4] need to be reviewed further. I hope that this review would be useful for a potential update of [4] against the future Rev. A of [1].

First of all, ARP4761 is a guideline. Companies usually have their own safety processes that go beyond the scope of ARP4761. Hazard identification and management is one example of this, and it goes in the direction of STPA in the sense that the identified hazards may involve human errors, specification errors or specific scenarios. Nevertheless it is not as structured as the STPA, and the management of the hazards (i.e. driving them to robust closure) may require different analysis methods. Still, it is potentially “the candidate area” to start applying STPA, because they both start with hazard identification, and the use of a common method to analyse hazards would not only bring a better structure to the work stream but it would also help the practitioners gain STPA experience in a relatively short timeframe.

Second, ARP4761 activities focus on more than just 14 CFR §25.1309 [5]. There are other certification chapters that require safety analysis to drive design decisions. For instance, §25.901(c) requires the following: “For each powerplant and auxiliary power unit installation, it must be established that no single failure or malfunction or probable combination of failures will jeopardize the safe operation of the airplane except that the failure of structural elements need not be considered if the probability of such failure is extremely remote.”. Another example is §25.1103(d), which requires that no hazard results if a duct failure occurs at any point between the air duct source and the airplane unit served by the air [5]. Similarly for engine certification, although the safety assessment primarily addresses CS-E 510 Safety Analysis [6], there are additional CS-E chapters that have safety requirements which need to be analysed by the safety teams to further identify physical safety requirements. For instance, CS-E 50(c)(3) states that the engine control system must be designed and constructed so that single failures of engine control system components do not result in a Hazardous Engine Effect [6]. Also, CS-E 810(a) (Compressor and Turbine Blade Failure) requires the following: “It must be demonstrated that any single compressor or turbine blade will be contained after Failure and that no Hazardous Engine Effect can arise as a result of other Engine damage likely to occur before Engine shut down following a blade Failure.” [6]. Heavy vibrations following a blade release could become a common cause threat that may challenge the independence of the lines of defence against a Hazardous Engine Effect. Therefore, the safety analysis generates requirements on one or more of these lines of defence so that they are designed to withstand (and remain functional under) this level of vibrations. So in summary, the result of the ARP4761 analysis does not only consist of probabilistic requirements.

It is true, however, that both ARP4761 and certification chapters relevant to safety analysis are failure focused. Instead of assuming that the analysed configuration meets the design intent and letting extensive system verification tests and analyses take care of this, the safety analysis could as well proactively set the requirements on how the design should work rather than how they should (or should not) fail. The system boundary definition for safety assessment could be extended to account for this, and that is where STPA provides a very clear advantage in terms of including specification errors, operators and maintenance personnel into the analysis.

Can STPA be used in combination with ARP4761?

As part of my keynote lecture in CM2019 conference [7], I presented how sensing systems can be used as safety devices to minimise dormant failures. To identify the architectural and safety requirements on the sensing system, I took a combined approach of ARP4761 and STPA in a case study. There was an overlap between some common cause analysis requirements and STPA constraints. This could potentially highlight which parts of ARP4761 could initially be transformed into (or replaced with) STPA. Second, as expected, most of the STPA requirements were on the system design specification. Some of the specifications which the safety engineer could traditionally consider to be outside their analysis domain were now identified as safety constraints via STPA. Just to highlight the different nature of these requirements coming from different analyses [7], they are listed below.

________________________________________________

Architectural requirements

• The sensing system, including its software logic, shall be developed at least to Safety Integrity Level X (or Development Assurance Level Y).
• The sensing system shall have dual lane redundancy*.

Safety requirements from Fault Tree Analysis and Common Cause Analysis (ARP4761 methods)

• Complete loss of air pipe failure detection shall not occur at a rate in excess of TBD per hour*.
• The detection system shall remain functional during and after air pipe failure*.
• Failure of the detection system shall not lead to failure of the air pipe.

Additional safety requirements from System-Theoretic Process Analysis

• Erroneous failure detection of the sensing system shall not exceed a rate of TBD per Engine Flight Hour.
• Sensing system shall detect failures for a crack size longer than TBD mm (lower threshold to ensure detectability for confirmation during post-event maintenance).
• Sensing system shall detect failures before the crack size reaches TBD mm (upper threshold to minimise/eliminate exposure to a potentially unsafe scenario).
• Sensing system shall provide measurement signals to the aircraft computer continually at a frequency higher than TBD Hz.

*: also identified by STPA

 ________________________________________________

A similar approach could also be applied to the analysis of a much bigger system, or even platform, combining ARP4761 and STPA.

Recommendations

In order to get STPA into aerospace safety assessments, this would be my recommended approach:

• To form an active worldwide community for STPA practitioners. It needs to be interesting to join, follow and contribute to this community via tutorials, use cases, Q&A, etc. with the end goal of having “STPA ambassadors” in the industry.
• To focus the initial use of STPA in hazard identification and management activities and to get feedback so that the process can be improved.
• To perform a full platform level safety analysis with STPA and with ARP4761 in order to have a comparison that goes beyond the limitations of the examples of [4] and [7]. There might be ways to do this – eVTOL and conventional aircraft start-up companies may be looking for support to perform safety assessments on their offerings.
• To win STPA friendly allies in the committees shaping the ARP4761. The ground work laid out in the earlier recommendation steps would be helpful in achieving this.
• To identify and review the ARP4761 chapters which could effectively be transformed to (or replaced with) STPA type of analysis.

References

1. SAE International (1996), Aerospace Recommended Practice ARP4761, Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment
2. Leveson, N.G. (2011), Engineering a Safer World, The MIT Press, Cambridge, MA, USA
3. Rising, J.M. and Leveson, N.G. (2018), Systems-Theoretic Process Analysis of space launch vehicles, The Journal of Space Safety Engineering, 5(2018), pp. 153-183
4. Leveson, N, Wilkinson, C., Fleming, C., Thomas, J. and Tracy, I. (2014), MIT PSAS Technical Report Rev. 1, A Comparison of STPA and the ARP 4761 Safety Assessment Process, available at: https://sunnyday.mit.edu/papers/ARP4761-Comparison-Report-final-1.pdf (accessed 23 January 2020)
5. Title 14 of the Code of Federal Regulations, Part 25 – Airworthiness Standards: Transport Category Airplanes, available at: https://www.ecfr.gov/cgi-bin/text-idx?tpl=/ecfrbrowse/Title14/14cfr25_main_02.tpl (accessed 23 January 2020)
6. European Union Aviation Safety Agency (2018), Certification Specifications and Acceptable Means of Compliance for Engines CS-E, Amendment 5
7. Türkaslan, M. (2019), Wider Use of Sensing Capabilities in Safety Analysis to Minimise Dormant Failures, The Sixteenth International Conference on Condition Monitoring and Asset Management (CM2019) (submitted for publication)