Pilot error versus sociotechnical systems failure: a distributed situation awareness analysis of Air France 447

This paper unpacks the Air France 447 crash from 2009 and provides an alternate view of the accident’s genesis.

While the official report “place scrutiny on the aircrew’s subsequent lack of awareness of what was going on and of what procedure was required, and their failure to control the aircraft”, this view is inappropriate and that it’s “systems, not individuals [that] lose situation awareness” (p1).

Providing background it’s noted:

·????????The misunderstood use of situation awareness (SA), such as from a pure cognitive/individual level “threatens its potential contribution to safety science” (p2)

·????????“focusing on individual cognition during accident investigation inevitably leads to countermeasures which focus on fixing human operators through avenues, such as retraining and reprisals, an approach that has long been known to be inappropriate” (pp2-3)

·????????Because of the myopic focus on individuals, “our discipline may no longer be doing what it should be _ supporting the design of safe sociotechnical systems in which humans are viewed as assets rather than the source of problems” (p3)

·????????SA is meaningful when considered from a systems perspective and why the system lost SA should be the explicit focus of investigations involving SA rather than why individuals lost SA and what “cognitive flaws” were involved

·????????They draw on a concept called distributed situation awareness (DSA). DSA argues that SA is an emergent property held by the overall system and built through interactions between human and non-human agents (tools, documents, displays etc.)

·????????SA isn’t held by one individual but is distributed across the sociotechnical system, in different forms, such as information held in one location that is needed elsewhere at a particular time

·????????Thus, DSA is “considered to be activated knowledge for a specific task within a system at a specific time by specific agents” (pp3-4), and SA is a dynamic phenomenon that changes moment by moment based on environmental interactions

·????????They describe this in the context of how systems operate akin to a network of information elements that are linked by salience, and activated by tasks belonging to different agents

·????????“Viewing the system as a whole, it does not matter if humans or technology own this information, just that the right information is activated and passed to the right agent at the right time” (p4), and a key facet of DSA are ‘transactions’

·????????A transaction is an exchange of SA between agents and is more than just communication

·????????They argue the finding that an operator had “poor situation awareness” has become almost meaningless since it fails to probe important system questions, like why wasn’t the operator aware of something so important? Why didn’t necessary transactions occur?

·????????A DSA approach leverages system networks (an example shown below). The network highlights transactions between agents and “provides a picture of the systems awareness at different points in time” and resultingly “determine who in the system had access to what knowledge at different points in time [and making it] possible to model the degradation of a system’s awareness in the lead up to the adverse event” (p5)

Note – I’ve skipped a lot of the specific findings from the Air France case.

Results

Overall, leveraging a DSA lens they argue that “it was the sociotechnical system comprising aircrew, cockpit and aeroplane systems that lost situation awareness, rather than the aircrew alone” (p1). One DSA network map is shown below.

Some critical facets is that spurious airspeed data were sent to the cockpit because the pitot tubes were frozen. Thus, this “transaction in awareness led to the autopilot disconnection and triggered the unfolding events. It is notable that this was a transaction between technological agents rather than human agents” (pp9-10).

They note that the network map also highlights the actual freezing of the pitot tubes and that the official report didn’t provide any information on whether the pilots were aware of this. There appears to have been no transaction that took place to alert the aircrew of the frozen pitot tube.

Several other transactional issues were evident between the Pilot Not Flying (PNF) and the Pilot Flying (PF). The PNF wasn’t aware of what control inputs the PF was making and wasn’t aware of the PF’s intentions around control inputs as they weren’t communicated. Further, the PF’s sidestick can’t be easily observed by the other pilot, and thus “control input information represents both a human-to-human transaction and a non-human-to-human transaction in awareness” (p10).

Based on their findings, they suggest four forms of transaction failure played a role. These are:

1) Absent transactions: instances where a transaction in awareness was required but not initiated

2) Inappropriate transactions: instances in awareness were initiated, but their content was incorrect, e.g. wrong info being provided or to the wrong location

3) Incomplete transactions: instances where appropriate transaction was initiated but the delivery incomplete and therefore not all required info was exchanged

4) Misunderstood transactions: instances where the receiver misunderstands the info or picture being transacted

The table below provides a range of different transaction failures from this example.

Next they answer “who lost situation awareness?”.

For most meaningful contexts, “situation awareness is not something that can be held by one individual alone, and therefore cannot be lost by one individual alone” (p12).

It was the entire sociotechnical system comprising the aircrew, cockpit and aeroplane systems that lost SA and this is evidenced by multiple failed transactions in awareness across both human and non-human agents (e.g. cockpit systems and displays).

They note that the initial transaction failure leading to the incident was “entirely between non-human agents (e.g. the pitot tubes and the cockpit systems” (p13).

Because of such, “it is inappropriate to point to a loss of awareness on behalf of the aircrew only” (p13).

They argue that as a result of DSA, countermeasures should focus on enhancing the transactions required during both routine and non-routine situations and between humans and non-human agents.

For instance, asking the questions are all important: what information is required, how best it can be communicated in high-workload situations, and who or what it should be communicated by.

Specifically in this example, critical info should be communicated between pilots both verbally as part of the aircrew’s threat and error management activities and also by the cockpit systems.

In sum, “The onus is not on human operators exchanging the wrong information or misunderstanding the information given to them. Rather, the onus is on the system and interactions between its components and so the issues can be associated with documents, displays, equipment, and the general work environment” and therefore from a DSA perspective, “Nothing is off limits” (p14).

Further, the focus isn’t on whether a person or agent was or wasn’t aware of something, but rather “what interaction between agents that led to the appropriate awareness not being distributed as it should have been” (p14).

Notably however, such DSA descriptions and network maps are more snapshots of the system awareness at particular moments rather than comprehensive descriptions of it.

In concluding, they argue that while SA is a key concept in safety science “it is not possible to improve situation awareness, performance, and ultimately safety by focusing on individual operators in the aftermath of adverse events” (p14).

Link in comments.

Authors: Salmon, P. M., Walker, G. H., & Stanton, N. A. (2016). Theoretical Issues in Ergonomics Science, 17(1), 64-79.