Did We Get Colgan Right?

Did we get Colgan right??

?

From a risk mitigation standpoint, the report recommendations and Congressional mandates that followed the investigation resulted in several significant improvements to commercial aviation safety, including FAR Part 117, focused and improved stall training, and captain’s leadership training. More ambiguously, the accident served as the catalyst for FAR 61.160, generally requiring 1500 total flight hours for Part 121 pilots, with several exceptions. The demonstrated safety improvement resulting from this requirement is difficult to measure, and even more challenging to normalize with respect to types of experience and quality of original training; ergo, the potential and realized benefits are difficult to quantify.

?

So, did we get it right? The question may seem rhetorical, insofar as much was accomplished and the safety record in the ensuing years has been exemplary. Nevertheless, a risk mitigation model, while adding value, is at risk of diluting emphasis on critical aspects, and quite possibly missing some of those completely. To explore that claim, it is worth revisiting a somewhat endemic theme that infused the final report, that of poor cockpit discipline.

?

In their final report, the NTSB emphasized the Colgan crew’s clear deviation from a sterile cockpit rule, going on to state that:

?

Because of their conversation, the flight crewmembers squandered time and their attention, which were limited resources that should have been used for attending to operational tasks, monitoring, maintaining situational awareness, managing possible threats, and preventing potential errors.

?

The word “squandered” is a pejorative, leading to a perception that both pilots were reckless and irresponsible. This theme emerged as the predominant narrative in the public understanding of the accident, embracing such things as the captain’s poor proficiency check performances, the first officer’s apparent illness, the haphazard management of crew rest prior to the flight and certainly their chattiness during what should have been sterile periods. All of these are important factors. That said, any use of such a pejorative highlights certain aspects while distracting from others. Moreover, it automatically isolates the humans involved into a separate class of people, distinct from how anyone reading the report perceives themselves.

?

It should be clear that the final three minutes of dialog captured by the cockpit voice recorder do indeed demonstrate a clear violation of sterile cockpit requirements. I can readily recall squashing such dialog on a number of my own flights, and in this case, neither pilot should have tolerated it. Nevertheless, this dialog is perhaps the most gut-wrenching output of any cockpit voice recorder that I have ever read or heard in my experience as an accident investigator. I still find it emotionally difficult to read. Why?

?

At 22:10:32, First Officer Rebecca Shaw said,

?

“oh yeah oh it's lots of ice.”

??????????????????????????????????????????????????????????

Eight seconds later, Captain Marvin Renslow replied,

?

“oh yeah that's the most I've seen— most ice I've seen on the leading edges in a long time. in a while anyway I should say.”

?

Ten seconds after that, First Officer Shaw went on to say,

?

“yeah that's another thing. all the guys— @ came in to our when we interviewed and he said oh yeah you'll all be upgraded in six months into the Saab and blah ba blah ba blah and I'm thinking you know what. flying in the northeast I've sixteen hundred hours. all of that in Phoenix how much time do you think actual I had or any in in ice. I had more actual time on my first day of IOE than I did in the sixteen hundred hours I had when I came here.”

?

At 22:12:05, she laid out, in comprehensive detail, her qualification at the time she was hired:?

?

“I've never seen icing conditions. I've never deiced. I've never seen any, I've never experienced any of that. I don't want to have to experience that and make those kinds of calls. you know I'dve freaked out. I'dve have like seen this much ice and thought oh my gosh we were going to crash.”

?

Twenty-eight seconds later, she very succinctly captured the problem, when she said

?

“but I'm glad to have seen oh— you know now I'm so much more comfortable with it all.”

?

At 22:13:01, Captain Renslow echoed First Officer Shaw’s recollection of her initial experience, when he said that

?

“but I— first couple of times I saw the amount of ice that that Saab would would pick up and keep on truckin'.”

?

Eight seconds later, he said:

?

“saw it out on the spinner. ice comin' out about that far my eyes about that big around. I'm going gosh. I mean Florida man— barely a little you know out of Pensacola.”

?

This entire exchange bears an eerie, and heartbreaking, similarity to the words of Captain Clarence Bates, the sole survivor of Northwest 5, a DC-3 crash at Moorhead, Minnesota in 1940 that claimed the lives of all thirteen passengers and his co-pilot. During the investigative interview, Captain Bates stated that “we did start to pick up quite a lot more ice”. However, the report went on to say that, “...having on previous flights experienced what he considered heavier ice accumulation, he still was not unduly concerned.”

?

Captain Bates’ aircraft stalled due to ice accretion when he levelled at the minimum descent altitude while executing a “look-see” approach. That he should have been concerned was rather obvious after the fact. The Colgan aircraft did not stall due to ice accretion, but the crew was concerned, which was the whole point of their dialog. That they invoked the same tired, seventy-plus-year-old myths to settle their nerves speaks volumes, loudly, about the quality of aeronautical knowledge that they had been trained with.?

?

From that perspective, I believe that the Board missed the point of the crew’s conversation. In fact, three minutes before they died, both pilots were not just discussing their perception of backfilling, with experience, the rather cavernous gaps in their knowledge of icing; they were circumscribing the normality of poor or nonexistent training and preparation throughout their careers. They did not really know very much about icing…both pretty clearly vocalized this… and I think that deep down, they had a sense that they were in over their head, rather suddenly with the detection of the “most ice I've seen on the leading edges in a long time”. Yet neither cited any particular training, knowledge or procedure appropriate to the situation, beyond that of operating the ice protection systems, which they were already doing. Instead, they were relying solely on the same interpretation of experience that Captain Bates had relied upon, and Captain Renshaw even went so far as to enunciate that interpretation. My belief is that while both of them wanted to feel “not unduly concerned”, they each felt some anxiety about the ice they had observed, which is what led to both making statements about how comfortable they thought they should be based on past experience. The captain amended his statement about “the most ice” by adding, “in a while anyway, I should say…”, which may have been a way of bringing an observation of what he believed to be considerable ice back into a realm of normality. Their “non-pertinent” conversation was really an effort to reassure themselves that they were within safe margins, when they both harbored a suspicion that they didn’t quite know where the margins were.?

In their final three minutes, Captain Renslow and First Officer Shaw left us with an authentic, honest and heartfelt summary of their anxieties, their quest for certainty and confidence, and in particular, the almost universal acceptance of a paradigm in which absent knowledge is colored in by the misinterpretation of experience…exactly as had been the predominant paradigm of aeronautical knowledge in Clarence Bates’ day. And then…within the blink of an eye…that state of knowledge killed them, and all of the people whose lives they were responsible for, exactly as it has everyone aboard the DC-3 that Bates was piloting.

?

This raises the fundamental question, a question largely unanswered in the report and in the subsequent industry activity, a question that should be deeply troubling to us all: how did that paradigm, and that state of aeronautical knowledge, persist over so many decades? How has it persisted throughout the advent of simulator training, evidence-based training, advanced qualification programs, and most astonishingly, throughout the advances in the understanding of human factors, both in the cockpit and in the classroom, that have taken place in the same period?

?

To fully understand the Colgan accident, it is necessary to break it into two distinct segments: what happened before the stick shaker, and what happened after the stick shaker.

?

Before the Stick Shaker

The operation of the ice reference switch introduced a scheduled increase in both the stick shaker and stick pusher activation speeds. To obtain the correct approach reference speed with the ice reference switch in the “on” position required that the word “icing”, or the word “eice”, be manually typed into the ACARS request for landing data. The crew apparently failed to do this, although it is impossible to know what they may have typed when making the request; at the time, the ACARS software had no error capture feature that would flag a meaningless entry, so a single incorrect keyboard entry would default the request to the clean, dry wing reference speed. And those clean wing speeds were the ones they accepted, whether they asked for them or not.

?

At 21:53:40, First Officer Shaw briefed Captain Renslow on the reference speeds as received via ACARS. At 22:05:29, Captain Renslow briefed the approach and included the same clean wing reference speeds. Further, they re-checked the airspeed bugs during the descent checklist at 22:13:42. This means that on three separate occasions, two crewmembers failed to detect a ref speed that was significantly lower than what should be expected with the ice reference switch selected on. We might think of this as six “person-opportunities” to capture this error, and yet no one did.

?

We know, from the interviews conducted with a crew that had also experienced a stick shaker while on approach to Burlington, Vermont, that there was considerable confusion over how to use the different ref speeds, and when it might be appropriate to deselect the ice reference switch and consequently re-schedule the stall warning speeds. Yet none of that debate occurred in the cockpit while approaching Buffalo. It is not unreasonable to consider that all of the distraction created by cockpit chatter, or perhaps the lack of focus created by fatigue, might cause one or two oversights of such an item. It seems much more unlikely that all six opportunities to capture this error were missed in this way. In my mind, the more likely conclusion is that neither crewmember understood the change in the scheduling of stall warning speeds that occurred when the ice reference switch was selected on. It seems likely that they were either unaware of this feature, or at a minimum, had so little experience using it that they saw nothing odd about the reference speeds they received when they requested landing data through ACARS.

?

And why would they? The Board’s report stated that:

?

Colgan’s training manual, which contained the syllabus for Q400 upgrade and transition training, did not specifically mention the ref speeds switch in ground school subject matter sections, including the one on ice and rain protection. Also, the simulator training modules described in the manual made no direct reference to the use of the ref speeds switch.

?

The Board further acknowledged this issue, stating that

?

Colgan also lacked standardized procedures for setting airspeeds and using the ref speeds switch, which did not promote effective cross-checking between airspeeds and the switch’s status. If such procedures had been in place, then the flight crew might have detected the inconsistency between the 118-knot Vref (a non-icing speed) and the position of the ref speeds switch (icing conditions assumed) and ensured that a Vref of 138 knots (an icing speed) was selected.?

Of all the various disparate factors which the Board identified, this is the one that really matters. Had they not “squandered” their time and attention, and instead attended to “operational tasks, monitoring, maintaining situational awareness, managing possible threats, and preventing potential errors”, then one has to ask: in the absence of effective training and standard procedures, what exactly they would have done differently? If they had understood the operation of the ice reference switch, and obtained the correct ref speeds, there would have been no accident, or at least not the one that happened. If they did not understand the operation of the ice reference switch, no amount of rest or cockpit sterility would have prevented their encounter with the stick shaker.?

?

This error sets up the scenario. The next question is, why did they not see the myriad indications that the decaying airspeed was approaching the scheduled stall warning speed? There were, after all, several pretty unambiguous indications of impending disaster.

?

In Finding No. 8 of the report, the Board stated that:

?

Explicit cues associated with the impending stick shaker onset, including the decreasing margin between indicated airspeed and the low-speed cue, the airspeed trend vector pointing downward into the low-speed cue, the changing color of the numbers on the airplane’s indicated airspeed display, and the airplane’s excessive nose-up pitch attitude, were presented on the flight instruments with adequate time for the pilots to initiate corrective action, but neither pilot responded to the presence of these cues.

?

One could argue that pretty much any unintentional 1G aerodynamic stall is the result of undetected airspeed decay. The Board was perplexed by the failure of several cues designed specifically to trigger a crew response. They have encountered this question before, in the very accident that they referenced for similarity to the Colgan case, that of United Express 6291 at Columbus, Ohio in 1994. In that report, the Board dispensed with any concerns regarding the transition from steam gauges to an EFIS with the statement,

?

Pilot acceptance of the displays has generally been favorable, and, more important, they have not been suggested as contributory to accidents. Moreover, their presentation format across aircraft types has generally been consistent with human factors principles of presenting visual information. For example, in the J-4100, the moving vertical display presents trend information, and, as the airspeed approaches the stall speed, the color of the display changes to red, the common color of warning. Therefore, because the airspeed display on the J-4100 was consistent with these principles, the Safety Board does not consider their format or mode of presentation to be a factor in this accident.

?

In the United Express report, there doesn’t seem to be any interest in questioning the human factors principles as they were understood in 1994. Nevertheless, in the statement of probable cause, the Board included the following:

?

Flightcrew inexperience in "glass cockpit" automated aircraft, aircraft type, and in seat position, a situation exacerbated by a side letter of agreement between the company and its pilots.

?

In the Colgan report, the Board discussed at length the lack of effective monitoring skills, but made no reference to the possibility that the substantially different displays incorporated in the EFIS might have had a role in the failure to detect the airspeed decay.

?

In their submission to the NTSB, the Air Line Pilots Association did question those human factors principles, noting that:

?

EFIS and the PFD is supposed to reduce the risks of visual illusions and spatial disorientation during instrument flight conditions by supplementing normal visual cues, but the tape system currently used is not intuitive. There is very little positional movement of the display due to the nature of the design, unlike a conventional analog airspeed indicator where experienced pilots can build appropriate mental models of airspeed based on position of the airspeed needle alone. The PFD airspeed tape appears static during the conventional instrument scan, and movement is not noticed unless focused attention is placed on the display, which is not a good information retrieval strategy when managing concurrent tasks during periods of high workload. Compounding this problem is the fact that the airspeed tape and altitude tape represent information in a radically different manner. If the airspeed tape is rising, airspeed is decreasing and a correction could require pitch down. Conversely, if the altitude tape is rising, the aircraft is descending and a correction would require pitch up. While this may be satisfactory to a pilot placed in an automation monitoring role, it is not intuitive to a pilot with who has predominant experience with conventional instrument displays and thus has that “embedded” in his or her mental model of the instrument scan.

It is actually not appropriate to any pilot who is attempting to extract information and use it immediately, in a very, very high gain environment, to make complicated and timed control inputs.

?

In fact, lying behind the issues of effective monitoring, and human factors principles embedded within display design, is the question of whether a pilot has developed an effective instrument scan in the first place. The FAA’s latest icing training video describes a King Air crew, using a conventional airspeed display, who managed to decelerate from 180 knots to 115 knots…without noticing. What is fairly obvious is that, regardless of display design, pilots involved in unintentional 1G stalls almost certainly had a deficient instrument scan, either consistently or at that moment.?

?

A deficient instrument scan, of course, leads directly into a discussion of automation dependence.?

?

After the Stick Shaker

The Board accurately summarized the captain’s actions following the onset of the stick shaker and subsequent stick pusher by stating that:

?

The NTSB is concerned that the captain pulled against the stick pusher three separate times during the stall event and that his control inputs fought the stall protection system’s attempts to decrease the AOA and reduce the severity of the situation.

There can be no doubt that Captain Renslow’s response to the stick shaker, and then the pusher, was inappropriate. A lot of debate was had over why he responded this way. The Board identified several cases in which the pilot of a stalled jet transport continued to apply back pressure on the flight controls.? One aspect that was discussed in the report is the notion of minimizing altitude loss during stall recovery. The Board noted that:

During postaccident interviews, the NTSB learned that, during the approach-to-stall recovery exercises for initial simulator training, pilots were instructed to maintain the assigned altitude and complete the recovery without deviating more than 100 feet above or below the assigned altitude, which had been previously required by the practical test standards for the checkride. Some company check airmen indicated that any deviation outside of that limit would result in a failed checkride, but other company check airmen considered this altitude limitation to be a minimal loss of altitude (which is consistent with the current practical test standards).?

In their submission to the Board, The Air Line Pilots Association noted that:

The FAA ATP PTS required that the applicant should recover with a minimal loss of altitude. The Colgan Stall Profiles, however, required a pilot to maintain altitude. In addition, Colgan Check Airmen were not evaluating to the FAA ATP PTS. Three Colgan check stated that the PTS standard for recovery during approaches to stalls was ±100’.?

The notion of minimizing altitude loss during stall recovery has been prevalent in flight training for decades. It appears in various forms throughout the history of powered flight, almost certainly due to stall/spin accidents occurring close to the ground. In his description of events following the crash of Northwest 5, Captain Bates had said that “the airplane started to act peculiarly and I knew something was the matter…I yelled “gear up!”…I increased to full horsepower to fly straight ahead at 1500 feet until I could find out what was the matter”.

Early flight training literature often emphasized the ability of the airplane to recover itself as long as the back pressure required to initiate a stall demonstration was relaxed. Civil Aeronautics Bulletin No. 23, “Fundamentals for Elementary Flight Maneuvers”, from June of 1943, states that:?

Always it [the stall] results from too much back pressure on the stick, and always an approaching stall can be stopped by releasing the back pressure.

The same year, the United States Army Air Corps stated that

Airplanes with high wing loadings require a great amount of altitude to recover from a stall. To correct stall, reduce the angle of attack and get flying speed by diving. The most dangerous feature is that so much altitude must be lost to regain control.

By the early 1960’s, the Federal Aviation Agency stated, in their “Pilot Instruction Manual”, that:

First, at the indication of a stall the nose is lowered positively and immediately. The amount of control pressure used depends on the design of the airplane and the severity of the stall. In some planes a moderate action of the control column – perhaps slightly forward of neutral – is enough, while in others a forcible shove is required.

However, the Manual goes on to state that

The recovery should be planned to produce a safe recovery with the least expenditure of altitude. Diving steeply in a stall recovery will hasten the recovery from the stall, but will cause a greater loss of altitude, which might be critical in an emergency recovery from an inadvertent stall near the ground.

Coincident with the Pilot Instruction Manual, in 1959 the FAA standard for Airline Transport Pilot demonstration of stall recovery stated that the applicant's performance:

?

... will be judged on ability to recognize the approach to stall, prompt action in initiating recovery, and the holding of heading and smooth recovery with the minimum loss of altitude.?

?

In 1974, the standard stated that the pilot applicant:

?

...shall accomplish recoveries positively and smoothly, using appropriate and coordinated flight and power controls and with the least loss of altitude consistent with the recovery of full control effectiveness.

?

With all of this history in mind, it is difficult to know where the one-hundred-foot altitude loss idea came from. However, in their submission to the Board during the 1994 Simmons 4184 investigation, ALPA noted that this standard had also been in use at Simmons Airlines.

?

The legacy of these criteria, and its effect on actual stall recoveries, should not be underestimated. In a report which I authored, “The Icemaster Database and an Analysis of Aircraft Aerodynamic Icing Accidents and Incidents” (DOT/FAA/TC-14/44), I noted the following:

?

Of the 308 events in the dataset, 163 involved a stall. In 22 cases involving a stall, the pilot applied a pitch-up control input in response to a stall manifested by either an uncommanded pitch down or an uncommanded roll. In 16 cases, there was clear evidence of a correct pitch-down recovery response, although nine of these events did not involve an actual stall, but only a stall warning. In 68 cases, there were no survivors or evidence from which to draw a conclusion regarding the stall recovery procedure. In the remaining 70 stall events, the data were insufficient to draw any conclusions regarding stall recovery procedures.?

It is fair to say that, from this standpoint, the captain’s initial response was not unique. However, his repeated application of back pressure may suggest a different perception.

The Board noted that Colgan Q400 pilots had been shown a NASA-produced video titled “Icing for Regional and Corporate Pilots,” during winter operations training in initial, transition, and recurrent ground school. This video contained extensive discussion of both conventional icing stalls and tailplane icing stalls. The Board said that:

Postaccident interviews with Colgan pilots about tailplane stalls produced varying responses. One captain stated that the video about tailplane icing made a big impression on him, and another captain stated that the video got his attention. Some pilots indicated that they would apply the tailplane stall procedure if they had clearly identified the symptoms of a tailplane stall, whereas other pilots stated that it would be difficult to determine if the airplane was in a conventional wing stall or a tailplane stall. Some pilots thought that the Q400 might be susceptible to a tailplane stall, some pilots were not sure about the airplane’s susceptibility, and one pilot (a check airman) stated that the possibility of a tailplane stall in the Q400 had “never crossed [his] mind.”?

The Board later concluded that it was unlikely that the captain had perceived a tailplane stall, based on this analysis:

However, the video also stated that tailplane icing symptoms were lightening of the controls, pitch excursions, difficulty in pitch trim, control buffeting, and sudden nose-down pitch, none of which occurred before the stick shaker activation during the accident flight. Further, the activation of the stick shaker was a clear warning of an impending conventional aerodynamic stall and not a tailplane stall, and the change in the IAS numbers to red (which occurred after the airspeed was equal to or below that of the low-speed cue) was a conspicuous signal that was not consistent with a tailplane stall. Also, indications of a tailplane stall, for those airplanes determined to be susceptible, would likely occur at higher airspeeds and/or higher flap deflections.?

For a tailplane stall recovery, the captain would have had to interpret the situation, identify the tailplane stall, and apply a recovery procedure that he had never practiced. The CVR showed that he did not verbally identify a tailplane stall, and the FDR showed that he did not fully apply the tailplane stall recovery procedure described in the video. The NTSB concludes that it is unlikely that the captain was deliberately attempting to perform a tailplane stall recovery.?

This is grossly misleading. As was noted in the video, the buffeting associated with a tailplane stall is control buffeting, not airframe buffeting, and the activation of a stick shaker could be confused with control buffet, particularly if you are anxious about icing and have been thinking about it for the preceding several minutes. In a 1998 article for Airline Pilot magazine entitled “The Handling Event”, I explained that the activation of a stick shaker can be misinterpreted as the control buffeting that may be encountered during a tailplane stall. Furthermore, the nose down pitch caused by a stick pusher can be misinterpreted as the elevator snatch associated with a tailplane stall. The NASA video was also explicit in describing these threats, as well as the critical importance of correctly differentiating them. The video further pointed out that such symptoms as control lightening and difficulty trimming can be masked by the autopilot, which in this case was engaged until the shaker activated. Arguing that the “activation of the stick shaker was a clear warning of an impending conventional aerodynamic stall and not a tailplane stall” is an armchair exercise that presupposes perfect understanding of both phenomena under minimal stress.

The Board drew an identical conclusion during the aforementioned investigation of the United Express 6291 accident. In that report, they discounted the captain’s substantial and recent experience with the Jetstream 3100, an aircraft that did indeed have a significant tailplane stall problem in icing conditions. The Board stated that:

However, the Safety Board discounted tailplane stall due to ice accretion, and the captain's actions as being related to an attempt to recover from tailplane stall, for several reasons. The J-4101 horizontal stabilizer is designed with negative camber on the upper surface to reduce the effects of ice accretion. In addition, the boots have been extended farther back on the horizontal stabilizer to ensure that any potential runback of ice can adequately be removed. Further, tailplane stall occurs as a result of a high speed with flaps extended rather than at the lower speed at which the stick shaker actuates. Additionally, the proper procedure to recover from tailplane stall in the J-3100 and J-3200 was to add power and retract the flaps to the mid-range position. If the captain had perceived, in error, a tailplane stall condition due to icing, the reduction of the flap setting to a lower angle would have been appropriate. However, the proper flap callout should have been "flaps 9 degrees," rather than the call for "flaps up." Finally, the airplane's pitch attitude time history obtained from FDR data was inconsistent with a tailplane stall caused by ice. Consequently, the Safety Board concludes that the Captain's actions were not in response to recovering from a perceived tailplane stall. The Safety Board was unable to determine why the captain called for flaps up.

This explains why the Jetstream 41 did not have a tailplane stall problem; it does little to nothing to explain why that could not have been the captain’s perception.?

We know that the Q400 is likewise not susceptible to ice-contaminated tailplane stall; it has a huge stall margin built into the design of the horizontal stabilizer. We also know that many Colgan pilots operating the Q400 were not aware of this. We know that exposure to the stick pusher in the simulator was not required, and only generally took place at the request of the student. Given Captain Renslow’s possible lack of exposure to the stick pusher, and his specific exposure to the NASA video discussion of tailplane stall, his awareness of what he perceived as significant ice accretion, and factoring in the flap extension that immediately preceded the event, it is highly plausible that he interpreted the stick shaker as the control buffet that may precede a tailplane stall.?

We also know that, in those cases in which the trainee was exposed to the stick pusher in the simulator, the typical response was to resist it by applying back pressure to the controls. This is consistent with the findings I reported in the Icemaster paper. In conjunction with the unjustified emphasis on minimizing altitude loss following a conventional stall, the tendency to apply back pressure in response to the stick pusher suggests a very plausible scenario in which Captain Renslow did not interpret the stick shaker and pusher as manifestations of a tailplane stall. We will never know.?

?

However, the fact that he did not enunciate that he believed they were encountering a tailplane stall is meaningless. The fact that he did not execute the correct, complete procedure is meaningless. We have no idea what he was thinking, and we have no idea what he had retained from his training and his exposure to the NASA video. We cannot read his final thoughts. The best that can be said is that we simply don’t know.

?

Likewise, it is impossible to know whether the United Express captain believed he was encountering tailplane stall or wing stall, and whether his response was consistent with what he recalled in that moment of the tailplane stall recovery procedures, or with the same issue of minimizing altitude loss following a wing stall.

?

The tailplane stall scenario may also explain First Officer Shaw’s decision to retract the flaps without a command from the captain. Given that she subsequently asked whether she should retract the landing gear, it seems far more likely that she was following her perception of the stall recovery procedure, or possibly just a go-around procedure. The Board considered several of these possibilities, could draw no conclusion, but still made sure to point out that whatever she did, it was non-compliant with procedure.?

Similarly, the Board discounted the United Express captain’s decision to retract the flaps completely as a response to a perception of tailplane stall, simply on the basis that the procedurally correct choice for a tailplane stall recovery would have been to retract the flaps to nine degrees, not zero degrees. In fact, the captain commanded flaps up at the initial activation of the stick shaker, then belayed that command, telling the first officer to “no, no hold it” when the stick shaker stopped. When it re-activated, he then commanded the first officer to retract the flaps. This is pretty powerful evidence that he was trying to sort out what he was encountering, and that it was not obvious to him. The Board simply dismissed this by stating that “The captain was obviously confused by the stick shaker and autopilot warnings.” In fact, the initial command, reversal of that command, and finally reversal again suggest some type of evaluation process taking place in the captain’s mind, with the final choice of action bearing no resemblance whatsoever to a conventional stall recovery. There was much more going on than simply startled confusion.

On April 3, 1990, a Mid-Pacific Airlines YS-11 freighter did experience a tailplane stall while on approach to West Lafayette, Indiana. In that case, the first officer also retracted the flaps without a command from the captain…and most probably saved the aircraft and crew. His actions were also non-compliant. As with Captain Renslow, or the United Express captain, we will never know what First Officer Shaw was thinking. The best that can be said is that we don’t know, but it is obvious that she was trying to do whatever she thought was right.

In the final analysis, it is clear that regardless of what they believed was happening, none of these pilots were properly prepared, and trained, to resolve the situations they encountered. How did that happen?

The Congressional Mandates

In the aftermath, Congress mandated that the FAA initiate two projects. The first, defined in Section 204 of the Airline Safety and Federal Aviation Administration Extension Act Of 2010, was known as the FAA Task Force on Air Carrier Safety and Pilot Training. This group was charged with:

evaluating best practices in the air carrier industry and providing recommendations in the following areas:?

(1) Air carrier management responsibilities for flight crew- member education and support.?

(2) Flight crewmember professional standards.?

(3) Flight crewmember training standards and performance.?

(4) Mentoring and information sharing between air carriers.?

?

The final report of this group, issued on July 31, 2011, contained twenty-four specific recommendations. Probably the most significant, relative to raw technical flying skills, was contained within paragraph 3.3.4, Maintain Manual Flying Skills. Implicit in this recommendation, albeit unmentioned, is the relationship between manual flying skills and an effective instrument scan. This recommendation has been encoded into the extended envelope training now required under FAR 121.423.

While the report contained a number of useful recommendations, it also fell into the trap of utilizing extensive boilerplate language, as in the following three excerpts:

…Instructors and evaluators who rate personnel must undergo reliability training to learn to utilize explicit strategies to verify the proficiency and standardization for crew-oriented, scenario-based training and evaluation tasks…

…To adequately train and evaluate other pilots, instructors and evaluators must achieve and maintain the highest levels of proficiency and knowledge as pilots…?

…An important element in reducing risks in air carrier operations is a well-coordinated flightcrew that expertly uses the principles of CRM and actively engages in TEM. Most recent accidents could have been avoided had flightcrews effectively used these skills…?

Phrases such as “reliability training” and “explicit strategies” suggest the existence of qualified definitions, but those definitions are nowhere to be found. There is no reference made to what is meant by such phrases as “highest levels of proficiency and knowledge”, or what degree of aeronautical and specific aircraft knowledge might be necessary to “expertly use” the principles of CRM, or “actively engage” in TEM. Indeed, the statement that “most recent accidents could have been avoided had the crew effectively used these skills” overlooks the significant role often played in these cases by both deficient aeronautical and deficient aircraft-specific technical knowledge.?

More interesting are the various ways that the word “proficiency” is used, and these reflect standard boilerplate usages within the industry. On one hand, the statement is made that “As with any complex task, proficiency is at its highest when the skills are exercised regularly, and proficiency can diminish as time passes without exercising those skills.” On the other hand, several references are made stating that pilots should be trained “to proficiency”, which, if proficiency reflects regular exercise of a skill, raises the question of how many repetitions of any maneuver, such as a stall recovery, are necessary to first establish proficiency before it can be reliably demonstrated. For example, is it possible for a pilot to demonstrate proficiency after only two repetitions of a maneuver? Is that proficiency with the maneuver, or proficiency with the curriculum?

?

In another usage, the report describes one advantage of Advanced Qualification Programs (AQP) with this statement:

?

For example, data may indicate that pilots demonstrated proficiency during recurrent training and in line operations in?a particular maneuver that is currently required under part 121 training. In that case, AQP would provide the air carrier the ability to modify its training program, refocus the curriculum, and address a different maneuver that data shows may need additional emphasis.?

?

This raises the question of how the originally demonstrated proficiency is maintained, and evaluated periodically, if optimal proficiency is achieved through regular exercise of a particular skill, but that skill can be removed from the training curriculum until data indicate that proficiency has diminished.

?

At no point is the actual meaning of the word “proficiency” discussed. It is left as an open-ended concept, subject to a wide range of interpretations which can alter the significance of the entire report.

?

Perhaps the most revealing language is contained in section 3.3.3, Implement Integrated Training. This section contains the paragraph:

?

For example, to teach the operation of the aircraft electrical system effectively, some classroom time would be dedicated to studying the basic components and characteristics. For maximum effectiveness, this relatively abstract discussion of theory would immediately be followed by graphic depictions of the entire system in the specific context of the aircraft, then by practical demonstrations using advanced training devices. Such advanced exposure shows the student the real-world application of the academic presentation, illustrating how specific controls affect the operation of the components and overall system, how normal operations are conducted, and how troubleshooting procedures are developed and implemented.?

The operative phrase within this paragraph is “this relatively abstract discussion of theory”. It is a remarkable echo of another phrase used in a 1939 Civil Aeronautics Authority report examining the breathing patterns of candidates for pilot training. The report characterized the ideal candidate as one possessing “such features in their daily lives as simultaneous interests in a number of business enterprises, frequent entertainment, lack of interest in abstract thought, hearty dispositions and carefree attitudes”. A “relatively abstract discussion of theory” reflects the “lack of interest in abstract thought”, which leads directly to the problem of declarative knowledge and its relationship with procedural knowledge.

Simon Wood, of Cranfield University, carefully differentiated these two ideas, describing declarative knowledge as “the knowledge that the system works in a certain way.”, and procedural knowledge as “knowing how to use the system in context.” The reference to “abstract discussion of theory” as subordinate to “practical demonstrations using advanced training devices” reflects the diminishment of declarative knowledge in favor of procedural knowledge, and this goes straight to the heart of the problem. A practical demonstration requires a particular context; while knowing how to use the system in that context is essential, it does nothing to prepare the pilot for what happens when the context suddenly changes. At that point, the abstract discussion of theory becomes extremely important, regardless of whether the pilot has a hearty disposition or a carefree attitude. A practical demonstration refers to training; an abstract discussion refers to education.

In other words, you actually have to know, not only how the ice reference switch affects stall speed scheduling, but why. You have to understand the abstract relationship between angle of attack, ice, and drag. You most definitely have to understand the abstract behavior of the lift coefficient in response to ice shape and roughness, a relationship that will quickly disabuse you of any certainty that your airplane can pick up ice and “keep on truckin’…”.

In the end, the report essentially fails to capture the fundamental, foundational deficiency that lies at the heart of the problem: the absence of a comprehensive, thorough, well-executed and costly optimization of educated training. This kind of training cannot subordinate the abstract, and the role of education in such training is to ensure that the abstract is accessible. While practical demonstrations will always be limited to a particular context, the proper use of the abstract discussion of theory will provide the basis from which to adapt the practical demonstrations to unexpected contexts.?

?

This kind of training must be delivered by instructors who are qualified well beyond amorphous phrases such as “achieve and maintain the highest levels of proficiency and knowledge as pilots”, or “achieve and maintain a high level of standardization”. Rather, at a minimum, it requires instructors who are technically well educated, not only in aircraft systems, but in design and certification objectives; not only in procedures, but in the purpose and history of procedures and their evolution and development; and most certainly in a comprehensive range of aeronautical knowledge. Examples of such aeronautical knowledge would include the effects of ice on all aerodynamic surfaces, the relationship of airspeed to aerodynamic forces generated by a trimmable stabilizer, or the interpretation of different weather radar displays. These people must be instinctive communicators, or perhaps more explicitly, instinctive teachers, in order to ensure that relatively abstract discussions of theory prepare the pilot for adaptation to unexpected contexts.

?

The second project mandated by Congress required that the FAA:?

...convene a multidisciplinary panel of specialists in aircraft operations, flight crewmember training, human factors, and aviation safety to study and submit to the Administrator a report on methods to increase the familiarity of flight crew- members with, and improve the response of flight crewmembers to, stick pusher systems, icing conditions, and microburst and windshear weather events.

This group issued their final report on July 6, 2011, entitled “Stick Pusher and Adverse Weather Event Training, Aviation Rulemaking Committee Final Report and Recommendations”. As chartered, this group focused on training for operations in inflight icing, encounters with microbursts and windshear, and training for stall recognition and recovery. The group made several very well-researched and specific recommendations, in particular recommendations to dispense with altitude-loss constraints when evaluating stall recovery, and a requirement for comprehensive training regarding a stick pusher system if installed.

?

I could not agree more with the emphasis on stick pusher systems. Back in my days as an instructor on Fairchild Metroliners, I made it a point to demonstrate the stick pusher system. We did not have a simulator, and accomplished training in the airplane. Before beginning work with stalls and stall recovery based on the earliest recognition…buffet or horn…we would take a couple of stall procedures through the horn to the stick pusher activation, so that the trainee had experienced what it felt like and how it functioned. It is inconceivable to me that this demonstration was not a routine part of simulator training, which is one recommendation made by the ARC Final Report.

?

Yet this report also makes the statement that “No evidence was found that any stick pusher system installed in transport category airplanes has a history of improper activation.” Herein lies one of the problems with the industry’s fascination with data-driven analysis…which data are we using, and what is the quality of that data? The Metroliner very much did have a history of improper stick pusher activation, and this was a central point in the investigation of the AVAir 3378 accident at Raleigh-Durham on February 19, 1988. The SAS clutch switch was found in the OFF position, and the SAS Fault light had been illuminated at impact.

The Metroliner was not a transport-category airplane, but it was very, very close in sophistication, and later versions were certificated under SFAR 41, the bridge across the gap between small airplanes and transport category airplanes. And one of the key issues emerging from the AVAir accident was also not identified by the working group, that of differentiating between a runaway stabilizer and a stick pusher. We never did answer these questions definitively in that accident investigation, primarily due to the absence of both a CVR and a DFDR, but also due to the substantial differences between the NTSB Systems Group chairman’s documentation of the pitch trim position as substantially nose down, and the manufacturer’s insistence that it was nominal.

In fact, a runaway stabilizer is probably far more likely than any kind of stick pusher malfunction, but we have generally placed a very low emphasis on runaway stabilizers in recent years. A spate of such events on the MD-80 caused the scenario to be re-emphasized in at least one operator’s recurrent training, but that never crossed over to the same operator’s Boeing 737 fleet until the Max accidents revealed that literally nobody had been discussing this malfunction beyond a cursory review of the QRH procedure for a very long time. The similarity between the two types of events, while not recognized in the working group’s 2011 report, now becomes front-and-center as we integrate the ARC working group’s recommendations with the more recent re-emphasis on runaway stabilizer training. Frankly, these issues always should have been front-and-center in the first place…but that really depends on what data we are using, what the quality of that data is, and how we interpret it.

These questions lead to the underlying issue that should have been evident following the accident, the very one the Colgan pilots were attempting to reassure themselves about in the final moments of their lives. How do we arrive at any of the assumptions and presuppositions that inevitably drive the content and quality of aeronautical knowledge?

The ARC working group developed a very useful and comprehensive set of matrices that outline areas of training content. Much of this content falls under the heading of “Knowledge-Based Training”, which could be interpreted as some of the abstract theory referred to in the Air Carrier Safety and Pilot Training report discussed earlier.? To the extent these matrices go…and that is much further than previous guidance…they are robust and meaningful. But buried within, the questions about assumptions and presuppositions are still not completely answered, and in this manner, the report falls perilously close to the failings of the first report. In the matrix for icing training elements, under the heading “General icing and meteorological information”, is the knowledge-based training category of “Aerodynamic Effects”. The same category is reflected under the heading “Type-Specific Operations”, in that case intended for both knowledge-based and flight or simulator-based training.

What are the aerodynamic effects of icing? Does that refer to the ubiquitous FAA diagram depicting loss of thrust, loss of lift, increased drag and increased weight as cumulative, thus strongly invoking a simple notion of linearity when the threat is decidedly nonlinear…a diagram that can be dated back the origins of stall recoveries graded by the least loss of altitude? Or does it include discussion of the rapid increase in drag that results when the angle of attack is increased, when ice has already been accreted at the lower angle of attack? This drag rise can be substantially greater than the drag induced by ice accreted solely at the higher angle of attack, and this effect, almost unknown outside of research, probably explains why a significant percentage of icing accidents are identified as hard landings.

Where did Captain Renslow get the idea regarding “the amount of ice that that Saab would would pick up and keep on truckin…”? Is that a reflection of the cumulative idea, the same idea that Captain Bates was referring to when the report of the Northwest 5 accident stated that “having on previous flights experienced what he considered heavier ice accumulation, he still was not unduly concerned”? Would that be the same idea that is encoded in the ever-present, but largely meaningless, icing severity index published in the current Aeronautical Information Manual, the very one the report matrix identifies as the “Definition of Icing Levels”??

?

These reports have, in fact, generated considerable improvement in many areas. After the Aviation Rulemaking Advisory Committee issued its recommendations for revising the practical test standards used for issuance of an Airline Transport Pilot rating, the FAA first published SAFO 10012, which began with a discussion of what the FAA apparently always meant by the term “minimal loss of altitude” in the Practical Test Standards. The SAFO went on to state that:

?

Operators and Training Centers are encouraged to ensure that their training program and checking modules are written and administered to ensure the evaluation criteria for a recovery from a stall or approach to stall does not mandate a predetermined value for altitude loss.?

Subsequently, the FAA published Advisory Circular 120-109, which stated that:

Additionally, recovery profiles that emphasize zero or minimal altitude loss and the immediate advancement of maximum thrust have been eliminated.?

Most recently, the FAA published FAA-S-ACS-11, Airline Transport Pilot and Type Rating for Airplane - Airman Certification Standards, in 2019. The stall recovery criteria have been significantly changed to read:

?

Evaluation criteria for a recovery from an impending stall must not mandate a predetermined value for altitude loss and must not mandate maintaining altitude during recovery.?

?

For its era, the investigation into the Northwest 5 accident at Moorhead was extensive, and involved significant flight testing of the DC-3 stall characteristics. The airplane had been believed to have relatively benign stall characteristics beforehand; however, the flight testing demonstrated that the application of full power during the stall, while attempting to maintain attitude as Captain Bates had done, resulted in a significant worsening of the stall and quite challenging aircraft behavior. The report noted that, as the result of the flight tests, and regardless of ice or no ice, that:

?

To regain normal flight, it was found necessary to push the control column forward enough to change the attitude of the airplane with respect to the horizon by approximately 4 to 5 degrees.

?

Nevertheless, for the next 70 years, the industry insisted on minimal altitude loss during stall recovery, with only peripheral discussion of the essential requirement to reduce angle of attack. This is precisely the result of a practical demonstration in context subordinating abstract discussion of theory.

?

The final report of the Northwest 5 accident also highlighted the profound dichotomy between the threat posed by icing and the powerful desire to operate in its presence. The report stated that:

?

A careful consideration of the evidence has satisfied us that the partial loss of control was not caused solely by the ice which had been accumulated on the airplane A collection of ice upon airplane surfaces is not an uncommon common experience and, while it as to be avoided to the fullest extent possible by the exercise of great caution, in the nature of things at cannot be eliminated entirely. Although the amount of ice which had been accumulated on the airplane was substantial, experience has demonstrated that aircraft may safely be flown with a far greater accumulation of ice than that which obtained in this case…

?

This was exactly the sentiment Captain Renslow expressed nearly 70 years later. And yet the report went on to chastise Captain Bates, stating that:

?

The action of Captain Bates in continuing his descent over Fargo after encountering more severe icing conditions than had existed at cruising altitude was not consistent with good operating practices.

?

How do we get from a cumulative diagram of icing effects, and the attendant presuppositions about how much ice can be safely flown with, to an understanding of the differences in drag profiles based on the angle of attack at which ice was accreted, or to the significance of a few thousandths of an ice of ice roughness on a leading edge during takeoff?

?

How do we get from “I'dve have like seen this much ice and thought oh my gosh we were going to crash” to “you know now I'm so much more comfortable with it all” without any discussion of how the stall speed schedule is changed by the ice reference switch, why it is changed, and what its role is in protecting angle of attack with ice accretion?

?

How do we get from “The recovery should be planned to produce a safe recovery with the least expenditure of altitude…” to the critical understanding of a positive, often “forcible shove” necessary to reduce the angle of attack, particularly when almost no inadvertent stall includes any back pressure on the controls at all?

?

Quite frankly, the frequent deployment of the word “proficiency” is not even close to adequate. Nor do the minimum flight time requirements of FAR 61.160 do anything to address these deficiencies. The only viable answer to these questions is a redesigned emphasis on abstract theory, regardless of whether the pilot has a hearty disposition or a carefree attitude, and regardless of total flight time. Reaching back to the notion of a comprehensive, thorough, well-executed and costly optimization of educated training, it is not enough to simply demonstrate the stick shaker and pusher in the context of an approach to stall. The pilot must understand the critical role of reducing angle of attack, by whatever means are necessary, in stall recovery. The pilot must understand how the stick shaker works and what criteria is required to trigger its operation, for example, the role of the angle-of-attack vane, and the rescheduling of the stick shaker thresholds based on ice protection system operation. The pilot must likewise understand why a stick pusher has been installed, if and when it is appropriate to override or disable it, and why and when it is not. The pilot must be thoroughly familiar with the effects of ice accretion on the lift curve, and how it can affect drag across a range of angles-of-attack, as well as control surface disturbances due to ice accretion in those designs which are vulnerable to such disruptions. This knowledge can and should transcend the limitations of simulator fidelity. In all likelihood, it will instill a career-long skepticism and verification of approach reference speeds, regardless of the context.

?

In an act that speaks volumes about the era and the character of the man, Captain Bates participated in the flight tests documented by the investigation, and actually flew some of the stalls himself. Rather than deflect or diffuse blame, or even bother to locate blame, he was apparently more interested in expanding and improving the knowledge that he had so painfully learned had been deficient. He perished almost a year after the Moorhead crash while executing a production flight test of a Consolidated B-24 Liberator bomber. His daughter, writing in 2010, said that her father “loved flying more than money. In that era, the idea of greed did not enter the minds of those who were pursuing their dream…”?

?

I will venture a guess that Captain Renslow and First Officer Shaw shared some of that love of flying. I will also venture a guess that, given the opportunity to fly simulations of their accident, they would have. We owe it to them, and to their passengers, to follow Captain Bates’ example, and become more interested in expanding and improving the knowledge that had been deficient. We owe it to all of them to step away from administrative boilerplate language, and continually ask ourselves, how do we know what we think we know?

Most importantly, we need to think about where else might we find gaps in aeronautical knowledge that we have, heretofore, believed will simply be colored in by experience, and how and when that experience can be misinterpreted?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?