Why Experienced Pilots Sometimes Get It Wrong

In the 12-year period he conducted more than 430 one-on-one human factor sessions with pilots who had not yet met the required standard in the simulator or on the flight line, Lex Rock Heemstra analysed the trends and pitfalls which resulted in those pilots requiring additional training.

The flight simulator can be quite a daunting place. There are not many professions where you are required to prove your competence every three to six months.

The inability to display the required performance during the checks could mean a removal from the line until trained to competency or, in the worst case, termination from the airline, especially those with a consistent poor track record. Second, exercises and emergencies required to display proficiency cannot be practised on the line. Third, the amount of emergencies in one session alone normally exceeds by far the actual emergencies that most pilots experience in a lifetime. Finally, to increase the stress levels, the emergencies or events in the simulator are conducted one after the other, until the required syllabus is met, which adds a time limit to each event as well.

Unfortunately, there were those pilots who treated the check as “just another flight” with scant preparation, expecting that their natural talents will be enough to get them through. Line flights – conducted mostly with the use of automation where very few emergencies or abnormal events occur – do not prepare one much for the simulator checks. Without proper preparation and a professional attitude, the outcome is destined to be unsuccessful.

“I have never failed before” was a statement heard quite often during one-on-one HF sessions, after a pilot had not met the required standard during training or a proficiency check. There was no specific demographic to the pilots attending the sessions and would range from new joiners to pilots in excess of 20,000 hours.

On the positive side, the vast majority of pilots do prepare adequately and manage to pass the checks each year.

Not Understanding the Rules of Engagement

Like any game, meeting, battle, social interaction, etc, it is important to understand the rules of engagement. In short, what is acceptable behaviour and what is not? During one year of CRM recurrent classes (4,500 pilots), students would be asked what the 9 EBT competencies were; it was somewhat surprising that the vast majority of the pilots only knew a few, but definitely not all nine.

Most had not even looked at the required behavioural markers.

On the upside, the fact that the majority of pilots do not know the behavioural indicators for each competency, yet still manage to achieve above-average to exceptional grades, is indicative of the validity of the behavioural markers. However, those that are struggling to display the required performance could do well to make themselves familiar with the rules of engagement, or in other words, the required behaviour for each particular competency.

Using Past Performance to Determine Preparation

Humans tend to push the boundaries of any rule or law. Consider a traffic radar, where the speed limit is 100 km/h. Most people are aware that the radar is set slightly higher than 100 km/h and so will progressively increase their speed until they are caught. They then back off slightly and drive at a speed just below the trapped speed. Still an offence, but not caught.

When reviewing the past performance of unsuccessful pilots, there was a gradual decrease in the overall grades obtained over a period of time. Some pilots admitted that since the previous check was successful, the need for intensive preparation for the next could be reduced. While they may have got away with it for some time, it did inevitably catch up with them. While previous grades were available to the pilots, very few had looked over them or kept a track record. When presented with the past performance, the trend in decline was quite obvious and would inevitably have led to an unsuccessful performance.

TRI/TRE Reputations

While all airlines strive to ensure that their examiners are standardised in the evaluation process, so long as humans are involved there will be some subjectivity in the assessment. However, there are those examiners that grade harsher than the average examiner and those that are more lenient. Both types, the harsh and the lenient, soon pick up reputations amongst the pilot fraternity, namely “The Hacker” and “Father Christmas”.

In discussion, unsuccessful pilots admitted that they adjusted their preparation in accordance with the reputation of the TRI/TRE. The reputation of “harsh” examiners is often unjust since those pilots who did not meet the required standard with a “harsh” instructor will blame the instructor for their poor grading, resulting in pilots who had never been examined by the examiner to further spread the reputation.

One pilot approached the human factors office and stated that he had been rostered with a known Hacker and was considering calling in sick on the day. By mutual agreement, a meeting between the pilot and the “Hacker” was arranged. The TRE explained that he marks it likes he sees it, as well as that the aircraft would show no compassion during an emergency, and therefore accepted procedure and behaviour was required. Understanding the rules of engagement, the actual session went quite well, resulting in an above-average grade.

Passed Over for Promotion

In both airlines where these sessions took place, the average time for a new joiner to be promoted from SFO to Captain was about three to five years, dependent on fleet. Research on individual grade averages of pilots who had not been considered for promotion to Captain within the perceived time period showed a decline from the time of being passed over, even though since joining the airline their grades had increased steadily. Furthermore, their grades would continue to decrease with each subsequent year they were passed over for promotion. In discussion with the HF manager of a major British airline, the same observation was made, except that this would occur at about the seven-year mark since that was the average experience required to be promoted to Captain.

One can only assume that the major factor affecting the lower grades was one of motivation. Pilots should be aware that above average to exceptional pilots will be considered for promotion based on their performance first, dependent on the requirements of the airline or particular fleet. The slots are limited. Aiming to continuously achieve good grades will make them more likely to be selected for promotion in the next round.

Direct Entry Captains Defaulting to Their Comfort Zone

Direct Entry Captains are normally required to conduct the full type-conversion course and are then released to line in the left-hand seat with very little or no experience on the flight deck of the new airline. While they had the savvy to play the game during the conversion course, once they were released to line they would adopt their previous company culture (their comfort zone) in the cockpit, which often created great confusion for the FOs since the required behaviour was unknown to them (a common complaint of FOs when flying with DEC’s). Having been provided free reign to conduct themselves on the flight deck as they want, without being checked for standardised behaviour, it was not surprising that the failure rate of direct entry captains was two to three times higher than FOs during the very first proficiency check after their conversion course.

While there is an economical benefit into placing a DEC into the left-hand seat as soon as possible, the consequences and costs of additional training should be considered. Here a period of a few months in the right-hand seat might allow the DEC to fully comprehend the flight deck and company culture and thus adjust accordingly.

Dunning-Kruger Effect

The Dunning–Kruger effect is a cognitive bias in which people with low ability at a task overestimate their ability. It is related to the cognitive bias of illusory superiority and comes from the inability of people to recognize their own lack of ability. The determination of this effect is achieved through conducting a test on the knowledge the participants should have and then asking them to rate how they thought they did on the test before the actual results are revealed. As depicted in the diagram below, novices (or underachievers) would rate themselves higher than they actually scored.

Also frightening is that the experts would underestimate their actual knowledge. This effect was especially true with below-average performers or those who had several check failures. When asked how they compared themselves to their colleagues, they rated themselves above-average and even at times well-above-average. Of course, when their grades over the years were revealed to them, they were somewhat surprised to see that they were consistently below-average performers. Providing them with a trend of their actual score compared to their peers did aid somewhat into them accepting their grades and further coaching could now progress.

False Competence

While Standard Operating Procedures are the glue that allows a flight crew to work efficiently and effectively together, even though they have never met before, they are in reality not standard, but constantly evolving. As the airline experiences various issues, flight management will adjust the procedures to accommodate for the lowest common denominator. While most pilots adjust accordingly, the strugglers tend not to update themselves and end up in the simulator inadequately prepared. Given a scenario to resolve, they will resort to the SOPs they know well and will execute the SOPs they know with eloquence, thereby attempting to display their competence. However, the required SOP which would easily resolve the issue is not conducted since it is not known. This is what we refer to as false competence – doing what you know well, but not doing what you should be doing.

Lack of Resilience

Resilience is the ability to bounce back from a bad situation. In some cases, the pilot may get one of the first exercises wrong and spend too much time worrying about the unsuccessful event. Further events do not go well since their mind is clouded by the result of the unsuccessful event.

Considering the components of resilience may do well to overcome these types of scenarios. Pilots falling into this category were asked to do the GRIT test by Angela Duckworth (“Grit is passion and perseverance for long-term goals”) and consistently scored low.

Coaching the five components of resilience resulted in a positive second training event. They are:

As Bob Hoover would say: “Fly the aircraft as far into the accident as possible”. In other words, don’t give up. Careful review of the five components shows a close link to the concept of Knowledge, Skills and Attitude.

“Line flights… do not prepare one much for the simulator checks.” Image credit: L3Harris

Progressive and Accumulative “Startle Effect”

While startle effect has received a lot of attention in recent years, it is normally presented as a sudden unexpected stimulus. Presented with the unexpected stimulus, the amygdala then prepares for a “fight or flight” reaction, and subsequently shuts down the pre-frontal cortex, which is responsible for planning, decision making, short-term memory, personality expression, moderating social behaviour and controlling certain aspects of speech and language – the required executive functions needed for challenging exercises. However, the process itself can also grow in small incremental steps, leading to a final total shutting down of the pre-frontal cortex.

Stress in its most basic form is caused by two factors, the perception of loss of control and the inability to predict the short-term future. Therefore, a series of small mistakes can lead the pilot to believe that they are progressively losing control of the situation, and now no longer have the ability to predict the short-term future. As the stress level increases, the amygdala will again fire up and start shutting down the pre-frontal cortex, which being responsible for rational thought is exactly what is required to undo the situation. Pilots in this category were taught to ignore the results of the previous event, regain control by flying the aircraft, and not allowing the results of the previous experience to affect the next event, i.e., clearing the mind for the next event.

Twin-Engine versus Four

Reviewing just over 34,000 graded sim events, the average grades of pilots flying the A340 or A380 were about 10-15% higher than those pilots on a twin-engine aircraft (A320, A330, B777 and B787). The overall conclusion is that when one engine is inoperative in a twin-engine aircraft, the margin for error is limited since any mistake would border closely to an unsafe situation. The unsafe situation, in accordance with the behavioural markers, would result in a “requires additional training” or “not yet competent” grading.

Some may argue that the shutting down of two engines on the A340 or A380 is equivalent to one engine shut down on a twin-engine. However, the shutting down of two engines on the A340 or A380 is conducted once during the conversion course and then maybe once every three years. So the window for a potential unsafe situation is less than that on a twin-engine aircraft with one engine shutdown. Below-average pilots on four-engine aircraft who were transferred to a twin-engine aircraft required extensive additional training during their conversion.

The Way Forward

Feedback from instructors indicates that an unsuccessful session is not very often due to a sudden and isolated poor event. The overall knowledge, skills and attitude of the pilot are indicative from the first step. Those who come adequately prepared, with the required knowledge, a positive attitude and an interest in honing their skills, seeking alternatives and using all tools available to them, do well most of the time, even those with mediocre handling skills. This positive attitude and conduct will reduce the possibility that a pilot may one day also have to utter the words, “I have never failed before”.

About the Author

Lex Rock Heemstra served as a Human Factors Specialist at Emirates Airlines and Vice President Human Factors at Qatar Airways.