reducing "Crew-caused"
approach and landing
accidents 

Pilot-in-charge Monitored Approach

Where's the DATA?

The arguments on this website in favour of PicMA might be regarded as "interesting, but still only a hypothesis". Supporting data is needed to make arguments convincing, and because PicMA-based operations are still relatively uncommon, there is no large database available directly to compare the effectiveness of PicMA to more traditional "assisted" flying approaches. But one limited data set does give some indications that PicMA may produce significantly better results.
 
Plenty of experiments have shown that it is beneficial to delegate the flying tasks in order to resolve abnormal situations, and this is generally accepted as a core part of CRM training. Authorities on low visibility operations also recognise the severe physiological limitations on the ability of a pilot to make simultaneous use of both instrument and marginal visual cues, and encourage or mandate delegation of the flying tasks either to the other pilot, the autopilot, or both, until a clear and unambiguous decision can be made that adequate visual reference has been achieved.  
 
An inadvertent experiment.
 
It would obviously be useful to be able to compare real world PicMA-based operations and "traditional" ones, to see if actual experience supports the theoretical benefits of making this the standard practice in normal operations.
 
Ideally, such data would eliminate other variables that affect crew errors, such as pilot background, training and licensing standards, regulatory supervision, aircraft complexity, operating environment, and national cultural factors. It would also encompass not just major accidents but also less serious incidents, which being far more frequent provide a bigger sample, and where the crew's survival has enabled more detailed study of crew factors to take place.
 
While never recognised as such an experiment, this data was actually recorded, several decades ago, in a major international airline that had two separate operational divisions. Both operations involved primarily second generation jets including narrow and wide-body types. Although no “glass cockpit” aircraft were included, a significant amount of flying was on sophisticated types operating routinely to Cat 3 minima.
 
The only other significant difference was in average sector length - one was mostly an intercontinental operation with a 4 hour average sector length, the other mostly an international and domestic operation of 1.2 hour average sectors.
 
Operating under a single top management and regulatory regime, the airline's safety department was independent of line management and reported directly to the company's board. The safety management system also pioneered detailed incident reporting, with common reporting and analysis criteria. Incidentally, it should be noted that the vast majority of safety events recorded were not related to "crew errors" but were events where crews had safely dealt with problems generated elsewhere in the industry. 
 
While the information does not result from what is now known as a Line Operations Safety Audit (LOSA), the reportable events were what would today be referred to as "undesired End States".  
 
Essentially, each division had received a random selection from a pool of otherwise identical pilots, who flew similar aircraft supported by the same operations and training organisation. The main remaining factor which might influence reported "crew-caused" incident rates would then seem to be the "error detection and correction" aspects of operating procedures.
 
One operation used only the traditional “assisted piloting” PF/PNF philosophy, while the other used the “delegated piloting” philosophy, including PicMA, to the maximum extent possible. In those operations the SOP included a handover of control to the P2 on all flights, regardless of the actual weather. This took place generally during the late part of the climb above 10,000ft.  
 
Other factors which might influence monitoring effectiveness, including initial pilot selection criteria, education, licensing and training standards, and the cultural background of all the pilots involved, were the same in both groups.
 
It should also be noted that the data predates any formal recognition of the concept of "Crew Resource Management" as a subject in its own right. Consequently all the operations took place without any of the pilots involved having had CRM training, which could have influenced events. At the time, both operators had mostly ex-military Captains, some of whom had what would today be considered "undesirable" attitudes to interpersonal aspects of crew relations. So differences in CRM training can be ruled out as a factor affecting monitoring effectiveness.
 
"Faulty Operating Technique"

In the incident records, one basic item analysed was the occurrence of events attributed to “faulty operating technique” - uncorrected crew errors leading to an accident or incident. Over a 7-year period encompassing nearly 1.7 million operations and 2.9 million hours, the rate at which these occurred in the traditional “assisted” operation was more than twice that in the “delegated” one. 

This analysis covered all flight phases including ground manouvring etc. and it could be argued that generalised statistics such as these aren't relevant to the basic problem of crew-caused accidents, especially CFIT accidents, since the events covered such a wide range of individual actual circumstances.

However it's hard to dispute that when a GPWS alert is involved, an otherwise serviceable aircraft is heading into danger as a result of some misdirection that the crew could and should have prevented. GPWS started being introduced during the period covered by the data set, and the data used here included a separate analysis of GPWS events. This enabled "false" and "nuisance" warnings to be filtered out, leaving only "genuine" warnings which indicate that the crew were deviating significantly from a safe path. 

This chart does not differentiate operating procedures and illustrates only that over a range of 14 aircraft types there seems to be a reasonable correlation between the rate of occurrence of "crew error" incidents of all types, and the more specific and highly hazardous events of Controlled Flight Towards Terrain. This could imply that if you can reduce overall crew errors you'll cut the CFTT rate, which seems reasonable.

The chart shows data for several years operation of GPWS equipped types covering about 800,000 sectors, with over 300 genuine GPWS events and many more “faulty operating technique” events.  (Anomalies arise on fleets such as "F" which had very limited exposure in the period involved.) 

Single type, same environment. 

The two operations used different aircraft types, and had different average sector lengths, which may have skewed the statistics. But there was one common type, operating similar medium to long-haul routes with identical sector lengths. Even with this much smaller sample size, the difference was still marked. Over a six year period the average faulty operating technique rate error rate for “traditional” operation was 120% of the overall type average, compared with only 70% for the “delegated” operation. Unfortunately the CFTT/GPWS data for this split is not available. 

Descent, Approach, and Landing.   

A fundamental point of the PicMA system is that separating responsibility for handling the approach from that of the landing minimises many of the physiological and psychological problems which arise when the PF is forced to make a transition from instrument information to external visual cues. There is a great deal of temptation to make this transition much earlier than is actually necessary, and to replace accurate and valid instrument data with much more compelling (but potentially misleading) external cues.

The problems associated with this transition when weather conditions are close to minima are well known, and are the fundamental reason for the recommendation by ICAO and others that the “monitored approach” be used for such operations. 

However, it can also lead to major flight path errors in relatively good conditions. These include making approaches to the wrong runway or even the wrong airport. While these are generally incidents with a high survivability rate, they are nevertheless very serious and potentially very hazardous as well as extremely commercially damaging for the operator concerned.    

Approach and landing errors.

In the previous charts data, the handover of control to the P2 in the "Delegated" operation took place during the climb, so both operations were similar up to that point. LOSA data indicates that 40% of all "threats" occur during pre-departure and taxy, and only 30% during descent/approach/landing.  On the other hand, 55% of crew errors occur during descent/approach during "traditional" operations. In other words, errors are more common in approach and landing although there are fewer threats. Hence one would expect that if the different philosophies generate different safety results, it would manifest itself most strongly in the descent/approach/landing. 

Unfortunately the data used here did not break down the "Faulty operating technique" events by flight phase. But different data from a much longer and earlier period, looking at the approach and landing phase only, showed that for uncorrected major deviations from a flight path which could have been flown correctly by instrument information alone, i.e. gross deviations prior to Decision Height (DH) or Minimum Descent Altitude (MDA), the incident/accident rate difference between the “traditional” and “delegated” concepts increased to some 9 to 1. This includes events of gross deviations during descent such as impact with terrain, and landing at the wrong airport.

Moreover, several of the events that occurred in the "delegated" operation actually occurred after the crew had elected NOT to use the PicMA procedure. As might be expected, there was a much smaller difference in the rate of events during the actual landing phase (i.e. after passing DH/MDA), when differences in procedures was minimal, although one might  expect to see fewer LANDING events as prior delivery to the runway environment had probably been more accurate (e.g. fewer rushed approaches).

Is this data still relevant?

At worst, the data seems to show a broad correlation between the use of the PicMA concept and an actual reduction in unwanted events. Certainly there is nothing to indicate that a handover of control for landing following a PicMA approach generates any increase in risk, as has been feared or claimed in some quarters. There was one major accident during a go-around following a PicMA approach, which was at least in part due to a combination of instrumentation deficiencies and poor aircraft handling qualities, and this accident led to a major changes in the safety requirements for low visibility operations including introduction of the approach ban concept and a significant increase in minimum RVRs.  

But it's true that this data is decades old, and that technology and training have improved since then. Nevertheless, a vast number of operations still use aircraft of the same generation, and in many parts of the world, flying is little changed from the period when these flights were made. Large numbers of non-precision approaches are still made in poor conditions, and as a number of accidents in the last 10 years have shown, the most advanced technology and up-to-date training methods give little certainty that relatively elementary crew errors won't still occur.  So, yes, the data is old and not terribly scientific. But it is still the best available, and should give serious cause for thought.