Where's the DATA?
In the incident records, one basic item analysed was the occurrence of events attributed to “faulty operating technique” - uncorrected crew errors leading to an accident or incident. Over a 7-year period encompassing nearly 1.7 million operations and 2.9 million hours, the rate at which these occurred in the traditional “assisted” operation was more than twice that in the “delegated” one.
This analysis covered all flight phases including ground manouvring etc. and it could be argued that generalised statistics such as these aren't relevant to the basic problem of crew-caused accidents, especially CFIT accidents, since the events covered such a wide range of individual actual circumstances.
However it's hard to dispute that when a GPWS alert is involved, an otherwise serviceable aircraft is heading into danger as a result of some misdirection that the crew could and should have prevented. GPWS started being introduced during the period covered by the data set, and the data used here included a separate analysis of GPWS events. This enabled "false" and "nuisance" warnings to be filtered out, leaving only "genuine" warnings which indicate that the crew were deviating significantly from a safe path.
This chart does not differentiate operating procedures and illustrates only that over a range of 14 aircraft types there seems to be a reasonable correlation between the rate of occurrence of "crew error" incidents of all types, and the more specific and highly hazardous events of Controlled Flight Towards Terrain. This could imply that if you can reduce overall crew errors you'll cut the CFTT rate, which seems reasonable.
The chart shows data for several years operation of GPWS equipped types covering about 800,000 sectors, with over 300 genuine GPWS events and many more “faulty operating technique” events. (Anomalies arise on fleets such as "F" which had very limited exposure in the period involved.)
Single type, same environment.
The two operations used different aircraft types, and had different average sector lengths, which may have skewed the statistics. But there was one common type, operating similar medium to long-haul routes with identical sector lengths. Even with this much smaller sample size, the difference was still marked. Over a six year period the average faulty operating technique rate error rate for “traditional” operation was 120% of the overall type average, compared with only 70% for the “delegated” operation. Unfortunately the CFTT/GPWS data for this split is not available.
Descent, Approach, and Landing.
A fundamental point of the PicMA system is that separating responsibility for handling the approach from that of the landing minimises many of the physiological and psychological problems which arise when the PF is forced to make a transition from instrument information to external visual cues. There is a great deal of temptation to make this transition much earlier than is actually necessary, and to replace accurate and valid instrument data with much more compelling (but potentially misleading) external cues.
The problems associated with this transition when weather conditions are close to minima are well known, and are the fundamental reason for the recommendation by ICAO and others that the “monitored approach” be used for such operations.
However, it can also lead to major flight path errors in relatively good conditions. These include making approaches to the wrong runway or even the wrong airport. While these are generally incidents with a high survivability rate, they are nevertheless very serious and potentially very hazardous as well as extremely commercially damaging for the operator concerned.
Approach and landing errors.
In the previous charts data, the handover of control to the P2 in the "Delegated" operation took place during the climb, so both operations were similar up to that point. LOSA data indicates that 40% of all "threats" occur during pre-departure and taxy, and only 30% during descent/approach/landing. On the other hand, 55% of crew errors occur during descent/approach during "traditional" operations. In other words, errors are more common in approach and landing although there are fewer threats. Hence one would expect that if the different philosophies generate different safety results, it would manifest itself most strongly in the descent/approach/landing.
Unfortunately the data used here did not break down the "Faulty operating technique" events by flight phase. But different data from a much longer and earlier period, looking at the approach and landing phase only, showed that for uncorrected major deviations from a flight path which could have been flown correctly by instrument information alone, i.e. gross deviations prior to Decision Height (DH) or Minimum Descent Altitude (MDA), the incident/accident rate difference between the “traditional” and “delegated” concepts increased to some 9 to 1. This includes events of gross deviations during descent such as impact with terrain, and landing at the wrong airport.
Moreover, several of the events that occurred in the "delegated" operation actually occurred after the crew had elected NOT to use the PicMA procedure. As might be expected, there was a much smaller difference in the rate of events during the actual landing phase (i.e. after passing DH/MDA), when differences in procedures was minimal, although one might expect to see fewer LANDING events as prior delivery to the runway environment had probably been more accurate (e.g. fewer rushed approaches).
Is this data still relevant?
At worst, the data seems to show a broad correlation between the use of the PicMA concept and an actual reduction in unwanted events. Certainly there is nothing to indicate that a handover of control for landing following a PicMA approach generates any increase in risk, as has been feared or claimed in some quarters. There was one major accident during a go-around following a PicMA approach, which was at least in part due to a combination of instrumentation deficiencies and poor aircraft handling qualities, and this accident led to a major changes in the safety requirements for low visibility operations including introduction of the approach ban concept and a significant increase in minimum RVRs.
But it's true that this data is decades old, and that technology and training have improved since then. Nevertheless, a vast number of operations still use aircraft of the same generation, and in many parts of the world, flying is little changed from the period when these flights were made. Large numbers of non-precision approaches are still made in poor conditions, and as a number of accidents in the last 10 years have shown, the most advanced technology and up-to-date training methods give little certainty that relatively elementary crew errors won't still occur. So, yes, the data is old and not terribly scientific. But it is still the best available, and should give serious cause for thought.