What's It Doing Now?: The Role of Automation Dependency in Aviation Accidents

Presented at 33C3 (2016), Dec. 27, 2016, 6:15 p.m. (30 minutes)

Legend has it that most airline pilots will at one time have uttered the sentence "What's it Doing now?", whenever the autopilot or one of its related systems did something unexpected. I will be exploring some high-profile accidents in which wrong expectations of automation behaviour contributed to the outcome.

"Pilot Error" is often publicly reported as "the cause" of an accident whenever a member of the flight crew did something which had consequences for the chain of events. We maintain that there is never a single cause, and every mistake a pilot may make has causes, and other factors contributing to it. We use the notion of a "necessary causal factor" to investigate the causes of accidents, and almost invariable there is a combination of both technical and human causal factors.

I will look in some detail at accidents in which a combination of a technical problem, misleading or missing indications, and inappropriate (but often understandable) crew actions contributed to an accident, and also some in which unprecedented actions of the human crew turned a problem with potentially fatal consequences into a survivable accident.

Automation in modern airliners has become so reliable and useful that it may be argued that it leads to a deterioration of hand-flying skills and, perhaps more importantly, of decision-making skills.

Sample accident cases will include Asiana Flight 214, Spanair Flight 5022, Turkish Flight 1951, TAM Flight 3054 and others. I will also briefly touch on technical and ethical problems with self-driving vehicles highlighted by the recent Tesla crash as well as increasing automation in General Aviation and its consequences.

In conclusion I will look at ways to improve safety and maintain the very high standard currently achieved in commercial aviation.


Presenters:

  • Bernd Sieker
    I am a systems engineer, computer scientist and aviation accident analyst; I specialise in root cause analysis, reverse-engineering and formal methods for safety-critical systems development and hold a private pilot's license. I have a German Diploma in computer science and a PhD in engineering (Dr.-Ing.), have held university lectures in computer networks and system safety and security for the last 7 years. I also work in a small engineering consulting company where I have done several analyses of aviation accidents, reverse-engineered system functions from publicly available user documentation and evaluated avionics systems and human-machine interaction, especially the role cockpit indications (or lack thereof) and of crew training in aviation accidents. I fly a small single-engine airplane for fun and sometimes for business trips. I also play the piano, keyboards, recorder and tin whistle. I am married and we have three children.

Links: