Xs

DOT/FAA/AM-91/7

Office ofAviadon Medicine

Washington, D.G 20591

The Use and Design ofFlightcrew Checklistsand Manuals

John W. TurnerEG&G Dynatrend

M. Stephen Hundey, Jr.U.S. Department ofTransportationResearch and Special Programs AdministrationJohn A.VolpeNadonal Transporadon Systems CenterCambridge, MA 02142

April 1991

Final Report

This document isavailable to the public throughthe NadonalTechnical Informadon Service,Springfield, Virginia 22161.

&U.S. Departmentof TransportationFederal Aviation

Administration

NOTICE

•This document is disseminated under the sponsorship ofthe U.S. Departments ofTransponadon and Defense in die interest

ofinformation exchange. TheUnited States Governmentassumes noliability for the contents oruse thereof.

*TheUnited States Government does notendorse products ormanufacturers.Trade ormanufacturers' names appearherein

solelybecause they are considered essentialtodie objectofthis report.

1. Report No.

4. Title and Subtitle

The Use and Design of FlightcrewChecklists and Manuals

7. Author's)

2. Government Accession No.

John W. Turner. M. Stephen Huntlev. Jr.9. Performing Organization Noma and Address

U.S. Department of TransportationResearch and Special Programs AdministrationJohn A. Volpe National Transportation System CenterCambridge, MA 02142

12. Spomoring Agency Nam* and AddrotiU.S. Department of TransportationFederal Aviation AdministrationOffice of Aviation MedicineWashington, D.C. 20591

IS. Supplementary Notai

16. Abitract

Technical Report Documentation Page3. Recipient'i Catalog No.

S. Report Date

April 19916. Performing Organization Cod*

DTS-458. Performing Organization Report No.

10. Work Unit No. (TRAIS)

FAOE2/A018011. Contract or Grant No.

13. Type of Report and Period Covered

Final Report

September 1988 - January 199014. Sponsoring Agency Code

AAM-240

A survey of aircraft checklists and flight manuals was conducted to identifyimpediments to their use and to determine if standards or guidelines for. theirdesign were needed. Information for this purpose was collected through the reviewof checklists and manuals from six Part 121 and nine Part 135 carriers, review ofNTSB and ASRS reports, analysis of an ALPA survey of air carrier pilots, and bydirect observation in air carrier cockpits.

f

The survey revealed that some checklists and manuals were difficult to locate andwere poorly designed for use in the cockpit environment, the use of checklists byflight crews was not always well defined, the use of checklists interfered withother flight operations, and flight operations often made it difficult to usechecklists effectively.

The report contains recommendations for the formatting and content of checklists andmanuals, their use by flightcrews, and areas of research relevant to checklistdesign.

17. Keywords

Checklist, Normal, Emergency, Abnormal,Readability, Interruptions, Manuals,Conformance

IS. Distribution Statement

Document is available to the publicthrough the National TechnicalInformation Service, Springfield,Virginia 22161

19. Security Clastil. (of this report)

UNCLASSIFIED

20. Security Clossif. (of this page)

UNCLASSIFIED

21. No. of Pages

75

22. Price

Form DOT F 1700.7 (8-72) Reproduction of completed page authoriiod

>-*-

/''

PREFACE

Followingthe investigation ofthe August 1987crashof Northwest 255, the National Transportation SafetyBoard (NTSB) concluded thatairline training andchecking practices do not promote effective use of checklists.One of the recommendations the NTSB made to theFederal Aviation Administration (FAA) was "to determineifthere isanytypeormethodofpresentingcheckliststhat producebetterperformance onthepartofuserpersonnel."

This report was prepared for the FAA in response tothatrecommendation. The document describes a study ofcurrentchecklistdesigns and practices ofPart 121 and Part135 carriers. Data for thisstudywerecollectedthrough anexamination ofaccident/incidentreports from NTSB andthe AviationSafety Reporting System, manualsandchecklists from Part 121 andPart 135 carriers, and a survey ofairline pilots conducted by the AirLine Pilots Associationto assessthe stateofchecklist use throughout the industry.Recommendations include guidelines for checklist design.

This paper was prepared for the Biomedical andBehavioral Sciences Branch of the FAA Office of Aviation Medicine by the Operator Performance and SafetyAnalysis Division of the Office ofResearch and Analysisat the Volpe National Transportation Systems Center(TSC). The report was completed under the direction ofTSC Program Manager M. Stephen Huntley, Jr.; researchwas the responsibility of John W. Turner of EG&GD.ynatrend, an on-site contractor.

ui

TABLE OF CONTENTS

1. INTRODUCTION 1

1.1 Reason for the Study 11.2 Approach 11.3 Products 1

2. METHODS 1

2.1 NTSB and ASRS Report Summaries 12.2 Study ofParts 121 and 135Operator Information 22.3 ALPA Survey 22.4 Additional Sources of Information 2

3. FINDINGS AND DISCUSSION 2

3.1 NTSB ReportsSummary 23.2 ASRS Reports Summary 43.3 Part 121 and Part 135 Checklist and Manual Review 53.3.1 Policy and Procedures for Checklist Use 63.3.2 Alphanumerics 73.3.3 Method of Presentation 73.3.4 Color Coding 93.3.5 Memory items 93.3.6 Manual and Checklist Contents and Organization 93.3.7 Summary of Findings 12

3.4 ALPA Survey 13

3.4.1 Introduction 133.4.2 Respondent Characteristics 133.4.3 Checklist Layout, Design, and Use 143.4.4 Checklist Interruptions 153.4.5 Compliance, Crewmember Variations,and Cockpit Resource Management (CRM) 173.4.6 Checklist Accessibility 173.4.7 Other Observations 183.4.8 Summary ofFindings 19

3.5 Other Sources of Information 20

3.5.1 NTSB and Related Meetings 203.5.2 Air Transport Association Flight Crew Checklist Working Group Meetings 203.5.3 Jumpseat Observation Rides 213.5.4 Corporate On-Site Visits :. 213.5.5 Cockpit Devices in Use 223.5.6 Summary ofFindings 23

TABLE OF CONTENTS (continued)

4. SUMMARY AND RECOMMENDATIONS 23

4.1 Findings 23

4.1.1 Conformance .....234.1.2 Interruptions 234.1.3 Checklist and Manual Design, Organization, and Contents 234.1.4 Readability 244.1.5 Color Coding 244.1.6 Inconsistency 244.1.7 Definition of "Abnormal" and "Emergency" 244.1.8 Emergency Checklists 244.1.9 Heads-Down Time 244.1.10 Summary of Factors Detracting from Good Checklist Design and Use 25

4.2 Recommendations 25

4.2.1 Checklists 254.2.2 Manuals.... 264.2.3 Checklist Training 274.2.4 Review of FARs 274.2.5 Research and Development 27

APPENDIX A. CHECKLIST GUIDELINES A-1

APPENDIX B. ADVANTAGES AND DISADVANTAGES OF CHECKLIST TYPES B-1

APPENDIX C SUMMARIES OF ASRS SPEOAL REQUESTS 1403 AND 1417 C-1

APPENDIX D. FORMAND RESULTS OF ALPA SURVEY D-1

LIST OF FIGURES

3-1 TYPEFACE SAMPLES 8

A-1 EXTREMES OF PRINT SIZE AND STYLE A-3

A-2 MD-80 CHECKLIST A-5

A-3 DC-9 CHECKLIST A_7

vi

METRIC/ENGLISH CONVERSION FACTORS

ENGLISH TO METRIC

LENGTH (APPROXIMATE)1 inch (in) « 2.5 centimeters (cm)

1 foot (ft) • 30 centimeters (cm)

1 yard (yd) > 0.9 meter (m)

1 mite (mi) a 1.6 kilometers (km)

AREA (APPROXIMATE)1 squareinch(sqin,in') a 6.5squarecentimeters (cm*)1 square foot (sq ft, ft') a 0.09square meter (m»)

1 square yard (sq yd, yd') a 0.8 square meter (m»)1square mile(sqmi,mi') o 2.6squarekilometers(km')

1 acre a 0.4 hectares(he) a 4,000square meters (m»)

MASS • WEIGHT (approximate)1 ounce(oz) a 28 grams (gr)

.1 pound(lb) a .45kilogram (kg)1 short ton a 2,000 pounds (lb) a 0.9 tonne (t)

VOLUME (approximate)1 teaspoon (tsp) a 5 milliliters (ml)

1 tablespoon (tbsp) a 15 milliliters (ml)1 fluid ounce (fI oz) a 30 milliliters (ml)

1cup(c) a 0.24 liter (I)1pint(pt)a 0.47 liter (I)

1quart(qt)a 0.96 liter (I)

1 gallon (gal) * 3.8 liters (I)1 cubic foot (cu ft ft') a 0.03 cubic meter (m>)

1 cubic yard (cu yd, yd1) a 0.76 cubic meter (m>)

TEMPERATURE iexacdI(x-32)(5/9)J,Fa y'C

METRIC TO ENGLISH

LENGTH (approximate)1 millimeter (mm) a 0.04 inch (in)

1 centimeter (cm) a 0.4 inch (in)1 meter (m) a 3.3 feet (ft)

1 meter (m) a 1.1 yards (yd)1 kilometer (km) a 0.6 mite (mi)

AREA (APPROXIMATE)

1 square centimeter (cm1) a 0.16 square inch(sq in, in1)1 square meter (m'J a 1.2 square yards (sq yd, yd1)

1 square kilometer (km2) a 0.4 square mile (sq mi, mi1)1 hectare (he) a 10,000 square meters (m>) a 2.5 acres

MASS • WEIGHT (approximate)

1 gram (gr) a 0.036 ounce (oz)1 kilogram (kg) a 2.2 pounds (lb)

1 tonne (t) a 1,000 kilograms (kg) a 1.1 short tons

VOLUME (APPROXIMATE)

1 milliliter (ml) a 0.03 fluid ounce (fl oz)

1 liter (I) - 2.1 pints (pt)lliter (I) a 1.06 quarts (qt)1 liter (I) a 0.26 gallon (gal)

1 cubic meter (m1) a 36 cubic feet (cu ft, ft3)1 cubic meter (m') a 1.3 cubic yards (cu yd, yd*)

TEMPERATURE.cixAcnI(9/5)y + 32j'C a x'F

INCHES

QUICK INCH-CENTIMETER LENGTH CONVERSION

12 3 4 5 6 7 10

CENTIMETERS 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 I25.40

QUICK FAHRENHEIT-CELCIUS TEMPERATURE CONVERSION

•Is -40* -22* -4* 14* 32* 50* 68* 86' 104* 122' 140# 158* 176* 194' 212#

-I 1 1 1 1 1 1 1 1 1 1 1 1 1 r-'C -40* -30' -20* -10* 0* 10* 20* 30' 40* 50* 60* 70* 80* 90" 100*

For more exact and'or other conversion factors, see NBS Miscellaneous Publication 286. Units of Weights andMeasures. Price S2.S0. SOCatalog No. C1310 286.

VU

EXECUTIVE SUMMARY

Checklists arevaluable, evenindispensable, toolsofairline safety. Yetitisclearthatchecklists arebeingmisused or ignored in the industry.

Checklist procedures werenotcorrectly performedin the August 1987 crash of Northwest 255 inDetroit. Thisconclusion wasmade by theNationalTransportationSafety Board(NTSB)after investigating the crash. The NTSB also concluded thatairline training and checking practices donot promote effective use of checklists.

Although itisnotclearthatchecklistdesigncontributed to the crash, the NTSB recommended as aClassIIPriorityAction(A-88-68)thattheFAAtakesteps"...to determine if thereis any typeor methodof presenting checklists thatproduce betterperformance on the part of user personnel."

This study was undertaken to help in making thatdetermination. We found that checklists can indeedbe improved and have made recommendations tothat end. Other recommendations include the needfor more training, and the need for review of theFARs concerned with checklists and manuals.

This executive summary describes our sources ofinformation, findings, and recommendations.

Sources of Information

We gathered information for the study as follows:

• Reviewed summaries of NTSB and ASRSaccident/incident reports;

• Reviewed selected operator manuals andchecklistsforPart 121andPart 135operators;

• Reviewed resultsofa pilot survey conductedby the Airline Pilots Association(ALPA)—this surveyexplored pilotuse ofchecklists;

Other sources included:

• Meetings with an NTSB investigatorandrepresentatives oftwo regional carriers;

• Meetings with the AirTransport Association(ATA)FlightCrewChecklistWorkingGroup;

• Jumpseatrideson regional andmajorcarriers

toobservechecklistperformance in anoperational setting;

• Visits to twocorporate aviation departmentsto discuss checklist issues;

• Examinationof guidelines in human factorshandbooks and military specifications (MILSPECS) concerning the designof checklistsand manuals.

Findings

The NTSB report summaries included the periodfrom 1/83to 10/86.Duringthis time,21 accidents/incidentsof multi-engine aircraftoccurredin whicha defective or a misused checklist was involved. Infiveofthesecases,achecklistwasnot used at all. (In17ofthesecases,theaircraftwasbadlydamaged ordestroyed.) The ASRS report summaries included195rcportsofoccurrencesinvolvingchecklistsoverthe past five years.The types oferrors found in theASRS report summaries were confirmed by anALPA survey, meetings with representatives ofNTSB,ATA,andregionalcarriers;andbyjumpseatrides on various aircraft Corporate on-site visitsprovided information on checklist technology inselected applications.The following problems wereidentified:

• A breakdownin crew coordinationorproceduresin checklistusecontributed to by a lackof training. There was also a lack of cleardirection to crews in the use of checklists inmany cases.

• Interroptionswereacauseofchecklistmisuse.There wereexternalinterruptionsto the useofa checklist by a flight crew and operationaltasks being interruptedby the necessity to usea checklistThese findingswere confirmedbythe ALPA survey.

• The design, organization, and contents ofchecklists and manuals were often nonstandard. There were missing, inconsistent andincorrectprocedures. Checklistswere sometimesnot in theorder in which they were tobeperformed. Items,and sometimes wholesetsofoperationally relevantprocedures, werenotcarried over from Airplane Flight Manuals(AFM) to checklists.Checklist actions some-

IX

times were different from the required procedure in the AFM.

• Readability varied widely, even within thesame company's checklists. Type size andclarity were dissimilar andthe need for guidelines was apparent

• Color coding of checklists was seldom usedalthough it could facilitate finding criticalchecklists.

• The use of the terms "ABNORMAL" and

"EMERGENCY," as they applied to checklists, was inconsistent What one manufacturermight callan ABNORMAL procedure,another called an EMERGENCY. A cleardefinition ofeach term promulgatedthroughout the industry might promote standard useand eliminate confusion.

• Emergency checklists were difficult to retrieve when needed.They were often carriedin poorlytabbedmanuals in flight bags.

• Heads-down time is reported as increasingwith the use ofchecklists on CRTs. This alsopertainsto the necessity to reprogram cockpitcomputers forchangesin flight plans.

RECOMMENDATIONSRecommendations address the need for improvedchecklists andmanuals andmore trainingin the useof checklists. These recommendations are detailedbelow.

• Design guidelines for checklists and flightmanuals shouldbedeveloped as follows(alsosee Appendix A).

Checklists

"Normal" checklists should:

• Include onlyoperationally pertinent items;

• Be listedin theorderto be performed;

• Have safety critical items such as gear andflaps as final items listed prior to takeoffandlanding;

• Have sufficiently largetype with the necessaryclarityofprintandcontrasttoensure goodreadability in all cockpit lighting conditions;

• faduderomoreindividualcheckliststhancan

fit on a single, easily stowed card.

"Emergency" checklists should:

• Be readily accessible in cockpits;

• Be available on a card as well as in a manual;on the reverseofthe"Normal" checklist card,if possible;

• Have a standard orderof presentation forallaircraftin acompany's fleet sothatindividualchecklists can be located easily,

• Haveclearvisualseparationofchecklists withtitles in boldface, all caps, and in type twopointslargerthanthe text, foreasy identification;

• Be no smaller in type than a well-designed'TCormaTchecklistandlargerifspacepermits;

• Containonly thoseitems neededtocombattheemergency.These checklistsshouldbeeasytounderstand and execute.

Manuals

Procedures specified in manuals for checklist useshould:

• aeariydefinecrewchecklistrolesindifferentphases ofaircraft operation;

• Require specificresponses wherever the"ASREQUIRED"responseiswritten; forexample,"FLAPS...^0o,""ANTI ICE^OFFforON)";

• Require dual response only to the highestpriority safetycritical items;

• Require immediate replacementofcheckhstsworn tothepoint ofreduced readability.

Requirements for the format ofmanuals should:

• Specifyaclearlyreferenced andstandardizedtable ofcontents;

• Specify standardized, color-coded tabs foreach checklist section and subsection with analphabetized indexasthe first page after thetab.

• Initial and recurrent training should be requiredin checklist use.

• Review of FARs should be conducted to determine the need for

• A clear definitionof"NORMAL,""ABNORMAL," and "EMERGENCY" toestablish uniform checklist classificationby manufacturers andairlines;

• A requirement that all operators, regardless of size, meet the same standards formanuals and checklists.

Researchanddevelopment shouldbe conductedto:

• Establish quantitativeandbehavioral criteriafor checklist accessibility and readability;

• Developandevaluate the usefulnessofastandard format organization, and table of contents for aircraft flight manuals;

• Evaluate the use of all caps vs. mixed caseletteringin checklist design;

• Develop and evaluate the use of a standardterminology for controls, displays, and inflightoperationsmchecklistand flightmanuals;

• Evaluatethe utility, safety benefits, andlimitsof audio checklists, checklists on CRTs, andchecklistswithartificial intelligence features,both in a laboratory setting and in an operational context; (There is currentlyan audiochecklist design available from Heads-UpTechnology that will be the subject ofa studyby United Airlines.)

• Evaluate the benefits of color coding anddifferent font styles on checklist readabilityfor electronicas well as paperchecklists;

• Evaluatethe operational feasibility ofcheck-listinterlocksthatwouldpreventaircraft takeoffwithoutcompletionofsafetycritical items;

XI

Evaluatethe utility, safetybenefits,andlimitsofmechanicalchecklistssuchasthoseusedbyAmericanAirlines for"BEFORETAKEOFF'and"BEFORE LANDING.,•;

Developandevaluateaprototypechecklist forParts 135 and 121 use. This list would bedeveloped asanexampleofhow human factors principles in the useof formatting, fontsize, and color coding canbe applied to improvechecklist design;

Determine theinfluence ofmemoryitemsonemergency checklists on the speedandaccuracy with which emergency procedures areperformed.

1.

The Use and Design of Flichtcrew Checklists and Manuals

INTRODUCTION

Checklists have been used, in one form or another,since thebeginning ofmanned flight and certainlysince theinception of theairline industry. Even themost rudimentaryreminderstoassureaircraftreadiness were an early form of checklist With theincreasing complexity of aircraft, theability of thepilot(s) to accomplish all the items necessary forsafety without some type of checklist was diminished, and with the advent of larger and multi-engine aircraft a more formal checklist becamenecessary to assure completion of the multitude ofitems to be checked. However, as aircraft grewlargerandmore complex, aschecklistsgrew insize,and as traffic increased, interferences to checklistuse also increased, with resultant increases in theprobability that errors would be made in the use ofchecklists and checklist-driven procedures. ASRSreports, datainNTSBfiles, pilotreports, anddirectcockpit observations indicate that checklists can bemisused easily and are sometimes even ignored.There is much concern throughout the industry andsome empirical support that such misuse or lack ofusehascontributedto theoccurrence andseverityofaircraft accidents.

1.1 REASON FOR THE STUDYFollowing its investigation of the crash of Northwest Flight 255 in Detroit, in August 1987, TheNational Transportation Safety Board concludedthat"...the flightcrewdidnot perform thechecklistproceduresinthemannerprescribedinthecompany'sAirplane PilotsHandbook." They noted that training and checkingpracticescurrently in use by theairlines do not promote effective use of checklists.

Although it is not clear that checklistdesign was animportant contributor to the Flight 255 crash, theNTSB did include among the seven recommendations produced by their investigation, the Class IIpriority Action (A-88-68) that the FAA take steps"...to determine if there is any type or method ofpresenting checklists that produces better performance on the part ofuser personnel."

The objectives of this study were: a) to identifyconditions that interfere withcockpit crewsexecuting or verifying normal and abnormal cockpit procedures through the use of checklists; b) to determine the need and nature ofFAA action to promote

good checklist practices; and c) to determine requirements for research on the design and use ofcockpit checklists.

12 APPROACHThe following processes were used to accomplishtheobjectives of the study:

• Determinethecontentsandreadabilityofcurrent checklists and handbooks;

• Identify operational conditions that interferewith checklist use;

• Identify flight crew practices that interferewith checklist use;

• Identify design, procedural, operational, andflight crewcharacteristics thatpromote goodchecklist use.

U PRODUCTS• Specification and discussion of conditions

that interfere with good checklistpractices.

• Guidelines for checklist design and evalua-tioa

• Recommendationsforfurtherstudyinareasofchecklist design where more information isrequired.

• Recommendations for changes in FARs topromote improved use and design of checklists.

2. METHODS

We used the following means ofgathering information for this study.

2.1 NTSBANDASRSREPORTSUMMARLESRelevant NTSB and ASRS accident/incident reports were reviewed to identify conditionsthat couldpromote the misuse of checklists, and to identifyoperationalerrors thatmayhaveresultedfromchecklist misuse.

12 STUDY OF PARTS 121 AND 135 OPERATOR INFORMATION

A sample of checklists cards and expanded checklists in handbooks from prominent Parts 121 and135 air carriers were examined:

• To identifydesignand implementation practices that should be promoted;

• To determineiftherewas aneed forguidancein the design and implementation of checklists;

• To identify designandimplementationissuesthatshould be addressed by research, regulations,or recommendations to the industry.

23 ALPA SURVEY

The Airline Pilots Association (ALPA) surveyedline pilotsto requesttheirexperiencesandopinionsconcerningthe checklists they use. It was expectedthat the information provided by this survey wouldindicatetheoperational significance ofvariouscharacteristics of checklist design and design options,serve to identify safety issues that we may havemissed in our analyses, and identify differences inpilot opinion regarding checklist issues.

2.4 ADDITIONAL SOURCES OF INFORMATION

• Discussions with an NTSB investigator andrepresentatives oftwo regional carriers.

• Meetings of the ATA FlightCrew ChecklistWorkingGroup. This group wasconvened toprovide a forum between the FAA groupresponsible forwritingthemanual andchecklistguidelines fortheDrq/iInspectors' Handbookand industry representatives.

• Jumpseat rides onregional and majorcarriersto observe useofchecklists by crews, and toascertainconditions that interfere withchecklist use.

• Visits to two corporate aviation departmentstodiscuss checklist technologyused incorporate cockpits, and to elicit opinions on thattechnology.

• Examination of guidelines for manual andchecklist construction in human factorshandbooks and military specifications (MILSPECS).

3. FINDINGS AND DISCUSSION

3.1 NTSB REPORTS SUMMARYFrom the beginning of 1983 to 10/7/86, there were21 accidents/incidents (involving multi-engine airplanes) investigated by the NTSB, in which theimproper use ofa checklist or a defective checklistwas suspected. In 24% (five) of these, the checklistwas notused atall. Oftheremainder,amanufacturer'schecklist was found to be inadequate in one case,and in the othercases the checklists were not properly followed.

The danger ofchecklist misuse is seen in the resultsof the accidents, 81% (17) of which resulted insubstantial damage or destruction ofthe aircraft. Abriefsummary ofthe NTSB investigations follows.

• Detroit MI, 1/11/83 - United Airlines DC-8-54F - aircraftdestroyed - threecrew fatalities- impropertrim settingcausedloss ofaircraftcontrol • might have been compounded byunqualified 2nd officeroccupying 1stofficerposition during takeoff - checklist not followed.

• BryccUT, 4/2/83- Republic DC-9-82- bothengines flamed out due to fuel starvation -emergencydeclared-enginesrestarted-check-list not followed due to distraction.

• Utile Rock, AR, 4/13/83 - Central HyingService Beech BE-58 - substantial aircraftdamage - gearuplanding excessiveworkloadand checklist not used.

• Luke AFB.AZ, 5/28/83 -RepublicDC-9-31• forced landing caused by engine fiameoutdue to fuel exhaustion-atripped fuel quantitycircuit breaker was not noticed during thepreflight checklist - checklist not followed.

• Blountville, TN, 10/28/83 - Atlantic Southeast Embraer EMB 110-P1 - substantial aircraft damage - 16 minor injuries - aircraftlanded gear updue to indication ofonegearnot down and locked - no confirmation madeon indication problem • checklist not followed.

• Longview, TX, 2/29/84 - Mid America Airways, Inc. Beech E-55 - substantial aircraftdamage - two minor injuries - total loss ofpower, forced landing - took off on almost

empty auxiliary fuel tanks, plentyof fuel inmain tanks - checklist not followed.

• Grand Island, NE,6/29/84 -PioneerAirways,Inc. SwearingenSA 227-AC - minor aircraftdamage-lossofcontrol ontakeoffroll,struckrunwaylight- left proponstartlocks- checklist not followed.

• Selawik, AK, 10/16/84 - Ryan Air Service,Inc. Beech3NM- substantial aircraftdamage- gearup landing- checklistnot followed.

• San Antonio, TX, 12/24/84 -K.ECohlimaBeech95-C55 - substantial aircraft damage -gearup landing - checklist not followed.

• Holly Springs, MO, 2/8/85 - ProfessionalAviation Beech 58 - substantial aircraft damage - gear up landing - couldn't lower gearmanuallybecause the pilot couldn't unstowthe crank - checklist not followed.

• Berkeley, MO, 2/13/85 - Britt Airways, Inc.Swearingen SA226-TC-bothenginesquitonfinal dueto iceingestion - plane landed with-outdamage-nothingonthechecklistconcerningtheuseofauto-ignitionin freezing outsideairtemperatures.

• Williston, ND, 4/7/85 - PioneerAirlines,Inc.Swearingen SA 227-AC - substantial aircraftdamage - landed gear up - improper use ofchecklist

• Potsdam, NY, 5/17/85 - Sair Aviation PiperPA-31-350-substantialaircraftdamage-gearup landing• checklist not followed.

• Atlanta, GA, 5/19/85- Basil AircraftServicesEmbraerEMB-110-Pl • substantial aircraftdamage - collision with parked aircraft onrollout - insufficient hydraulic brake pressureduetoincorrectmonitoringofwarningannun-ciator light and useof incorrect procedure -checklist not used.

• Nashville, TN, 5/31/85 - General Aviation,Inc. Gulfstream G-159 - aircraft destroyed -twocrewfatalities-lossofcontrolafterenginelossontakeoff,propdidn'tfeather-H.P.cocklevers notin"cruise lockout" position - itemnot done on checklist before takeoff.

• Dallas, TX, 8/7/85 - Air Midwest, Inc.Fairchild/Swearingen SA 226-TC • substantialaircraft damage - gear up landing - couldhave manually extended gear - didn't usechecklist

• Orlando.FL,4/22/86-CraigAirCenterBeech95-B55 - substantial aircraftdamage•gearuplanding-lateextensionofgear.aircraftlandedon geardoors - checklist not followed.

• Indianapolis, TN, 7/9/86 - PDQ Air ServiceBeech BE-58 - substantial aircraft damage -gearup landing- checklist not used.

• Jacksonville, FL, 10/7/86 - Top Flight Inc.Ted Smith Aerostar600 - substantial aircraftdamage-gearup landing-checklist notused.

• Santa Barbara, CA, 10/30/86 - Wings WestAirlines, Inc. Fairchild/Swearingen SA-226-TC - substantial aircraft damage - oneseriousinjury, twominor injuries - gear uplanding -prop fragmented and punctured passengercompartment - gear warning horn circuitbreakerdeliberately pulled andgearcalled forbut not extended - checklist not followed.

• Florence, SC,2/5/87 - Atlantis Leasing, Inc.Swearingen SA-226-TC - substantialaircraftdamage - gearup landing - checklistnot followed.

In one of these cases, the incident was directlyattributable to the use of an inadequatemanufacturer's checklist Inanother case, inflightdistractions contributed to a lack of conformity tochecklist procedures. One report cited excessiveworkload as a factor. In another case, the NTSBcitedthecompanymanagement for"improperemergency procedures training"ofits pilots.

Of the 21 cases reviewed, 20 involved lack ofconformance with the FARs regarding checklistuse. Inthecases not involving extenuating circumstances, it is not possible to ascertainthe reason fornonconformity from the information we have. Butthe large proportion of instances of nonconformityindicates that this problem may beas great aproblem as is.checklist design, ifnot greater.

32 ASRS REPORTS SUMMARYASRS reports provide a rich sourceof informationregarding problems inaviation. Theyare submittedonavoluntarybasisby pilots,controllers, and othersin the operational side of the industry. Becausesubmissions are voluntary, thecontentsofthisdatabaseshould notbeconsidered representativeenoughfor use in describing all errors and problems thatoccurinthe cockpit The crews report the problemsthat they want to report Nevertheless, there is noreason todoubt that theproblems that are reporteddid in fact occur.

Those submittingreports are asked toidenti fy themselves for purposes of phone contact by ASRS foramplifying information; however, all reports aredeidentified shortly after being received. The reports are available for research onspecific subjects.Werequested reports on any occurrences involvingchecklists over the past five years. We receivedsummaries of 195 reportsthat were relevant to ourstudy. A summary of each of those is included inAppendix C. The following shows categories oferrors made andgives examplesofeach.

• Sixty-five were casesofchecklist items beingmissed orincorrectly performed by thecrew:

- Engine flamed out at altitude from fuelexhaustion. Declaredanemergency.Crewhad not turned on all boost pumps asinstructed in the checklist

- Control lock still installed on the yokeduringtakeoff. Aborted flight 40 intheairafter noticing lack of control response.

- Altimeter mis-set by 1", not checked bycrew, altitude overshoot on short final,warnedby the GPWS.

• Taihadnothingonthe"beforelanding"check-listtoaccomplish therequired action:

- Aircraft landed with fuel badly out ofbalancelimitations.no item on the checklistto check fuel pumpconfiguration.

- Altitudeundershootinclimb.Theresetofthe altimeterat 18.000" to QNE (the setting of altimeters to 29.92 at 18,000 feetand above) was not on the checklist

• Eleveninvolvedpoorly designed checklistsor

manuals:

- Checklist called for throttle tobepulledout 1/2" on start whetherengine washotornot Onstart, thepilotcouldnotcontroltheplaneandhitthe fuel pump(the throttleshould beclosed for hot-engine starts).

• Altitude overshoot on climbout Check-Ustpn>xdurehasaltimetersresetatlO.OOO*inthe climb - far too late when assignedaltitude is below that

Six had no checklist to use:

- Aircraft failed to pressurize because neitherair conditioning pack was functioning.No abnormal checklist was availableto coverthatcondition(this was onawidebody airplane).

- Aircraftlandedgearup.Nochecklistandthe pilotdidn't use aGUMP check.

Twenty indicated that theappropriate checklist wasnotusedby thecrew:

- Atl^OO'inclimb.anexperiencedCaptaincutthe fuel to both engines (two-engineaircraft) in response to an annunciatorlight for right engine EEC. Copilot (PF)reported thatthe Capt didnot referto theabnormal checklist or coordinate withhim prior to the actioa

- Crewlookingforunfamiliarairportdidn'tdo the final checklist and landed gearup.Warninghorndidn't sounduntil the flare-too late.

Seventy-fourshowed poorcrewcoordinationin the use ofa checklist

• Engine shut down needlessly in flightduring performance of electrical abnormal checklist procedures. First Officerstarted APU forbackup- Captain sawthelow oil pressure light at APU start andmistookit foranengine low oil pressurelight shutting downtheengine. First Officerdidn'tinform CaptainofstartingtheAPU, and Captain didn'tconfirm enginelow oil pressure with First Officerbeforeshutting down the engine. Emergency

declared with unscheduled landing.

- Aircraft taxied across an active runwayafterinstructions to hold short FirstOffi

cer gotinstructions, assumedCaptainhadheard them and started doing the checklist heads-down.

- Early turn to a SID (StandardInstrumentDeparture) heading with traffic conflictCrew busy reading the checklist and notbackingeachotherup.

Eighteen involved the use of an incorrectorincomplete procedure as prescribed by thechecklist

- Aircraftdeparted10,000lbs.lightonfuel.New fueling procedureprovided no clearmeans offuel load verification for fuelers

or crews.

• Hist Officer lost his instruments and the

radarashe wasabout topenetratea lineofcells. Captain and Second Officer weredoing an electrical abnormal checklistwhich knocked off the instruments andradar.

One-hundred thirteen involved an interruption or distraction, either from the use of achecklist from operational matters, or fromsome extraneous event

- Overshotaltitudebyseveralthousandfeetinexr«riencedcrewbusydoingthecheck-list and workingATC radios.

- Altitude overshot on descent BetweenFL310 and FL180,crew had fivespeedchanges andtwoheadingchanges.Subse-quentlytheyhadthreemorespeedchanges,two more heading changes, and threerunway changes - the last occurring at400* on final. The altimeterof the pilotflying did not get set properly.

- Aircraft almost departed with a spoilerextended. Crew taxiingwith one engineshutdowaControlleradvanceduieirtakeoff position. Rushingto complete everythingandmissedtheannunciatorlightfortheextended spoiler. Caughtbycrewinafollowing aircraft

(The percentages add up to more than 100%becausemany samples involved multiple considerations.)

Since these reports are provided to NASA/ASRSona voluntary basis, information which would nototherwise be available is provided about problemsin aviation. Although they may not be completelyrepresentative ofthe industry, these findings help topoint out the variety of the problems encounteredwith regard to checklist misuse.

SUMMARY OF FINDINGS

• CHECKLIST USE

In 43% of the reports studied the crew hadeither not used the checklist at all, or hadmissed important items on the checklist

• CHECKLIST AND MANUAL DESIGN

These factors accounted for 20% of the reports. Design problems included items missing from checklists and inaccurate or incomplete procedures which could lead to potentially dangerouspractices.

• TRAINING

Thirty-eight percent of the reports involvedinadequate crew coordination. This could indicate an absence of instructions in the AFMor inadequate trainingin checklistuse.

• INTERRUPTIONS

Interruptions accounted for 58% of the reports. Therewas about an even division ofthefollowingtwo types ofdisruptions:

- events,suchasATCcaIls,intenuptingthecrew's use ofchecklists;

- thenecessityto readachecklistinterrupting an operational task, such as maintaining a positionin a departure queue.

3J PART 121 AND PART 135 CHECKLISTAND MANUAL REVIEW

We reviewed six Part 121 operators' and nine Part135 operators' manuals and checklists as one meansof identifying good and bad aspectsof current aircarrierchecklistpractices.Thesematerials were not

randomly selected and so are not assumed to berepresentative ofwhatis usedin theindustry. Theyare, however, examples ofmaterials indaily usebymajor carriers.

3.3.1 Policy and Procedures for Checklist UseAllofthePart 121 operators studied specified somepolicyregarding theuseofchecklists fortheircrewsto follow. Somehadveryspecificguidelines regardingwhowas toread eachchecklistbywhat phaseofflight it was to be accomplished, in whatmanneritshould be read (e.g., challenge/response or silent),whether with single or dual response, and whatresponses should be given in lieu of"CHECKED"or "AS REQUIRED." Others only used phrasessuch as "Checklist use is mandatory.", and "Safeoperating procedures are notoverlooked while givingattention to the checklist." Still others merelyspecified who should read each checklist and atwhat phases of flight they should read it Oneexampleofthisis theairline specifyingthattheFirstOfficer should read all "Normal" checklists whilethe aircraft is stationary, and the pilot not flyingshould read all "Normal" checklists while the aircraft is in motion.

Of the Part 135 operators, only one did not havesome sortofpolicy fortiiecrewsto follow. The otherpolicies ranged from numbered notations on eachchecklist margin as to who should answer eachchallenge, to the very detailed and explicit directions from one of the carriers to their crews. Theirpolicystatementswereasgood assomeofthelargercarriers, and better than others.

Onecarrier was unique amongallthe carriers studied in that it specified that its "Normal" checklistswere to be used as "work" lists rather than "done"lists. Rather than the itemsbeingaccomplished andthen checked for completion by the use of thechecklist, it specified that the challengebe read,theitem be accomplished, and then the response begiven, indicating accomplishment While this issometimes the case with "Emergency" checklists,and often the case with "Abnormal" checklists, it isnot usual with "Normal" checklists.

Three issues arise with policy and procedures forchecklist use. They are:

• When should checklists be used?

The timeachecklististobeusedisspelledoutin part in the name of the checklist e.g.,

"BEFORE TAXI," "BEFORE LANDING,"etc. Someofthecarriers in their policy statements are even more specific; prescribing inwhatphase of flight andatwhatpointin thephaseof flight a checklist is to be read. In anumberofthe caseswe studied,however, thiswas left to the pilot

• Who should read/respond to the checklistitems?

This washandledby theairlines inamultitudeof ways. Some addressed the issue with adetailed policy statement stating which pilotshould read which checklist and which pilotshould respond. Others made a margin nota-tiononeachchecklistwith anumberdesignat-ingwhichpilotwastorespond. Othersdidnotaddress the issue.

Another point in this issue is that of dualresponse.This involves items which must bechecked and responded to by at least twocrewmembers, frequently at busy phases offlight some airlineshave items to which allmembers of a three-person crew must respond. This creates a division ofattention forthe pilot flying. Ofthe Part 121 carriers studied,mostusedsomedual response itemsinall"Normal" checklists, whereas, ofthe Part 135carriers, only one did. One of the Part 121canierslimiteddualiesponseiternsto"GEAR"and"FLAPS,"andthenonlyontwochecklistprocedures; "FLAPS"on the'TAXT procedures list and "GEAR" and "FLAPS" on the"LANDING" procedures list Limitingdualresponse requirements to.one or two itemsreduces the amount of time when bothcrewmembershavetheirheadsdown,yetprovides an additional level ofattention to ensurethatthegearand flaps are positioned properlyforhigh-riskphases of flight

• How should the checklists'be used?

This issuewas not addressed by many oftheairlines. And those that did address it were notalways consistent As anexample, let us usethechecklistresponse"ASREQUIRED." Onecarrier did not use any "AS REQUIRED"responses on some of its aircraft, but did onothers.

The general issue of requiring a specific responseinlieuofthe"ASREQUIRED"shownonachecklist was addressed.Therequest foraspecificresponse requires thatthecrewlookattheitem beingchecked inordertogive thatresponse. The discretion to answer "AS RE-QUIRED"permitsoirelesscheckingandpoorchecklist habits. Six of the Part 135 carriersallowed the use of the "AS REQUIRED"response, as did two of the Part 121 carriers.ThehandbooksofthreeofthePart121carriersstatedthataspecificanswershouldbe substituted for"AS REQUIRED," andonePart 135carriervery specifically disallowed "AS REQUIRED" and specified precise responses.Examples of this would be "12 QUARTS,""ON," etc. Onemajorcarrier eliminated theproblem by nothaving "AS REQUIRED" asa checklist response.

3.3.2 Alphanumbrics

The comparison ofprintsize andlettercaseused inthe text of the checklists revealed a number ofproblems. This was trueofboth thePart 121carriersand the Part 135 carriers.

"Normal" checklists for all but one of the Part 121carriers and 50% of the Part 135s werein 10-pointtype, and usually in all caps (see Figure 3-1). Thiswas normally quite legible, but in some cases, thequality of print was poor and that affected thelegibility considerably. MIL SPECS (MIL-C-81222C and MfL:-C-38778A) recommend the useof 12-point type for the body ofthe text OneofthePart 121 carriers used six-point type, mixed case(see Figure 3-1), their checklists were difficult toread, and it wouldhavebeeneasytoloseone'splaceif distracted by other operational requirements. Inthe Part 135 checklists, of the 50% that did not use10-point type, the type size varieddown to seven-point,mixed case,andwasnotvery legible.One setof regional checklistsincorporated aV speedtablein five-point type (see Figure3-1), andthenumberswere almost illegible.

"Abnormal" and "Emergency" checklists showedeven greater inconsistency in alphanumeric sizesthan the "Normal" checklists. One major carrier intheir "Normal" checklist used 10-point type, allcaps.Yet their"Abnormal" checklist althoughkeptin a well-tabbed pilots' handbook and easy to find,was in six-point type and mixed case, and difficultto read. Their "Emergency" checklists were presented on a color-coded papercard with one side in

10-point type, the other side in eight-point type.Both sides were in all caps. The eight-point wasslightly lesslegible than the 10. It appears that thiscombinationoftype wasused in orderto includeallthe checklist items on a single card. Another Part121 carrier, although using legible 10-pointtype intheir"Normal"checklist used eight-point type andall capitals with theletters spaced closely togetherfor their other checklists.

Among the Part 135 checklists, the same sorts ofproblems, but more pronounced, were often seen.One of the regionals used legible 10-point type forthe"Normals"and then reduced toseven-point typefortheir"Emergency" checklists. The reverse wasfound in anothercase, with the "Normal" checklistsin the small, difficult-to-read print

The practice of using smaller, lesslegible type for"Abnormal" and"Emergency" checklists than for"Normal" checklistswas found amongstbothmajorand regional carriers. Since these are checklistswhich areused under conditions ofstress, andoftenwithpoor illumination, theyshould beaslegible aspossible, and surely not smaller than the "Normal"checklists.

Clear, 10-pointtypepresents alegiblechecklist andisusedby anumberofthemajorcarriers we studied.However, with type larger than 10-point, as is recommended by the aforementioned MIL SPECS andby the Human Engineering Guide to EquipmentDesign, the checklist page becomes larger, ormorepagesarenecessary,andchecklist stowageandhandlingbecomes moreof a problem.

3.3.3 Method of PresentationAll of the Part 121 carriers studied used paperchecklists for at least the bulk of their "Normal"checklists. By contrast, only 50% (five) of the Part135 operatorsdid this. One Part 135 carrierhad its"Normal" checklists on a laminated card, and theother four were in either a manual or a separatechecklist booklet.

Oneofthe majorcarriers studiedused paperchecklist cards for all but the "BEFORETAKEOFF' and

"BEFORE LANDING" checklists. These weremechanical, in either a lighted slide or a lightedtoggle switch configuration, depending on the airplane type. They did have a printed backup in theOperating Manual to cover the possibility of amechanical checklist malfunction. The use ofthese

mechanical checklists for this limited use was re-

FIGURE 3-1. TYPEFACE SAMPLES

BEFORE STARTING ENGINES

LOG BOOKS AND SEL CHECKED* RUDDER PEDALS AND

SEATS ADJUSTED AND LOCKED* WINDOWS CLOSED AND LOCKED

02 PANELS/MASKS/INTERPHONE/GOGGLES SET AND CHECKED

EMERGENCY LIGHTS ARMED* PROBE HEAT CAPT* WINDSHELD ANTI-ICE ON

ANTI-SKID OFFPRESSURIZATTON AUTO (UP)AND SET

* AIR COND SHUTOFF AUTO* FLIGHT GUIDANCE PANEL SET AND CHECKED* FLT INSTR/SWITCHES/BUGS SET AND

CROSSCHECKED* FUEL PANELQUANTITY AND

DISTRIBUTION SET/ LBS ANDCHECKEDGEAR HANDLE AND

LIGHTS DOWN AND GREEN* TRANSPONDER SET* STABILIZER TRIM SET

SPOILER LEVER RETTHROTTLES..... CLOSEDFUEL LEVERS OFFFLAPS/SLATS UP/RETRACTED

* AILERON/RUDDER TRIM .ZERO/ZERO* PARKING BRAKE/PRESSURE PARKED/NORMAL* SHOULDER HARNESSES (IfOperative) ON* FLK3HTF0RMS CHECKED* NO SMOKING SIGNS ON* SEAT BELT SIGNS (S Minutes Prior To Departure) ON

PRIOR TO ENG START OR PUSH-OUT

GALLi-IY POWER OFFENGINE IGNITION CONTINFUEL PUMPS ONAUX HYDRAULIC PUMP ONANTI-COLLISION/EXTERIOR LIGHTS ON/AS REQUIREDDOOR ANNUNCIATORS OUTAIR CONDITIONING SUPPLY SWITCHES OFF

10 POINT

MD-80UTEttUL aCCTtIC 4 MBMMIC SOUKE - SIMT

nouunc i-fum ion cusuMUUTtC All fOUKt COKCTUKMrwiMMic x-ruM onttMUUTtC NIIHU CIS Ml MM) OBccHrim . tuou miT cornier

trru tmtmt maiuxcomuiatic t-ruo* Km cuntsUfCRIC NWI •OBIurutuL uxcnic 4 ncurunc ... DIICOUKCnOconrun • inn nurt eaaam

kfokstmtirr

WIMDUIILD mat »mrat hmm •luw)cum rustuu CtMTMUU «urauc vrauuuc nw 4 rtusuu *on 4 encimmit iuabis **amAUTOUMB CKD

uoios. altimctui 4 ruarr die —aa 4 mfUCll OIL «*<quMiriTIU) 4 (KIT

ICNITICM ON

MAT MLT MM ONON

ahu stmtANNUMCIATOKIGNITIONUICTIIC rowAfU All *<AI UQ)All CONOinONINB tUffLT MtTOCU •AUTOMUJIATIC I-rUS *0NE CUUOTiAKifu rua> 4 mtAuuc irrnia .... «om 4 en

6 POINT

B99 SPEEDSKIAS

30% Film ,0* riw* uD%rmW«ent V, V«l v» v» V.M V«H »«!»

10900 « •4 ia 91 Ml 114 1?toooo •4 It w M Ml lit nMOO *t « m M M 1M MMOO « W M 10 H tn ct

C99 SPEEDS-KIAS

JOUFIw* o* rite* tooKntnWtleM V, VM v« v» WaiVm V«w

11300 NO 110 107 11* (t til torItOOO Ml 115 KM 114 II 114 MOtOOOO Ml til Ml 111 II lit Ml0000 Ml W M Ml II tOI toooooo tot 10> f» rat it tn M

5 POINT

ported on very favorably by the pilots using themduring our cockpit observationon that airline.

One Part 121 airline used paper checklist cards for"Normal," "Abnormal," and "Emergency" checklists, and stowed them all in the cockpit The size ofthe paper checklist cards studied varied, and isimportantonly inthatitmustbelargeenoughtoholdlegible checklists, and small enough to be stowedreadily in some location in the cockpit.

Those studied ranged from a fourfold 107/8" x 5 1/2" to a no-fold 8 1/2" x 11." The former was verycrowded and difficult to read, whereas the latterwasvery legible. Insomecases, thelargecards designedto be no-fold were observed to have been folded bythe crews, presumably for convenience.

Most of the carriers kept their "Abnormal" and"Emergency" checklists in manuals or booklets ofsome sort. All of the Part 135 manuals studied, andsome of the Part 121 manuals, lacked tabbing forquickreference and easyidentification.This lackoftabbing could provide an added impediment to acrew at a time when they arc alreadydealing with asituation other than normal. The use of a booklet,capableofbeing stowedin thecockpit, is preferableto a manual stowed in a flight bag from the standpoint ofaccessibility. Handier yet would be a separate card of"Emergency" checklists stowed in thecockpit.

If a booklet or a manual is to be used, it should beproperly tabbed for quick reference. Each majorsection should be tabbed with the name of the

section, and each subject within a section tabbed tocorrespond with the appropriate subject shown inthe section index.The section index should beonthefirst page of each section, following the tab. If themanual contains a section on aircraftsystems, thereshould be a tabbed subsection for each individual

system, (e.g., engines, flight controls, etc).

33A Color CodingTwo of the Part 121 carriers, and three of the Part135 carriers used color coding for easy identification of "Abnormal" and "Emergency" checklists.There have been instances cited in ASRS reportsinwhich crews have had difficulty in locating"Emergency" checklists. Human factors research indicates thatcolor coding can be effective in helping toidentify emergency checklists. Advisory Circular25-11, dated 7/16/87 recommends red be used forthe most serious conditions, and yellow be used for

abnormal conditions of a less immediate nature.

335 Memory ItemsMemory items on "Emergency" checklists havebeenapointofdifference in corporate philosophiesfor years. Of the Part 121 "Emergency" checklistsreviewed, allhad some form ofmemoryitems; thoseitems which the crew must commit to memory forperformance in anemergencysituation, tobringtheemergency under control before referring to thechecklist. One major carrier, which was not includedinourstudy, hasadoptedthe philosophythatmemory items arenot only not necessary, but mayprecipitate amistake through too much haste.Theyhave eliminated memory items from their "Emergency"checklists, andinstead usethemaslists fromwhich to work. This is not the case with mostcarriers. They rangefromhaving memory items forallthe initialsteps in allthe"Emergency" checkliststo avery limited number ofitems on asmall numberof checklists. The former is more common. Thelatteris represented by one ofthe Part121 operatorsin pur sample. Only three of their "Emergency"checklists contained memory items: "ENGINEFAILURE," "ENGINE FIRE," and "ENGINETAILPIPE FIRE," andeach list contained only onememory item. In all three cases the item was thesame.'THROTTLE.CLOSE CLOSE."

The Part 135 carriers were apparently not muchdifferent from the Part121carriers in this regard. Ofthe 10 studied, eight used memory items. One didnot require them, andthe tenth providedno "Emergency" checklists for study.

3.3.6 Manual and ChecklistContents andOrganization

The Part121carriers generallyexhibit more legibleand professional-looking checklists and manualsthan their Part 135 counterparts. However, there isstill room for standardization and improvement.Despite the generally high quality of professionalstandards and performance of Part 121 scheduledcarrierpilotgroups, therehave been many instancesof lapses in checklist use, some with catastrophicresults. Ifminimum standards for legibility, accessibility, and quick recognition were adopted, theavailabilityofachecklisteasy to read anduse woulddiscourage checklist misuse, whereas lack of standards in the past has contributed to this misuse.From that point it would become a question ofairline training and discipline, and individual professionalism.

The material from the regional Part 121 carrierstudied illustrated some of the shortcomings foundin the manuals and checklists of smaller carriers,especially thePart 135 carriers, manyof which flyairplanes produced outside the United States. Although the manuals and checklists of U.S. aircraftmanufactured for the regional and Part 135 marketdon't generally come up to the standards of thoseproduced by the U.S. manufacturers of large aircraft, the problems seem to be even worse in manuals and checklists for aircraft of foreign manufacture. Part of this is a problem of language andterminology. Part of it seems to arise from the factthat themanual and checklist material from foreignmanufacturers isapproved by theirequivalentoftheFAA under the bilateral agreement. Problems include:

• Lack of tabs in the manuals, which makes itmore difficult to find important informationquickly. One manual was tabbed but most ofthe tabbed sections were not numbered, eventhoughreferenceswere made to those sectionsby number.

• Accessibility of important informatioa OneAFM hadnosystemsdescriptions ofanysort.Another, inits"Abnormal" and"Emergency"sections, frequently made references to figures andparagraphsinotherpartsofthemanualrather thansupplyingtheneededinformationat that point. These characteristics decreasethe value of the manual as a reference inaddressing abnormal and emergency situations.

• Anexcessivenumberof"Emergency"checklists, and a classificationof"EMERGENCY"whichwasnotconsistentwithgeneralusageinthe United States. The AFM forone foreignairplanecontained82"Abnormal" and"Emer-gency"checklists,ofwhich 39 wereclassified"Emergency." Manyofthe39wouldnothavebeen classified "Emergency" by most U.S.standards.

• An excessive numbcrofmemory items.Thesechecklists were for an airplane operated by aregional carrier, sometimes flown by low-expericncc-lcvcl crews. This combination ofan overwhelming number of memory itemsand low-time crews is conducive to errors inemergencies.

10

• Missing itemsonchecklists. Examples ofthisare seenin the following.

- Carrier B

No mention of "GEAR" on the "BEFORE STARTING" checklist, and nomention of "FLAPS" on any checklistpriorto takeoff.

- Carrier E

Onallthreegroups ofchecklists—"Normal," "Abnormal," and "Emergency"—thereapoearchaUengeswithoutresponses,as in "EXCESSIVE LOADMETERFAILURE,"'BATTERY " (noresponse).

- Carrier G

Operationally important itemsnotcarriedover to the checklists from the AFM included:

• From "ENGINE FIRE OR SEVEREDAMAGE," "FUEL CROSS-FEED. SHUT."

• From "ELECTRICAL SMOKE ORFIRE,""RECIRCFAN. OFF."

• In some cases, "Emergency"checklistswerenotcarriedoverrromtheAFMtotheoperatingchecklists. FAR 125.71 states that"Each certificateholdershall prepare and keepcurrentamanual. A copy of the manual... shall be...furnished to - (1) Its flight crewmembers."FAR 125.73 says "The manual mustinclude...(m)procedures forensuring compliance with emergency procedures,..." FAR25.1581 states"An Airplane Flight Manualmust be furnished with each airplane, and itmust contain the following: ...(1) Informationrequired by 25.1583 through 25.1587."25.1585,"Operating Procedures," includesemergencyoperationofthesystems.Onecarrierwas using checklists that did not include11 "Emergency" checklists that were in theAFM.This certainly circumvents the intent ofthe FARs. Among the checklist proceduresthatwere missing were the following:

- "ENGINE OVERSPEED"- "PROP OVERSPEED"- "FUSELAGE SMOKE OR FIRE"- "DOUBLE GENERATOR FAILURE"- "BATTERY OVERHEAT'

The "Emergency" checklists of anothercarrieralsolackedmanyoperationallysignificantprocedures which werein the AFM. Amongthese were:

- "PROP MALFUNCTION — OVER-SPEED"

- "FUSELAGE FIRE"

- 'TOTAL ELECTRICAL FAILURE"- "LOSS OF ALL SYSTEM FLUID"

Manufacturers as well as operators were remiss. An example can be shown from theAFM ofone Part 135aircraftIt lacks procedures orcheckliststodealwithproblems suchas "LOSS OF ALL GENERATORS."

• Procedures were not presented in the orderinwhich they shouldbe accomplished. OnePart135carrier's"Normal"checklisthad"SHUT-DOWN" following "BEFORE TAKEOFF."Normally "SHUTDOWN" is the last of the"Normal" procedures. Procedures should bepresentedin chronologicalorder.

• lriternalinconsistendeswerea]sofound.Theseconcerned a variety of issues such as:

- Crewsize. Oneoperator's "Emergency"section preface contained the followingstatement:

"Emergency procedures have been formulated based on single-pilot operationof the airplane."

However, throughout the section of theCompanyAircraftOperating Manual devotedtoFlightOperations, therearemanyreferences to"Pilots"(plural) and"CrewCoordination." Althoughthe aircraft canbe flownsingle-pilot, it wasobviousthatthe company intends it to be flown as atwo- pilot operation at least part of thetime. Yet, nowhere was it addressed howemergencies wereto be handled duringtwo-pilot operation.

11

- Aircrafteo^ipmenL Anotherexampleofconfusion in a Part 135 carrier AFMconcerned the response to a warningliglu. The instructions were as follows:"Any illumination (or flicker) of eitherCHIP DETECT annunciator light (ifinstalled) requires immediate shutdownofthe affected engine."

It is strange that an annunciatorlight soimportant that its illumination requiresimmediate shutdown of an engine couldbe placed on the list of options for anaircraft, andnot be required equipment

- Procedures. Another carrier exhibited

confusion between the AFM and the operational checklisL In the "AIR START

— NO STARTER ASSIST' checklist,one item in the AFM called for "PROPLEVER. FULL FORWARD."

The same item in the checklist from theCAOM said "PROP LEVER...

FEATHERED." Since the two are

opposite actions, we wonder which iscorrect

If flight crews areto be expected to haveconfidencein anduse checklists,the procedures that the lists describe must be

correct and must be consistent with theprocedures described in the associatedmanuals.

A lackofclarityofpurposeofthechecklistandthe AFM. An AFM is designed to presentspecific information to anoperator's personnel, including flight crews, about the operations ofthe aircraft. It is not nor is it intendedto be, a trainingmanual.This is also the casewith a checklist, which is to be used to assurepropercompletionofitemsnecessary forsafeoperation of the aircraft Despite this, someoperators use AFMs am! checklists for conveying messages which should be given intraining. Examples ofthis areillustratedfromthese instances in one carrier's checklists andanother's AFM.

- "Immediately priorto touchdown,lowerup-wind wing andalignthe fuselage withthe runway by use of the rudder."

- "Piloting with an engineinop."- "Use

rudder and control wheel to control aircraftheading,maintainingaircraftwingsessentially leveled."

- Tr«"SYNPim(syTOhrophaser)FAIL"checklistgives aprocedure foreliminat-ing the beat between the enginesif thesynchrophaser is inoperative.

Pilotsatthecareerstageof flying foranairlineshould not need basic flying lessons. If theyare notaware ofthe proper techniques by thistime,training would seem amore appropriatemeans for correcting this than a checklistIncludingtraining information in AFMs andchecklistsonly increases theirsize anddetail,and makes them more difficult to use for theirintended purpose.

The format andcontent ofa number ofthe regionalcarrier AFMs, Company Operating Manuals, andchecklists that we reviewed indicated a need forstandards and careful oversite concerning theirdesign andpublication. While some carriers providetheir crews with manuals and checklists that areaccurate and easy to use, others do not appear torecognize the importance of these documents toflightsafety. Oneofthe worst examples wasseen inthe "Emergency" checklist ofone Part 135 Carrier.These had been stamped "FAA APPROVAL" andsigned offby aPOI(even thoughnot required foraPart 135 operation) but lacked procedures for 11"Emergencies" that were in the AFM. There wereseveral carriers using checklists that were missingprocedures that were specified in their AFMs; anumberofthese involving operationally significantitems. Some of these omissions are in violation ofFAR 135.83 (c). This may be symptomatic of theregional Part 121 andthe Part 135operators, andthesurveillance given them. The interpretation of theFARs by POIs is sometimes inconsistent, and variableenforcementmay result from this. This leads topractices in the use and design of manuals and tochecklists which are questionable, and which attimes detract from the safety standards intended tobe provided by these documents.

3.3.7 Summary of Findings

• POLICY AND PROCEDURES FORCHECKLIST USE

All of the carriers had some direction for theuse of checklistsby theircrews.The policies

12

varied widely from carrier to carrier, thoughnot necessarily differing according to thecarrier's size. Some werevery detailed policies,spelledout inoperating manuals, covering allaspectsofchecklistuse, andsome wereonly notations in the margin of a checklistnoting who was to respond to a challengeditem.

Several NTSB and ASRS reports identifiedpoorcrewcoordination in the useofchecklistsas a likely contributor to aircraft accidents.The absence of detailed policies and procedures concerns the responsibilities of individual crewmembers in the use of checklistsincreases thepossibility ofpoorcrewcoordinationduringsafety-critical activities involving checklist use.

Dual responses to checklist items were usedby mostPart 121 carriers, butby onlyonePart135 operator. Many pilotsconsider multipleresponses to checklistitems to reduce safety.Checklists arc frequently done on the roll.When theheads ofbothpilots godown,evenfora moment safety is compromised.

The response"AS REQUIRED"was allowedbytwoofthesixPart 121carriersandsixofthenine Part 135 carriers. Manyrequired a specific response ofaquantity orsetting in placeof"AS REQUIRED."

• ALPHANUMERIC

The bodiesofthe checklists varied from clear,legible 10-pointtype,allcaps, withgood printquality,to six-pointtype, mixed case,difficultto read. In some cases, the type size used on"Emergency" lists was smallerthanthat usedon the "Normals." Closely packed six-pointtype is difficult to read quickly under anyconditions. Itiseasilymisreadunderthestressofemergenciesand/orunderlow cockpit illumination.The sizeand resources ofthe carrierhadnoapparentbearingonthelegibilityofthechecklist* amajorcarrierhadone ofthe mostillegible checklists examined.

• CHECKLIST PRESENTATION

Paperchecklists were most commonly usedfor"Normal" checklists,althoughone carrierused laminated cards. Another carrier used a

mechanical checklist for "BEFORE TAKE

OFF' and"BEFORE LANDING," althoughthey used paperchecklists for all other "Normal" checklists.

With one exception, "Abnormal" and"Emergency" checklists werekept inmanuals,manyofwhichwerenottabbed forquick reference.The carrier thatwas theexceptionused papercardsin color-coded folders kept in the cock-piL

• COLOR CODING

Only fiveofthecarriersusedanycolorcoding,despitethe fact thatit could facilitate locationofa critical checklist Carriers citecost as theirreason fornot using colorcoding.

• MEMORY ITEMS

Most carriers studied used memory items in"Emergency" checklists. One Part121carrierhad reduced them to one item oneachofthreechecklists, and one Part 135operator had nomemory items.

• CONTENTS AND ORGANIZATION OF

MANUALS AND CHECKLISTS

Manuals and checklists foraircraft producedoutsidetheUnitedStatesoftenhaveproblemswith language,they lack tabs, there is insufficientdetail, they contain too many modificationsandchanges,andhaveaclassificationofchecklists different from what is normallyfound intheUnited States. Inaddition, operatorsrerwrt thatchangesareverydifficulttogetapprovedby the Administrator.

There were a number ofinstancesofmissingitems on checklists,and groups of checklistsnot carried from the AFMs to the operatingchecklists.

Also, a numberof thingswhich couldcreateconfusionforthecrewsusingthemwerenoted.In some cases the order in which checklistswere listed differed from the sequence inwhich the actions should be taken, therebymaking them more difficult to use. Inconsis-temr^Ucystatementsonthehandlingofemer-gencieswereseen.And therewasone instanceof opposing actions being prescribed by the

AFM andthe operating checkliston one"Abnormal" checklist item.

The manuals and checklists of the Part 121

carriers are generallybetter than those of thePart 135 carriers, but they could still be improvedandstandardized.There are,however,majorPart 121 carriers that areworse in thisrespectthansomePart 135carriers, so it is notpossibleto judge quality only by the size andprominenceof the carrier. AFMs for aircraftflown by regionalcarriers, whether producedby foreignmanufacturersor in the U.S., wereoften not of the quality of content of thoseproduced by the large U.S. manufacturers.

Frequently, there were large discrepanciesbetweenthe contentofthe AFM and whatwas

includedin the Company OperatingManualsand checklists. Yet,therewereinstances wherethe abbreviated checklists, although lackingparts, werestamped"FAA APPROVAL"andsigned off by a POI. This would seem todemand more cautious and knowledgeablesurveillance.

3.4 ALPA SURVEY

3.4.1 Introduction

A survey ofairline pilots was done by the Air LinePilots Association(ALPA) to obtainopinionson thedesign and use of checklists from those who usethem on a daily basis. Surveys were mailed byALPAto theCentralAirSafety ChairmenandLocalAir Safety Chairmen of eight airlines, for distribution to "pilots in different crew positions and flyingdifferent aircraft, if possible." Survey questionsranged from the subject of pilots' use of checkliststo the design of checklists. ALPA promised anonymity and requested a return within a one-monthperiod. Eighty survey forms were sent out andreturned. (A copy ofthesurvey, includingimportantresults, is attachedas Appendix D.)

3.42 Respondent Characteristics

• Thenumberoftypesoftransportaircraftflownranged from 1 to over 10 per individual, withan averageof3.83 types.

13

• The axsoge. hours in each scat were:

Captain 4,140First Officer 5,570Second Officer 2,910(22 had no S/O time)

• The lowest hours in each seat were:

CaptainFirst Officer

Second Officer

03,0002,000

• The highest hours in each seat were:

CaptainFirst Officer

Second Officer

20,00010,0005,000

• Ag&ranged from 31 to 66 (the oldest being aretread Captain returning as Second Officer)with an averageageof45.78 years.

• Forty-one percent wore corrective lenses tofly.

3.4J Checklist Layout, Design, and Use

• POLICY FOR CHECKLIST USE

Ninety-three pointsix percent responded thattheirairlinesspelledoutastandardizedmethodforthe useofchecklists. (This is considerablymorethan we found in ourreviewofParts 121and 135carriers.) Almost asmany feltthatthecrews followed the prescribed method. However, when asked if the prescribed methodcould be improved upon,-almost half said"Yes." Some of the pertinent suggestionsincludedsimplification,enforcementandstandardization.

• "Simplified (checklists) to prevent'crewsnot using prescribed method', and useenforced by all levels of administrationand training."

"Responses fromaircraft(type) toaircraft(type) should be the same."

(One problem with this is thatthe manufacturers can't agree on whatthename for

14

anobjectis—i.e.,"powcrlevcr"/"»hrottle,"cta.andmanychecklistresponses arctiedto placards on cockpit panelsor aircraftmanual terminology.)

- "Dorotrequiredualresponsebythepilotflying the aircraft"

- "On two-man crews, checklists are toolong, especially final items before takeoff. And I feel the F/O (First Officer)should read the challenge and respondwhile on the ground." (The respondentwants the F/O to be responsible for allaspects of the checklists on the ground,freeing the Captain for operational duties.)

• ALPHANUMERICS

Thirty-nine percent felt it was easy, with currentchecklist typography anddesigns, to skipitems unintentionally. Although 94.5% indicatedthatprintsize was adequate,when askedlaterin the surveyifthey felt thatlarger printwould be an improvement 75% said "Yes."The fact that41% of those responding wearcorrective lensesto fly may be pertinenthere.

• METHOD OF PRESENTATION

- LAMINATED CARDS

Of those responding, 66% arecurrentlyusing laminated cards, either for their"Normal" checklists or for all checklists.

Ofthese, 20% use another form ofchecklist in addition(suchas"Emergency" and"Abnormal"checklistskeptinamanual).Eighty-eight felt that it was not advantageous to use a mix or combination ofchecklist types, such as paper and mechanical checklists.

- ELECTRONIC CHECKLISTS

The small number (six) of respondentsusing electronic checklistson CRTs feltthe CRT wassuperiorto the papercheck-listexcepton"r«ads-downtime"required.On that, three felt the CRT took more"heads-down time," two felt the paperchecklist did, one declined to answer thequestion.TheyaflfeltthattheCRTcheck-

lists were easier to use in all cockpitlighting conditions; that theywere easierto get at thatthey wereeasiertouse in alloperating conditions; thatthey facilitatedquickeruse;and,thatifiternswereskipped,theycouldbemoreeasUyretumedtothanwith a paperchecklist

The suggestionofusingautomated (electronic) checklists wherever possible metwithapositiveresponse. Fifty-eight pointsixpercemofmerespondentsfeltitwould

. be hdpful, but the following qualifications aretypical:

"No mailerhowtheyare presented, automated orclaytablet theymustbereadand followed."

(This indicates that at least one of therespondents is doubtful that reading andfollowing checklists is done consistentlyanduniformly.)

"I don't like the idea of automated ormechanical listsbecause of tiie frequentchanges to our checklists. The cost ofchanging these would make it hardertogetthe companyto makechanges."

- MECHANICAL MARKERS

The suggestion to "use a mechanicalmarker to markchecklist progress" metwithhttleenthusiasm. Many feltitwasanarchaicconceptOnesaidhealreadyusedone - "called a finger." However, injumpseat observation rides we had theopportunity to watch acrew using amechanical slide checklist for "BEFORETAKEOFF'and',BEFORELANDING."They wereenthusiastic aboutit, feltthatitprovidedapositiveindicationofchecklistprogress, and eliminated the problem oflosing one's place in interrupted checklists.

COLOR CODING

When asked if they felt "use of color codingfor easy identification of checklists" was agood idea, 83.7% said"Yes." This isused bysome airlines, both Part 121 and Part 135.Some ofthe comments elicited were:

- "Ourcurrent procedure."

- "For Emergency checklist at least"

3.4.4 Checklist Interruptions

Checklist interruptions come in two varieties:

• Interruptions to checklistuse.

• Interruption ofoperational tasksby checklistuse, suchascanoccurduringabusy approachor an emergency.

While most ofthe respondents felt thatinter-roptionswereaproblem.noteveryoneagreed.One sheltered soul said:

"Checklist procedures are not compromised bv intemmtions. I have never seen

anerror from an interruption"

Hewas,l»wever,adefiniteminorityofone,inthat respect as the following survey resultsregarding interruptions will show.

The respondents were questioned about theimportanceofpotentialinterruptionstochecklist use, and asked to rate them on a scale of 1to10,withlOindicatingveryimportantWhileafewscoredsomeofthoselistedveryhigh,theaverage scores were middle ofthe scale. Thetop-ranked four were as follows:

• ATC communications

"ATC should be eduMted/mdoctrinatedto the hazard(s) associated with multiplefrequency changes (which takesattentionfromUTedie^i^ookoutdcxanneAiavi-gating, etc.) during descent/approach(VFR and in the weathert This aim n»-

mfiyss the pilotnot flying from the 'network' atacritical time. Frequencychanging requires intense attention inside thecockpit.."

Others voiced similar sentiments:

"Most disruptive area of operation andchecklistinterruption: ATCtransmissionin initial approach area. Tjy.and read achecklistbecweenaVET(52.4milesNEofLAX)and LAXonaVFR day.Typicalto have six frequency changes, a dozen

15

transmissionswhile 'setting-up'bugs andradios for two different approaches, andbeing assigned to side-step to land on athird runway. Usually flight crewcannotrespondascontrollergoesfromonetransmission to another in steady stream ofclearances and modifications to clearances."

Ground personnel communications

Respondentsidentifiedconversationswithgateagents, fuelers, push-back crews,mechanics,etc., asdisniptiveofchecklistoperations priorto taxi.

Flight attendant requests

One respondent felt so strongly about thissourceofinterruptionthat he scored it 11on ascale of 1 to 10, and most felt that this was aproblem inatleast some phases of operation.There was noagreement onwhich phase wasmost affected. One respondent said:

"Interruptionsaremybigdeal.F/As(flightattendants)whoeitherdon'tknowordon'tcarewhat you're doing, ATC, etc. Howdo you stop that?"

Externaltaxiing distractions

Thiscoveredeverything from complexairportlayouts, to poorly marked taxi- and runways,tootherairporttraffic. A majorcontributiontothis problem is ground vehicles which do notgive way to aircraft, and over which groundcontrollers claim to haveno authority.

It has been suggested from time to time thattaxiing distractions could be eliminated bystopping the aircraftuntil the checklist wascomplete. When queried about this, about72%said"No." The following commentsarctypical:

- "Very difficult to stop and run takeoffcheck at most airports."

- "Not practical."

- "Checklists can be distractingwhen taxiing, but can be managedsafely."

16

- "Pilots are capable of responding whiletaxiing."

- "We canwalk andchew gum."The consensus seems to be that they canhandle the distractions. However, ASRS andNTSB data indicate that distractions maybemoredisruptivethanmanypilotsarewillingtoadmit

This last category, "External taxiing distractions," also contains elements of the secondtype of interruption — that of the checklistbecominganinterruptiontooperationaltasks.

Asked iftheyfelt"therearetimeswhentheuseofachecklist creates an interruption to goodoperatmgr^ocedures?",39%said"Yes." Onefelt thatduring anAbnormal/Emergencysituationheshouldhandletheproblem andusethechecklist when and if he had time. Anothersaid theproblem wasworse during taxi out

"Whilechecklist isbeing run itiseasytomiss radio calls. It is better without somuch dual response."

A report from theAll Nippon AirwaysFlightStandards Committeequotesthe 1979NASAA^B59uiQuarterlyreport,concerningchecklistsbecomingan interruption to operationalprocedure. And an analysis from that 9thQuarterly report of ASRS aircarrier distraction reports associated with checklists, foundtwo characteristics commontoallthereports.

1. "Every reportindicatedthat checklist accomplishment received cockpit priorityover ATC requirements. Every incidentended in a potentialor actualviolationofATC rulesor regulations."

2. "Thechecklistactivitywasalmostalwaysgoing on at the same time other cockpittasks were being performed; radar monitoring, minor malfunctions, system operation, traffic watch, etc Checklist accomplishmentbecame acauseofdistraction, not by itselfbut as a partofcockpitworkload. In the incidents) reported,theworkload became 'excessive' and 'timeran out' before all tasks could be com

pleted."

Clearly, the use ofchecklistsin the cockpit is required for safeoperations. Justasclearly, they must be used in an environmentthatisdisruptiveandpromoteserrorin their use. At the same time, checklistuseisanimportantcontributor tocockpitworkload. Checklists thatareeasytoreadand use will be more resistant to errorandwillcontribute less to cockpit workloadthan those that are not

3.4J5 Compliance, Crewmember Variations, andCockpit Resource Management (CRM)Oneissue thatsurfacedduring thesurveywasthatofcrew compliance. One respondentcommented:

"ChecklistsarenotthatimportantAbadcrewcan screw up a good checklist A good crewcan work safely with any checklist"

Other comments included were:

• "Checklists are mandatory for safety. However, they arc only as good as the personsreading them."

• "Personal discipline seems to be the majorvariant"

• "Don't give intocomplacency - it's our biggest foe."

Though tiie overwhelmingmajority indicated thattheir airlines prescribed methods of checklist useand theircrews adhered to them, 72.6% also felt thatindividual crewmembers influenced the manner inwhich checklists wereperformed. Sixty point fivepercent felt that this resulted in variations in checklist performance, and 43.6% felt that this meantchecklists were done in a nonprcscribed way, orwerenotdone.There appears tobe an inconsistencyintheseresponses. Althoughstatingthatmostcrewsfollowed prescribed procedure, they also felt thatindividuals had a great influence on the manner ofchecklist performance. The following commentsshed lighton the stateofcockpitresource management and crew coordination:

• "This (the lack ofstandard useornonuse) willbedifficult to correct until theattitudeofthoseindividuals is changed."

• "Ourcaptainsaresononstandardthatthe FirstOfficer's job is much more difficult Our air

line provides us with basically good procedures andchecklists,butthecaptains(particularly the olderpilots)refuseto use them."

• "Some captains continually fail to call forchecklists, leaving it up to the othercrewmembersto be a littleaggressiveandaskif they're readyfor it (the checklist)."

Whenasked if their airline had a policy of CockpitResource Management (CRM), 73%ofthe respondents indicatedthattheirairlinehadadefinitepolicy.The following comments are representative, althoughcontradictory.

• "Most 'old heads' don't even understand theconceptsin CRM, theyare fromthe schoolofZeus."

• "Ourairline hasaverygoodstandard operating procedure. Eventhough the Captain hasthe ultimate authority, all crewmembers areencouraged to actively participate in cockpitoperationsand not hesitate to voice their concerns regarding irregularities or any sort of'judgment' call."

These two respondents are apparently from different airlines, which espouse differentphilosophieson CRM. One appears to have a strong, definitepolicy which has been impressedon the crews, theothercither no CRMpolicy,or a policywhich is notbeing followed.

3.4.6 Checklist Accessibility

When queried about the checklists they currentlyuse,31(35.6% of thosewhoanswered thequestion)felt that their"Emergency"checklistswerenoteasyto locate when needed.

• "I would have to dig into my flight bag foremergency checklist handbook."

• "Emergency checklists should be red for allfleets/airlines (color coding) and should berequired bv FAA to be readily accessihle

(emphasis added) - not in binders in flightbags."

• "BAe-146 needs a place to stow both 'Normal' & 'Emergency' checklists."

• "AbrK)rma]/Emergencyinmanuals...difficultto find."

17

• "I would like to see acard(s) with the immediate action emergency procedures with theirnone-memory [sic] reference actions in thecockpit sowewouldn'thavetobe finding itina book at a critical,busy moment"

3.4.7 Other Observations

Although almost 70% said that they had a personal"must check" list which they used in addition to theformal checklists, only about 1/2 felt this would beuseful to all front-end crews. Whether this indicatedthat they felt this "must check" wouldn't work withothers,orwere reluctant to suggestimposingsomething else on other crews, was not clear.

A numberused some form ofmemory jog to remindthem to complete some items on achecklist (such aswhentaxiingwith fewerthanallenginesoperating).Examples ofthis are a coffee cup inverted over theflap handle, the checklist between the throttles, or a"post-it" note on the windshield. However, 62%saidthey just repeat theentirelist Fromtheperspective of21.5 yearsin airlinecockpits, the writer findsthis difficult to believe. We think 20% would becloser to the actual number.

When asked iftheir procedures were such that theyfound themselvesreading checklistsduring periodsofhigh workload, 62.5% said"Yes." The mannerinwhich they coped with this is cause for alarm.Whilemany saidthey stoppedthe checklist until they hadmore time, 30% saidthey "presson and hope thatnothing gets missed." To againquote JohnLauberin his Flight Safety Foundation address- "Anotherstepinvolvesthequestionofhandlingdisruptions ordistractions, someofwhich are notunderthecontrolof the crew, and others of which are. It must berecognized that any disruption or interruption ofsequentially dependent tasks is associated with ahigh probability that some or all ofthe elements ofthese tasks may be missed entirely, especially if asignificant amountoftime passes during the periodof interruption. Thus, operatingprocedures shouldexplicitly state thatany interruption to anongoingsequence of activities, especially running checklists,will automatically trigger a restart ofthe process which was interrupted. Obviously, this has tobe done in a reasonable manner, but it should be thedominantmode of operation for all pilots."

Responses to one survey questionindicatethatmostcrews followthe standard companyprocedures forchecklist use. However, when asked later whetherindividual crewmembers influence the manner in

which checklists areused, a majorityof the respondents respondedaffirmatively. The following comment is a case in point:

"Some two-man crews tend to abbreviate oruse silent checklists during high workloadtimes."

Our own cockpit experience reflects the fact thattwo-man crews tend to be less formaloperationallythanthree-mancrews, andthe above comment supports this.

The suggestion of a core checklist with allowablevariations for aircraft type and operating environmentelicitedmixedresponses.The commentsrangedfrom negative, to advisory, to positive. Some comments were:

• "A large groupof pilots will never agree onanything."

• "An industry standard checklist will accom

modate the lowest common denominator."

(This ties in with an ASRS report receivedwhichtitesafleetwithgenericchecklists.Thewritercomplained of illogical flow patternsresulting from an attempt to accommodatedifferent aircraft types, andofPA announcements on final approach.)

• "Would allowlessconfusionwhen moving todifferent aircraft"

• ''This should be donewith much input fromlinepilots. Not supervisory typesand inspectorswhodonothavetheexperience. I've beenin both situations."

The sectionrequesting suggestions from therespondents to improve checklists elicited many comments. The following representative comments arequoted as received.

• "Keep them asbrief andsimple as possible."

• "Stole of the art-deoronic checklists withthrottle interlock (for critical items such asgearand flaps) forT/O(takeoff)andlanding."(Fourof the respondents suggested someversion ofthis.)

• 'last itemsonpre-takeorf:killeritemsdouble-checked. Pan Am uses this." These would

18

include itemswhichifnot properly checked,couldpose imminent dangerto aircraft, crew,orpassengers, aswellasdamage to personsorpropertyon the groundduringtakeofforlanding. Examples of these would include fuelquantity and flaps on the "BEFORE TAKE- .OFF' checklist and flaps and gear on the"BEFORE LANDING" checklist

• "Checklistsarelikethmgs-tc^oUsts.They'reonly helpful ifyou rememberto look atthem.Checklists get forgotten in entirety. If a keyboard response wasrequired foreachitemona 'BEFORE START' checklist before theengine start valvewouldopen,thatchecklistcouldnot be forgotten, etc."

• "We haveto 'sell' theaverage linepilotthatitisprofessional aswellas'cool/manly,etc.', toaccomplish each checklist thoroughly everytime! We have to show how it will help theflight crewmember himself to do the checklist"

• "In some fleets, skinning checklist items is

routine because ofthe design ofthe checklist

That's whereeitherthechecklistortheprocedureshould he changed" (emphasis added)

• "My company management pilots need tomore strongly endorse checklist importanceand standardization."

• "Our airline has excellent checklists and procedureswhicharecarefullyfollowedbycrews.Errors stillcreepin."

• "We mustexpect errors, and plan and designknowing there will be errors."

• "We don't needanother gadget tocheckT/Owarning systems. A specific 'Killer Item'recheckis appropriate."

• "Checklists must cover a dead tired crew."

• "Brevity and simplicity."

3.4.8 Summary of FindingsFrom this survey, we may draw some conclusionsregarding checklists in everyday use.

• Larger print and/or better letter spacing onchecklists would be desirable.

19

• The small sample of respondents who useCRTs for checklists find them preferable toothertypesofchecklists.They all feel thattheCRTchecklists are easiertouse overall cockpit lightingconditions; that they areeasiertoget at; that they are easier to use over alloperatingconditions;thattheyfacilitatequickeruse;andthatifitems areskipped,they canbereturned to more easily than with a paperchecklist

Ourdiscussions with somecorporate usersofelectronic checklists revealed a negativesidetothesedevices.TheyindicatethatCRTcheck-listscanbemoredifficultto use; thattheycanrequire a greatdealofheads-downtime; andthat it is cumbersome to return to skippeditems.

• Pilotsfdtthatthecreationofa"core"diecklistacross industry lines would only meet the"lowestcommOTderomuiator"andthuswouldpenalizethe innovators andtheconscientious.

• Color-coding foreasy recognition of checklistswasreportedtobedesirableandisalreadybeing used by some operators. This takesdifferent fornis.finom coloredbordersonchecklist cards, to solid colored cards, to coloredfolders to hold the cards. Variations of all oftheseare beingusedby airlines at present

• There are many sources of interruption tochecklists. Some,suchasmultiple ATC communications at inappropriate times, are reported ascausing distractions and increasingworkloads.

• Mostoftheaiiiineswhichwerecoveredinthissurveywerereported to have a policy fortheuse of checklists which the crews followed.However, 1/2of the respondents stated thatindividuals inthecockpit influenced whetherchecklists were done correctly, oratall. Thisindicates alack ofcompliance which shouldbe addressed by theairlines.

• The survey questions concerning proceduresforusing checklistsverifyourconcemsthat infact,(Aecklistsareused inanenvironment thatprevents crews from dedicating predictablechunks of their attention tothe completionofthese lists, and thatthey accomplish these listsunder conditions that are ideal for causing

mistakes. Rather than dedicating chunks oftime to checklist use, many crews performtheselistsconcurrentlywithotherflight tasks.About 1/3of thosewho responded thattheyfound themselvesdoingchecklistsattimesofotherwiseheavyworkload saidthattheycontinued with the checklist as they did othertasks.completingchecklistitemsastheyfoundtime.

• Emergency checklists are often not easilylocated whenneeded. Itwassuggested thatitbemademandatory forthemtobecarried inareadily accessible place in thecockpit ratherthan within amanual in a flight bag.

3S OTHER SOURCES OF INFORMATION

33.1 NTSB and Related MeetingsWe participated indiscussions withan investigatorfor theNTSB and representatives ofaregional Part121 carrierwhoweredevelopinganewchecklistfora foreign manufactured aircraft that they had inservice. The carrier's people expressed theirconcerns with the manuals and checklists that are available foruse with the foreign manufactured aircraftthatthey are operating. We subsequently reviewedthe AFMs and checklists for those aircraft.

One aircraft type had an AFM that covered theinformation required by the FARs; e.g., Limitations, Emergencies, and Performance (the greaterpartof the manual was devoted to performance).There was also a Normal section which encompassed."Normal" and "Abnormal" checklists. Nosystems descriptions were included. Otherconcernsand problemsthat this operatorexpressedincludedthe following:

• One AFM contained 82 checklists for abnormal and emergencysituations. Ofthe 82,39were "Emergency" checklists. Many of the39, such as "UNPRESSURIZED FLIGHT,"wouldnot havebeen classified"Emergency"checklists by many U.S. manufacturers orairiines.However.theoperatoisarecor&nuhedtousethesechecklists asthey stand,withtheirmultiple memory items, which put a heavymemory load on their sometimes low-experience-levelcrews. We quote from anAdvisoryNotice from the manufacturerpertaining tothese checklists:

"Operators are reminded that abbreviated

checklists (as opposed to lengthy, detailedexpanded checklists) are not published by

as a document approved by anAirworthiness Authority and, iftheyare tobeused, they must comply at all times withcurrent procedures as set forth in the latestrevision ofthe Approved FlightManual."

FAR 125.75statesthat"...thecertificateholdermay revise...if the revised operating procedures and modified performance data presentation are approved by the Administrator."This regional operator told us, however,thatthey had little luck trying to modify thesemanuals and checklists. Whether due to pooroperator modifications or reluctance on thepart of the POI to allow change, we don'tknow.

• Thisaircraftsinceitsmanufacture(4+years),has had an average of300modifications peryear. Some of these modifications involvemajorhardwarechangesorpiDceduralchangesthatnecessitatechecklistchanges. Becauseofthevolumeofchanges, theoperatorhas foundit difficult to modify the aircraft, keep theircrews adequately informed, and maketimelychangesto manualsandchecklists which thenmust undergoPOI approval.

3jS2 Air Transport Association (ATA) FlightCrew Checklist Working Group MeetingsThe ATA hostedaworkinggroupon checklistandmanual design to workwiththeFAA indevelopingguidelines for use by POIs in evaluating Part 121and Part 135 manuals and checklists. This groupwas assembled to provide the FAA with industryinput for the checklist and manual section of theDraft Inspectors' Handbook. We were invited toparticipate.

Priorto the two meetings that we attended,we metwith the FAA member responsible for writing thissection of the Handbook. We provided him withdata we had found on recent MIL SPECS whichprovided guidance in manual and checklist construction (MEL-M-7700C. 18 May 1989, MEL-C-81222C[AS],22Feb.l978,MIL-C-27278B,5July1973). In addition, we advised him ofchecklist andmanual problems that we had encountered in meetings and discussions with airlines. He, in turn,provided us with the results ofthe first Flight CrewChecklist Working Group meeting, which we hadmissed. This included the progress to date on the

20

writing ofthe Handbook. Also included was writteninput he had solicited from the airline representativesregardingtheirpositionsonmanuals andchecklists, and input for possible use in the Handbook.

Since this section of the Draft Inspectors' Handbook was something which would govern theirmanuals and checklists for the foreseeable future,the airlines participated actively. Their views wereunderstandablyquite parochial, and included muchdebate on semantics, toeliminate, as far aspossible,anybut very narrow interpretations by POIs.Therewas general agreementamong the airlines that if itwerenot necessary to mention aspecific point in thehandbook, it shouldbe left out completely, ratherthan havinga general statement subject to varyinginterpretations.

353 Jumpseat Observation RidesWe took jumpseat observation rides on seven occasions, on four different airlines. We did this to seehow checklists were actually beingused in flightThe aircraft flown included two DC-9s, a MD-80, aDC-10, aL-1011, a B-727. and a Saab-340. None ofthe aircraftused acomputerized checklist on aCRT.All usedpaper"Normal" checklist cards invaryingsizes. On three aircraft, a mechanical checklist wasused for the"BEFORE TAKEOFF' and "BEFORELANDING"checklists. The crews usingthese mechanical checklistswerehighly in favor ofthem.

The mannerinwhichthechecklists wereperformedvaried widely. Three crews from the same airlineperformedinauniformmanner,indicatingthorough,standardized training.Two crews ofanother airlineperformed in a loose manner—sufficiently loosethat one of them never ran the "BEFORELANDING" checklist

It appeared, from these jumpseat rides, that theperformance of checklists in an airline that has astrong emphasis on training and standardizationwill be more likely to be uniform. Where lessemphasis is placedon those factors, and less discipline prevails, checklistuse willbecorrespondinglymore variable.

35.4 Corporate On-Site VisrrsCorporate aviation often makes use of the latesttechnology before the airlines, since corporationsare notsubject tothe economicconstraints imposedby alarge fleet They also frequently carry executiveswhoselosstothecompany inan accidentcouldbe critical. We believe this colors their thinking

regarding technology vs. cost decisions. Interestedin this tendency to use the newest equipment wemadeon-sitevisitstotwo corporate aviationdepartments toassesstheircurrentchecklisttechnology.Apeculiarityofcorporate aviationdepartments is thatthey can change their checklists whenever theywant as they see fit, and without priorapproval,since they operateunder Part91.

One corporation flew two Canadairs and oneWestwind. All three aircraft at the time ofour visitused a backlit fold down, scroll checklist for all"Normal" checklists. This was mounted in the centerof theglare shield. The pilots reported thattheylikedit asthey alwaysknew wherethey werein thechecklists,regardlessofinterniptions."Emergency"and "Abnormal" checklists were carried inthe cockpit in a laminated, color-coded, well-tabbed booklet prepared by Flight Safety Canada, Inc. Thisbooklet alsocontained backup "Normal" checklistsforuseifthescrollwasinoperative.These "Normal"checklists were not as comprehensive as thecorporation's own, used on the scroll. All threeaircraft have the capability of upgrading to automated checklists on CRTs, and the corporationstated their intent to do this in the near future. Sincethechecklists wouldusurptheradar presentation, inbad weather the crew would revert to the scrolls.

The othercorporation hada larger aviation department encompassing aGulfstream G-4, a Westwind1and 2, a Beech King Air, and a numberof Bell JetRanger and Bell 222 helicopters. The fixed wingaircraft all require two pilots. The only casewherea rotary wing aircraft requires two pilotsis the222in EPR weather.

All their aircraft used laminated card checklists,despite the fact that the Westwind 2 had checklistsavailableontheradarCRT.Thereasongivenbythechief pilot was standardization.He also felt that theCRT checklists weremorecumbersome touse,andtook more time.

The G-4 will have the automated checklists installedinitsSperry. all-glass cockpit this year. Itwillhavea dedicated CRT. Whether that installation willsupplant the laminated cards remains to be seen.

The rotarywing aircraft crews did not use availablechecklists when underway. The only check normally done when underway is an engine gaugecheckondescent During an engine lossortail rotorfailure, thecrew is toobusyto read achecklist. We

21

were told that they deal with "Abnormal" procedures instinctively, from an ingrained habit, andthen refer to the Operations Manual kept in theaircraftChecklists are alsonotusedintwo pilotIFRflights, where each pilotknowstheStandard Operating Procedure and follows it when underway.Although we anticipated that we might find examplesofthe latesttechnologyinchecklistsinthesevisits, we did not. As noted above there was someinterest in automatedchecklists on CRTs, but forthemost part more conventional types were the standard.

355 Cockpit Devices in UseIn order to determine whether there was some newtechnology available whichcouldbeeasilyadaptedto general use, andcouldhelp toeliminatechecklisterrors, we did asmall survey ofwhat was available.From the results of this survey, we have listedadvantages and disadvantages of the various kindssurveyed (see Appendix B).

The automated checkliston aCRT is likedby manyof those who use it Some who use it on a regularbasis and report favorably onitalso report that it cantake more heads-down time if anything unplannedoroutof the ordinary occurs. Others report it astoocumbersome anduse paperor laminatedchecklistsinstead, even when tiie other technology is available. In some cases, it usurps the radar CRT. Manyaircraftwouldrequire averycostlyretrofit toenablethe use of this technology.

The checkliston ascroll hasbeen around formanyyears, and is still used enthusiastically by many,including crews of some Air Force planes in thecurrent inventory. It can be cumbersome to use ifone needsto returnto apriorportionofthe checklistIt also takes up cockpit space; which is in shortsupply inmany aircraft Inaddition, itneeds apaperchecklist backup in case ofmechanical failure. Onecorporation we visited used scroll checklists thatwere generated on a personal computer with a dotmatrix printer—not the best combination for legibility. Their checklists did not require approvalfromaPOIsincecorporationsoperateunderPart91,andthis allowed them to make changesas they sawfit Their preflight checklist contained 129 items,and other checklists also seemed excessively long.

By far the most prevalent types of checklists arepaper or laminated paper. They come in varioussizes and shapes, some big and unwieldy, some sosmall as to be unreadable except in perfect condi

tions.One majorproblemwith these is the easewithwhich you can lose your place through interruptions.

We observed thatmechanical checklists are used for"BEFORE TAKEOFF' and "BEFORE LANDING." Theirusers likethemsince theyareapositivemeasure ofchecklist progress. The other"Normal"checklists thatthecrews useare paperorlaminatedcards.

We haveseenoneexampleofaunitwhichreads thecheckliststo the userin asynthesizedvoice. It willrestate missed items until they are complete, ifprogrammed to do so. As far as we know, it iscurrently only in limited use, with some corporatePart 91 operators. Onemajor airline is consideringdoing an evaluation ofthis technology withan eyeto possibleuse. One drawback that we can foresee isthe addition ofanother noisein cockpitswhicharealready noisy enough.

Some users kept all checklists in booklets in thecockpit. Some checklists were partially laminatedthroughout, somewereinplasticsleeves.Thosethatwere well tabbedandindexed wereeasy to use. Oneof the best examples of these was the checklistbooklet from FlightSafety Canada, Inc., foruse inthe Canadair Challenger601. This included color-coded, laminated tabs, well-indexed "Abnormal"and "Emergency" sections, and heavy, hard-finished paper pages with 10-point type or larger. Itwas easy to use and very legible. Moreover, theaircraft forwhich it was designedhad aconvenientstorage slot for it; its compactness would make iteasy to adapt other aircraft to accommodate it

The worstexample we saw was thatofthechecklistbooklet from the Horizon DHC-8 involved in anaccident at the Seattle-Tacoma International Airport, on 4/15/88.It was printed in eight-point type,mixed case(sometimesalllowercase),andnot goodquality of print The tabbing can best be explainedby quoting from the NTSB "Human PerformanceInvestigator's FactualReport" ofthe accident:

"Locatingaspecificchecklistrequirestheuserto identify the desiredchecklist in the tableofcontents, note the number of the divider atwhich tiie checklist is filed, and turn to thedesired checklistwhichis inserted before (forward of) the numbered divider."

22

In a drill, at an informal meeting with the NTSB, aDHC-8 Captain was asked to locate the "ENGINEFIRE" checklist in the Horizon booklet He wasunable to do so in a reasonable amountoftime. Thisinability to locate criticalchecklists is perhapsonereason why the "ENGINE FIRE" checklist wasnever completed in the Horizon accident.

35j6 Summary of Findings

Apart from paper and laminated card, no checklistdevices were found which wereeasily adaptable toall aircrafttypes. And, one respondentto the ALPAsurvey commented that the aircraft he flew didn'teven have a place to stow them.

As far aswe cansee,no manual devicecurrently inuse hasthe potential, by itself, to entirelyeliminatepilot errorin the use ofchecklists.

4. SUMMARY AND RECOMMENDATIONS

This includes a summary ofthe data gathered andrecommendations for improving checklists.

4.1 FINDINGS

4.1.1 Conformance

Twenty of 21 NTSB reports illustrate that lack ofconformance with standard operating proceduresmay be as big a problem as checklist layout anddesign, if not bigger. Forty-three percent of theASRS reports indicatethatalackoftrainingcontributed to this lack of conformance. Comments byALPA support this indication. We observed aninstance of this during one of ourjumpseat rideswhere thecrew did not read their"BEFORE LANDING" checklist

Theinconsistent application of policies and procedures for checklist use may also adversely affectconformity. Some operators were very specific intheguidance they gave their crews, others gavenodirection oneitherpolicyorprocedures forchecklistuse. The latter were frequently vague as to whochallenges,who responds,andwhen.

4.12 Interruptions

Fifty-eight percent ofthe ASRS reports mentionedinterruptions as being the cause of problems inchecklistuse.Theinterruptions fall into two categories:

• External interruptionstothecrewduring their

use of a checklist

• Interruptions to operational tasks causedbyusing a checklist

The ALPA survey confirmed the disruptedanddisruptingaspectsofchecklist use anditsimplications for flightsafety.We also observed that operations activitiesoften led to checklists being done frommemory, responses being given without thecorrespondingactionbeing taken, andchecklistitemsbeingmissed. Similarly, ourcockpitobservationsrevealedthatdiligentuseofcheck-listsby flight crewswhiletaxiingcouldeasilydetract from the safeoperation ofthe aircrafton the ground.

4.1J Checklist and Manual Design, Organization, and ContentsMissing,inconsistent, andincorrectprocedures weresaid to contribute to 20% of the problems in theASRS reports. In fact, we found many of theseproblems in our review of Part 121 and Part 135operators' manuals and checklists. And many ofthesemanuals andchecklists alsolackedorganization and the completeness needed to support informeduse by flight crews.The manuals andchecklists provided by large U.S. manufacturers wereusually more organized and easier to use than thosefrom foreignor small U.S. manufacturers.The lackoforganizationandclarityin themanualsandchecklists from the smaller and foreign manufacturersoftenpresented a problem forregional carriers flying the smaller, commuter-type aircraft. However,even the manuals and checklists from large U.S.manufacturers suffered attimes from changesmadebytheoperators.Thisresulted inanendproductthatwas no better, and occasionally worse, than whatwas available to small carrier crews.

Examplesofthe problem foundincludedthe following:

• checklist procures not m theorderin whichthey should be used;

• items missing from checklistsand/ornot carried over from the AFM;

• procedures specified in the Airplane FlightManuals (AFMs) inconsistent with actionsprescribed in theoperating checklists;

23

• wholesetsofproccduresitttcamedovertromthe AFM to the operating checklists;

• incomplete procedures;

• checklists difficult to locate in manuals eitherbecause of poor tabbing, poor indexing, orpoor titles.

4.1.4 Readability

The typography of manuals and checklists variedwidely, from five-point type to 10-point type orlarger, the smaller type being difficult to read. Oftenprint was blurred, and contrast of print to background poor, despite the obvious fact that if manualsand checklists are difficult to read, they will bedifficulttouse.TheAirCarrierOperationsBulletinPart135 No. 88-5 - FlightCrewChecklists (NTSBSafety Recommendation A-88-72.) says:

a. "The National Transportation Safety Board(NTSB) in their investigationof a commuterair carrieraccident discovered that the flightcrew checklist was not constructed in such amannerthatwouldprovideadequatelegibilityin normal or emergency conditions. NTSBbelievesthatunderoperationalcircumstances,adeficiencyinlegibilityandsizeofprintcouldcompromise the intended use of this device.

b. Principal operations inspectors should takeappropriate actionsduringthe courseof routine air carrier surveillance, inspections, orflight checksof their assigned operators forreviewofcurrentchecklistformat Flightcrewchecklists used by air carriers should includethe appropriate actionsnecessary for normaland emergency procedures, printed in clear,concise, and legible form."

Althoughdirected at Part 135operators, thisappliesto all operators. The regulationsshould be changedto reflect the same standards for Parts 121 and 135operators. The current regulations reflect a lack ofclear and consistent direction for manufacturers,operators, andPOIs alike.The manufacturers shouldhave clear guidelines to follow in producing usablemanuals and checklists for new aircraft The operators should have clear manuals and checklists fortheir crews. And the POIs and evaluation groupsshould be given unambiguous guidance on whatstandards to apply to the design of manuals andchecklists.

4.1.5 Color CodingColorcodingofchecklists andmanuals isused verylittle, although itcould facilitate locationofacriticalchecklistThe airlinesusuallycite cost as the reasonfor not using color coding.

4.1.6 Inconsistency

Often there was a lack of consistency betweenAFMs and checklists. In some cases, checklistitemsand even some procedures were not carried overfrom the AFMsto the operating checklists.

4.1.7 DHiNmoNOF"ABNORMAL"and"Emergency"The use ofthe terms "ABNORMAL" and "EMERGENCY"were inconsistent amongmanufacturersandoperators andfrom aircraft typetoaircraft typewithinthe same operator's fleet.The use of "NORMAL," "ABNORMAL," and "EMERGENCY" issometimesinconsistentthroughoutafleetThetermsthemselves vary, with the terms "NONNORMAL"and "IRREGULAR" used somewhat interchangeably with ABNORMAL" and "EMERGENCY."but there are also differences in meaning.

The lack of a standard definition for "emergency"hascreatedparticularproblemsforchecklistdesign.Excessive numbers ofemergencies result in emergency checklists ofextreme length, excessive numbersofmemoryitems,andinconsistent responses torealemergenciesthatarenotalwayssolabeled,e.g.,loss of all generators. One foreign aircraft that had39setsof "Emergency"procedures, manyofwhichwould have been classified "Abnormal" by majorU.S. manufacturers. Inflight events that are classified as emergencies (for example, low-levelunpressurized flight) in one aircraft type but notanother in the same fleet reduces the flight crews'respect for the term and contributes to their confusion regarding their priorities for action.

4.1.8 Emergency Checklists

"Emergency" checklists are sometimes difficult tolocate when needed. They are often in manualsstowed in flight bags and are reported to be difficultto retrieve.

In some cases in our study, we encountered groupsof "Emergency" checklists that had an excessivenumberofchecklists (39 in one case).Thismadethechecklists cumbersome to use and made it moredifficult to find a single checklist

4.1.9 Heads-Down TimeThe use of CRT-presented rather than hand-held

24

checklists may be expected to increase flight crewheads-down time. This, coupled with the amount ofheads-downtime necessary forreprogrammingcomputers when changes of routing are received, couldcause important decreases in the capability of thecrewto concentrateonotherduties such as monitoring traffic.

4.1.10 Summary of Factors Detracting fromGood Checklist Design and Use

Flightdeckobservations.pilotreports.relevantaviation safety databases, and our review of checklistsand handbookscurrentlyin use by some aircarriersindicate:

Operational conditions andpriorities limitthetime available to flight crews forexaminingchecklist items.

Useofchecklistsinvolves flightcrew heads-downtime thatcanbedangerousduring terminal operations.

Some flightcrewsonly usechecklistswhen itdoesnot slow down otheraircraftoperations.

Regardless of time available, some crews donotuse checklistsduringsomeoperations forwhichlists are provided.

The print on some checklists is difficult toread under poorlighting.

Responsibility of individual crewmembersconcerningchecklistuseisnotalwaysclearorwell defined.

Thetypesofitems includedonchecklistsvaryamong carriers.

Someinflighteventsare considered emergenciesby somecarriers butnotby others.

Emergencychecklists andhandbooksarenotalways quickly accessible to the flight crew.

It is difficult to quickly locate emergencyproceduresinsomechecklistsandhandbooks.

Procedures indicated on some checklists areinconsistent with those described in the companionflightmanual.

• Some checklistsdonot includeprocedures forall common emergencies.

• In some cases, the size and formatting ofemergency checklistsmakes them more difficult to read than normal checklists.

42 RECOMMENDATIONS

We did not collect sufficient data to determine ifpoor checklist design and poor habits in the use ofchecklists were widespread throughout the industry. However, our data do support the conclusionthat there are Parts 135 and 121 carriers who areoperatingwithpooiiydesigned checklistsandmanuals, and who have flight crews who are not welltrained in the use ofthese aids and who admit to notusing them when they are expected to.

Accordingly, we make the following recommendations regarding the design and use ofchecklist andmanuals. We also recommend supporting researchand development activities.

4.2.1 Checklists

• "Normal"Checklistsshouldbe shortandeasyto use. They:

- Should includeonly those items that arepertinent to the safety andcontrolof theaircraft

- Should be listed in an order that mini

mizes heads-down time and the attentionofmore than one crewmember at a time.

- Sublists, eg., "BEFORE TAXT checklist and "AFTER TAKEOFF* checklistshouldappearon the checklistcardintheorderin which they will be used.

- Shouldhave selectedsafety critical itemssuch as gearand flaps as final items on"BEFORE TAKEOFF' and "BEFORELANDING" checklists, even if this repeatsanearlieritem in the checklist Thiswill facilitate quick andlast-minutereference to these items.

- Shouldhave alphanumerics ofsufficientsize, clarityof print, and contrast to beeasilyread underanyillumination conditionslikely to be encounteredin the cock-

25

pit In the absence of cockpit researchdealing specifically with this issue, werecommend, in "Guidelines" (AppendixA) that the checklist body be 10-pointtype, boldface,allcaps,andthatthechecklist title be 12-point type, boldface, allcaps.

- To the greatest degree possible, shouldhave no greaternumber ofitems thancanbe presentedon asinglcchecklist cardandcan be easily readandstowed in areadilyaccessibleplacein the cockpit

"Emergency" checklists should be quick toaccess and easy to use under stressful conditions. They:

- Should be quickly accessiblein the cockpit by both the Captainand FirstOfficer.

- Should be available on a card (on thereverse ofthe "Normal" checklist card if

possible) as well as in the manual.

- Should be in a standard format The orderin which the emergencies are presentedon the card should be standardized. Thisshould cover all aircraft types in acompany's fleet and should take a formsuch as all engine problems first or allfires first etc., (to be decided by eachcompany). In this manner, a crew flyingforaparticularcompanywill know whereto look for individual checklists regardless of what aircraft they are flying. Inaddition, the order in which the procedures are presented for each emergencyshould be standardized to the greatestdegreepractical, particularly withintype.

- Should have a clearly defined start andfinishwith atitle set offby type two sizeslargerthanthatofthetext boldfaced,andallcaps. Eachlistofproceduresshouldbeclearly separated from other lists. Thisshould facilitate quick identification underconditions ofstress and tow illumina-tioa

- Should be composed of type no smallerthan that of well-designed "Normal"checklists, and if space permits, larger."Emergency" checklists are often used

undercircumstancesofenvironmental andpsychological stress, and consequentlyshouldbe as readable as possible.

- Shouldbeeasytounderstandandexecute.Each "Emergency" checklist should becomposed ofonly those items needed tocombat the emergency.They should belistedin theorderin whichthey are to beperformed.They shouldbe statedincommon teiminology, in a positive manner,and in as few words as can be used toconvey the action.

Subsequent procedures which must beperformed as a result of the emergencyprocedure, (e.g., "SINGLE GENERATOR" procedure after a generator lossdue to shuttingdown anengine asaresultofanengine fire),shouldbecoveredintheexpanded checklists in the manual.

422 Manuals

• Procedures for checklist use:

• Should be clearly defined in the manual.This should include clear direction as towhich flightofficer reads whatchallengesand which responds, andshould specifythis foreach phaseofoperation; i.e., air-planestationary.airplanetaxiing,airplanein the air.

- Should require quantitative ordiffeientiatmgresponsesforallappropriatechecklistchallenges. Wheneverpossible,responses should specify position or.quantity; e.g., FLAPS....20,FUEL.....48,00O#, etc. The answer "ASREQUIRED" should not be allowed.

- Should limit dual response items to thehighest prioritysafety critical items.

- Should require thatchecklistswornto thepoint of reduced readability be immediately replaced. No Minimum EquipmentList (MEL) delay should be allowed onthis item.

• Formatrequirements:

- Should specify a standardized table of

26

/contents,includingclearreference to thechecklist sections.

- Should include tabbed dividers for sectionsthatmayhavetobeaccessedquickly.For checklists, these should include standardized, color-coded tabs, by section("Normal," "Abnormal," and "Emergency") and appropriately labeled tabswithin each sectioa Each section shouldbeginafterthe tabwiththefirstpage beinga clear, alphabetized index.

423 Checklist TrainingThe required training curriculum for each airlineshould incorporate checklist training, including:

• Proper use ofchecklists.

• Crew coordination in the use ofchecklists.

• The necessity forcompliance withchecklists.

4.2.4 Review of FARs

This review should be conducted to determine theneed for

• A clear definition of "NORMAL,""ABNORMAL," and "EMERGENCY." IfnotaccomplishedbyFARchange.this shouldbespecifiedinanAdvisoryCircular.Thiswillstandardize the use of these terms for bothmanufacnirersandairlines,andshouldprovidethemeans to design "Emergency" checklistswhich are similar in length and content Atpresent some manufacturers include in their"Emergency" checklists manychecklists thatwould be considered "Abnormal" by others.This has resulted in some "Emergency"checklists ofexcessive length.

• A rewrite of theFARs, oran AdvisoryCircular,to indicatethatmanualsandchecklists forPart 121 and Part 135 operators have essentially the same, well-defined basic requirements. This should include all stages frominitial approvaltooperatorrequested changes.Those partsnotrequired bythescopeofoperation of smaller Part 135 carriers could beeliminated.

425 Research and DevelopmentResearch and developmentshould beconducted to:

27

Establishquantitative and behavioral criteriaforchecklist accessibility and readability.

Developaprototypechecklistforuseby safetyinspectors forevaluating aircarrierchecklistsand flight manuals.

Develop andevaluatethe usefulness ofastandard format organization, and table of contents foraircraft flightmanuals.

Evaluate the use of all caps vs. mixed caselettering in checklistdesign.

Develop and evaluate the use of a standardterminology forcontrols,displays, andinflightoperations in checklistsand flight manuals.

Evaluate theutility,safetybenefits,andlimitsof audio checklists, checklists on CRTs, andchecklists withartificial intelligence features,bothin a laboratory setting and in anoperational context (There is currently an audiochecklist design available from Heads-UpTechnology thatwillbe thesubjectofastudyby a major airline.)

Evaluate the benefits of color coding anddifferent font styles on checklist readabilityforelectronic aswell aspaper checklists.

Evaluate theoperational feasibility of safetycritical checklist item interlocks that wouldpreventaircrafttakeoffwithoutcompletionofsafetycritical items.

Evaluate theutility, safetybenefits, and limitsofmechanicalchecklists suchasthoseusedbya major airline for "BEFORE TAKEOFF'and "BEFORE LANDING."

Developandevaluateaprototypechecklist forParts 135 and 121 use. This list would bedeveloped asanexampleofhow human factors principles in theuseof formatting, fontsize,andcolorcodingcanbeapplied tochecklist design.

Detennine theinfluence of memory items onemergency checklists onthe speed and accuracy with which emergency procedures areperformed.

APPENDIX A |

Checklist Guidelines

r

CHECKLIST GUIDELINES

Theneed forasetofstandards toguide manufacturers and airlinesin developing manuals and checklists is becoming more and more apparent Anyproposed guidelines would have to encompass anumberofareas, suchasprintsizeandstyle, format,colorcoding, overall coloruse, brevity, clarity, etc.Another area of concern is readability under allconditions ofcockpit lighting, from bright sunlightcruisingat altitudeto night flightwith low ambientcockpit lighting. Although supplementary lightingwould normally be used in the latter case, too muchwhite light will temporarily destroy night vision.

Bearing these points in mind, the following set ofguidelines areproposed as the first stepin thefinaldevelopment of a setofstandards for industry use.

PRINT SIZE AND STYLEFigure A^l shows two extremes of print size andstyle. The first is a copy of the actual checklist on aJetstream 31 involved in an accident in New Orleansin 1987. It is representative ofthe size and style ofprint used in the checklists ofsome smaller carriersand isclearly too small (0.075") and tightly spacedfor adequate legibility under the range of lightingconditions which anaircrew will normally encounter. Figure A-2 is a copy ofthe actual checklist on anMD-80 that was involved in an accident in Detroit in1987. The print is the same size as that of theJetstream 31checklist, andalthough it is formattedbetter, we still find it too small for easy readabilityin all lighting conditions. The second example inFigureA-1demonstratesthe recommendationmadein the Human Engineering Guide to EquipmentDesign, for use if any lightingconditions less thanone-foot candle can beexpected. Although highlylegible, the letters are too large (0.20") for practicaluse.

What we recommend is between the extremes citedabove and finds its basis in MIL SPEC recommendations and current applications by a number ofmajor airlines. An example is shown in the DC-9checklist in Figure A-3. In that example, the printsize is0.15" (14 point) fortheprimary heading (DC-9 NORMAL...); 0.125" (12 point) for the checklistnames (i.e.,BEFORESTARTINGENGINES); and0.1" (10 point) for the checklist text It is also doneinallcaps,boldface type, withtheexception ofthenotes, which are in initial caps with lowercasefollowing. MIL-C-81222C and MBL-C-38778Aspecify theuseof14-point (0.15") type forchecklistheadings, and the use of 12-point (0.125") type for

the body ofthechecklist Both of these are slightlylarger than that used in the DC-9 checklist andappear to represent a good compromise betweenlegibility and practicality.

From the practical standpoint, theuse of 12-pointtype (0.125") throughout the text of a documentresults in54lines of type, with 1"margins topandbottom,onan81/2" x 11" page(i.e.,thesizeusedinthis report). The size shown in example two ofFigure 1(0.2")resultsin 29 lineson an 81/2" x 11"pagewithlessthan1"margins topandbottom. Sincemany checklists contain more than 29 items, thiswould result in an increase inthenumber of pagesrequired to accomplish a checklist. We feel thatnormal checklists should be kept to no more thanone 8 1/2"x 11"page — either laminatedor trifold— if a card checklist is to be used. The reasons forthat are as follows:

a. Many pilots clipthechecklists to theyoke orparts ofthe window apparatus foruse.Thisiseasy with one page — more than one pagebecomestoo bulky.

b.Having toflip through more than onepage toreadnormal checklists inamultiple-legdayiscumbersome.

c. A checklistof one page can be found moreeasilyandquickly.

d. A single-page checklistis easier to stow andretrieve when needed.

e. Wefeelthatanythingthatpromotcseaseofusewith a checklist will discourage misuse, orneglect of checklists.

Based onthe above, ourrecommendations forprintsize and style are as follows:

1. CHECKLIST HEADINGS — 12-point(0.125") type, all caps, boldface, ina typefaceequivalent to those recommended in the MILSPECS.Theseshouldbeblacktypeonawhitebackground,orwhiteletteringonadark background.Thelatteris recommended in MTL-C-1472C, in "Human Engineering Guide toEquipment Design,"and iscurrentlyinusebyFlight Safety Canada, Ltd. in their Canadairchecklists.FlightSafetyvariesthebackgroundaccording to the typeof checklist: white for

A-2

FIGURE A-1. EXTREMES OF PRINT SIZE AND STYLE

tEFORE TAKEOFF IFIWrU 1TBC)—I. UlndsMtid feat - W

2. Pilot HMt - ON3. Trtftsponstr • OR4. Oil Cooler Haps - CLOSED/TOTS RNML5. limits - AS REQUIRED

CM. Ic* Protection - AS REQUIREDCR7. Flow Selectors - OFFCRB. Speed Lmrs - 10W tMEN aEARED

JEFONE TAKEOFF (FIHAL ITPS)i. mndsnieitf neat - on

2. Pilot Heat - ON3. Transponder - ON4. Oil Cooler Flaps - CLOSED/TDK NftWU.5. Lights - AS REQUIRED

CR6. Ice hrotection - AS REQUIREDCR7. Flow Selectors - OFFCRB. Speed Levers - 100% WHEN aEARED

A-3

normal, yellow for abnormal, and red foremergency. In the interests of economy, theusers maywishtostaywithblack letteringona white background,however, the whiteon adarkbackground we havefound to be easilyread under all light conditions and we recommend it

2.CHECKLIST TEXT — 10-point (0.1") type,allcaps, boldface, ina typeface equivalent tothose recommended in the MIL SPECS. Thisshould be black lettering on a white background.

3. NOTES — 10-point (0.1") type, initial caps,lowercase following, ina typeface equivalentto those recommended in the MIL SPECS.This should be black lettering on a whitebackground.

If space and economypermit, we recommendmoving up to 14-point type (0.15") for checklist headings and 12- point type (0.125") for checklist textandnotes.FlightSafetyhasdonethisintheirCanadairchecklists and it produces superior readability.

FORMAT

We recommend a formatofchallengeandresponse—consisting of the query to the left margin, followed by a dotted separation, followed by the required response (to be right justified). This is thespecifiedformat in MIL-C-81222C, is quite common in industryuse, and is illustrated in FiguresA-2 and A-3.

COLOR CODINGThroughout the industry the use of color-codedannunciator lights is standard — red indicates"WARNING" or danger, yellow indicates "CAUTION,"greenindicatessafety.FlightSafetyCanada,Ltd. and some air carriers have carried this colorcoding through in checklistuse. "Abnormal"checklistsareidentifiedbyheadingsofyellow,and"Emergency" checklists by headings of red, with the"IMMEDIATE ACTION" items boxed in red.

We recognize that to do this is more costly, but werecommend it strongly. Color coding such as theabove lends itself to ready identification, and henceease of use.

OVERALL COLOR USEThe MIL SPECS previously quoted specify the useof black type on white paper, with the exception ofthe checklist headings recommended to be white

print on a dark background. However, a limitedstudydone by the headof the Publication Department of a regional carrier, in conjunction with anoptometrist indicates that better readability is attained undernormal variations of ambient cockpitlighting by the use ofblacktypeon a brightlemonyellowbackground. This wouldappearto be borneout somewhat by the studydone a number of yearsago by big city fire departments which led to newequipmentbeingdelivered withbright yellow paint.They found that the equipment was more visible toother drivers with that paint scheme than with thestandard fire-engine red. Once again, economicsentered thepicture, andmostfireequipment is stillred.

Wehaveseentheresultsofthe regional carrierstudyand agreethatit promotes better readability underavariety of ambientcockpit lighting conditions.

BREVITY AND CLARITYThe following is a quote from MIL-C-81222C:"...procedures shall be presented in checklist form,abbreviated from the amplified checklistor proceduresintheNATOPS FlightManual.This abbreviationis to beaccomplished by omitting explanatorymaterial and reducing the check item to the minimum necessary to describe the required action. Forexample, the step 'Reduce airspeed to 130 knotsIAS for best glide' can be abbreviated 'Airspeed -130 KIAS Glide'." MTL-C-27278B says: "Theprocedures of the checklist shall be derived byabbreviating the procedures and eliminating theamplifications of the procedures in the proceduresections ofthe parent manual..."

As indicatedby the above, no ambiguity or excessverbiage should be allowed in checklists. The required items and no more should be covered. Onechecklist studied had 139 itemson the"AIRPLANEACCEPTANCE"checklistThis isexcessive.Theseitems should be checked on a defined preflight butto cover every item on a preflight in a checklist is tocourt checklist neglect by crews.

LEXICONStandardized terminology, consisting of commonaeronautical terms, should beused in all cases. MIL-M-7700C says: "Standard terminology. In mostcases, use the terminology for equipment that isconsistent with the intended operator's standardusage and is preferable to some of the more technicallydescriptivenomenclature [sic].Someexamplesare: 'throttle' vs. "power control lever', 'circuitbreaker' vs. "fault circuit detector'..."

A-4

MD-80CIIIMM. CUCTIIC ft PNEUMATIC ttUfttt • HAM

muhatic' i-ruDt »cmi cuouMUNATIC All MUM COMMHTU ft ONmimatic mum orwmimatic nummt lit r»i mn> eneomm • urou mar aoonjir

ATTU BBINH ITAIIUtIBMllttTIC MUM Iiifcnie toutUTUMAL IUCTIK ft MIMATIC ...coNntn • irm rait cmcuist

•BMESTmT

NINBMHU MATtn*0M

*<uuq>«UT

•MA CDCAIWIaim arauuue iwr ft ruuuu

ClftCUIT I

•ADIM, ALTUnUS ft lUOT Ml .... -CO ft ICTronton. —lauMgiTiu) ftimr

ICNITIM OMUATULTIUM CM

Ann stmtciATca en

imitim •orrtucntcroaa *cnAN All HU UQ)All COMBSTIMINO MJMU MITCHU •AUKmimatic Mm -M CUHUTBAMIU flMP ft muUUC ITfTU* .... «M ft CO)

FIGURE A-2. MD-80 CHECKUST

1AIITUM •MUTTIM)TUN »(UTTtNC)IN ft AIUKU MCI ••(UTTIMCt)AITS IISHA)rUCNT IMTMMCim ••(HOC) ft SUVINCCONTaOU ft IUVATM NMU «C*P-TOT

CU'IOTTM

IIAUS ft

AMMCIA1ICNITIONructllc

AW All .All CONDI

MUTED ENSME SIMT

MUTU AFTU ITAIT

TIONINS VmVt MITCKU . . •Aim

tMSINI ANTI'ICC ft NU IUT IAS UQ)MtURATIC MUM •clousAMI •(AS ICC)

•CFME TAKCWFrtlOTT ATKMMNT

AMUCIATMICNITIM ...

•Nortriia

CD

. M

aimNO WOO SIGN *<AS UQ)ICNITIM *(AS MQ)rati raw •(*» uq)CMIN nUMU CONTMUU -CD•mc •«HTMAUUC MM* •OCT 4 UNrur TABorr uuctm

IN IIACCALTUKTUS •MUTTINS) ft I-«DUl >(CA>AIUKU MB ••(UTTINB)UATKITIIGN *MCAIIN RUtUU CONRBUU •CBIitmauuc rain •ONiaica

A-5

KTOai UbolNBNO MMX ttCN ! •«ICNITIM >Mrun mm *ur roi unoim•we «orrcua •nuicuuiroiuit •AINU

AUTO MAUI (UUQ>

run •(uttimi*

AFTER UKMK3WOIUU •KM)run *<u uq)ict noncriM «orrimitim •errAfU •WW)mumtic x-rua •ok orwIAMI ft TIAMNMU •Ott ft RANMTAfO All ft IIXCTIIC KM! *(U UQ) ft OS

rMKINfiAll COMDITIMIHO aUMU MITCUSIMIMATIC 1-rUMnu CONTMUUAT lltT AIM

CNOCUft MAUIAutftnAwna nunrun nun •(aiuq)NIMMItU UAT >(M UQ)IMMM INTIT (U UQ)

TOMHAIINBtiaaccNar uean «orrrrraruAT •orrAll CMDITIMIW rANU *(U UQ)UDIM ft CAUIT r —

•err ft auto

<UW)

MCUtlNB AIMAJK AT MM MM STATIONSAW ..•••.. MM

oumMCMraot"!'. '."'.'.'.I'.'.'.'.'.'."'.'.'."*•(«uq>iattut wiTca * «orr

Included in this standard terminology should be adictionary of abbreviations to be used wheneverabbreviations are needed.To quote MIL-M-7700C:"The glossaryofeachmanual shallcontain alist ofthe abbreviations used in the manual, except fornormally accepted and understood abbreviationssuchasac, dc, and rpm." Although theMIL SPECmentions "manual" specifically, the same wouldapply tochecklists, since they derive from the flightmanuals. In MIL-M-7700C there is a list of approved abbreviations, and MIL-STD-12D is dedicated to abbreviations. Some of them are differentfromthoseused incivilianaviation,butalexicon forstandardization would resolve these differences andcreate a set ofabbreviations, with a basis in the MILSPECS, for industry use.

We feel thatin theinterestofstandardization, and toease crew transition from one aircraft type to another, alexicon ofcommon terms and abbreviationsmust be developed.

CLARIFICATION OF"NORMAL," "ABNORMAL," AND "EMERGENCY"There must be clear definitions of what arc to beregarded as "NORMAL," "ABNORMAL," and"EMERGENCY." The manufacturer of one imported aircraft flown by the regional airlines includes 39 "EMERGENCY" checklists outofa totalof 82 checklists. An example ofone checklist classified improperly as an "EMERGENCY," in ouropinion, is "UNPRESSURIZED FLIGHT."

One set of definitions of "ABNORMAL" and"EMERGENCY" has been created byFlight SafetyCanada, Ltd.

"EMERGENCY PROCEDURES"—"Thissection deals with foreseeable but unusualsituations in which immediate and preciseaction may be required by thecrew."

"ABNORMAL PROCEDURES"—"Procedures in this section address foreseeable situationsinvolving failures, inwhichthesystem'sredundancyorselectionofan alternate systemwillmaintain anacceptablelevelofairworthiness."

lnMIL-M-7700Cthere are definitions for"WARN-INGS" and "CAUTIONS" which could be borrowed for "ABNORMALS" and "EMERGENCIES."

"WARNING"—"C)rjeratmgprocedures.tech-

niques, etc., which could result in personalinjuryorloss oflifeifnotcarefully followed."

"CAUTIOrT*—"C)peratmgprocedures, techniques, etc., which could result in damage toequipment if notcarefully followed." To thelatter, we would add, "and if not carefullyfollowed, could eventually lead to personalinjury or loss oflife."

The Flight Safety definitions are not as stronglyworded as the ones inthe MIL SPEC, butdo conveythesenseofurgency, nonetheless. A combinationofthese definitions would satisfy the need to providestrict guidelines for use by aircraft manufacturersand airlines in the preparation of aircraft flightmanuals and checklists.

MANAGEABILITY OF CHECKLISTSPaper checklists should be of ah easily used andstowed size. We recommend in "PRINT SIZE andSTYLE" thatcard checklistsbe 8 1/2" x 11,"eitherlaminated ortrifold. We also recommend, if possible inkeeping with the recommendations on printsize and style, that there be a combination on onecard of"Normal" and"Emergency"checklists. Onegroup on one side ofthe card, one on the other. Oneairlineusesthiscombination.The combinationmakesthe task of location ofneeded checklists far easier.However, in this case, the recommendations forprintsize andstyle are not met.

To retain therecommended sizeof print we recommend that there be two cards, one for "Normal"checklists, and one for "Emergency" checklists —color-coded for easy identification. These shouldboth bekept inthesame, easily accessible place inthecockpit These two groups ofchecklists aretheonesthat shouldallow readyaccess. The "Normal"checklists are used all the time in daily operation."Emergency" checklists will not be needed on asteady basis, but should be immediately availablewhen they areneeded.

It is normal practice with many airlines to keep"Abnormal" checklists in the flight manual. Sincethey are not needed on an immediate basis, thisaccess is adequate.

We recognize that these guidelines do not addresstheconcern ofthe properuseofchecklistsby pilots.However, we feel strongly that if easily usable,readable checklists are available to pilots, the tendency to neglect or to misuse checklists may bereduced.

A-6

FIGURE A-3. DC-9 CHECKLIST

DC-9 NORMAL PROCEDURES CHECKLIST

BEFORE STARTING ENGINES

LOG BOOKS AND SEL CHECKED* RUDDER PEDALS AND

SEATS ADJUSTED AND LOCKED* WINDOWS CLOSED AND LOCKED

02 PANELS/MASKS/INTERPHONE/GOGGLES SET AND CHECKED

EMERGENCY LIGHTS ARMED* PROBEHEAT CAPT* WINDSHELDANTMCE ON

ANTI-SKID OFFPRESSUREATION AUTO (UP)AND SET

* AIR COND SHUTOFF AUTO* FLIGHT GUIDANCE PANEL SET AND CHECKED* FLT INSTR/SWITCHES/BUGS SET AND

CROSSCHECKED* FUEL PANELOUANTTTY AND

DISTRIBUTION SET/ LBS AND CHECKEDGEAR HANDLE AND

LIGHTS DOWN AND GREEN* TRANSPONDER SET* STABILIZER TRIM SET

SPOILER LEVER RETTHROTTLES CLOSEDFUEL LEVERS OFFFLAPS/SLATS UP/RETRACTED

* AILERON/RUDDER TRIM .ZERO/ZERO* PARKING BRAKE/PRESSURE PARKED/NORMAL* SHOULDER HARNESSES (IfOperative) ON* FUGHTFORMS CHECKED* NO SMOKING SIGNS ON* SEATBELTSKSNS (S MinutesPrior To Departure) ON

PRIOR TO ENG START OR PUSH-OUT

GALLEY POWER OFFENGINE IGNITION CONTINFUEL PUMPS ONAUX HYDRAULIC PUMP ONANTI-COLLISION/EXTERIOR LIGHTS ON/AS REQUIREDDOOR ANNUNCIATORS OUTAIR CONDITIONING SUPPLY SWITCHES OFF

TAXI

BEFORE TAXI

GALLEY POWER ONENGINE ANTMCE AS REQUIREDHYDRAULIC PUMPS CHECKED AND HI/ONAPU AS REQUIREDPNEU X-FEEDS (One EngineTaxi) LCLOSED/ROPENTAXI

AIR CONDITIONING SUPPLY SWITCHES AUTOANTI-SKID (After Leaving Ramp Area) ARMR ENG (One EngineTaxi) SHUTDOWNFLIGHT CONTROLS CHECKEDFGS TO MODE

•mmumuBmmmmmmmmmmmmmmmmmmmKmmmmmmmmmmmmmmmmmmm

BEFORE TAKE-OFF

Use Mechanical Checklist

AFTER TAKE-OFF - CLIMB

After Airplane Clean Up When Workload Permits.

GEAR UP AND NO LIGHTSSPOILER LEVER DISARMEDAUTO BRAKES OFF/DISARMEDFLAPS AND SLATS UP/NO LIGHTSPRESSURtZATION AND AIR COND CHECKED

10.QPPFt,MSLENGINE IGNITION AS REQUIREDFUEL SYSTEM CHECKEDSTERILE COCKPIT CABIN CHIMEALTIMETERS RESET AND CROSSCHECKEDHYDRAULIC PUMPS LOW/OFF

18.000 Ft. MSL

EXTERIOR LIGHTS .AS REQUIREDALTIMETERS RESET AND CROSSCHECKED

(Outside Continental U.S., Reset At The SpecifiedTransitionAltitudeObtained FromCharts Or ATC.)

A-7

APPENDDC B

Advantages and Disadvantages of Checklist Types

B-l

Type ofchecklist

Mixed -paper-slideor

paper-sw/lt

Advantages

1. Positive check on checklist progress forthose lists on the mechanical portion

2. The lists on the mechanical device canbe interrupted without losing trackofprogress

Paper 1. Easy to use and move around as thechecklists are done

2. Easy to stow

3. Inexpensive to reproduce

4. Inexpensive to update

Laminatedcard

CRT

1. Tough and hard to destroy

2. Difficult to mark on and mess up

3. Fairly easy to stow

4. Remains legible longer than paperchecklists

1. Can't lose checklists

2. Can present systems schematics in thecaseof "Abnormal" or "Emergency"checklists

3. Color-coded for ease of use

4. No stowage problem

Scroll 1. Permanent fixture - can't get lost

2. Promotes "heads-up" posture

3. Relatively easy to make changes tochecklists

4. Stows out of the way on the glare shield

5. Easy to mark progress

pisaflYantflges

1. Necessitates the use of two sets of lists

2. Slide or switch/light combination takes upcockpit real estate

1. Easy to mark on and mess up

2. Becomes worn easily

3. Easy to misplace or remove from theairplane

4. May be difficult to use under poor lightingconditions

1. More expensive to produce than paper lists

2. Bulky in comparison to a folded paperchecklist

1. May displace anotherdisplay such as radar

2. Requires a lot of "heads-down" time

3. Takes up cockpit real estate

4. Can be cumbersome to find a list or goback to a point in a list

1. Can be hard to read (size of print anddistance from the viewer, and some are notlighted at night)

2. Difficult to go back to a prior item on achecklist

Checklist"booklet"

1. Groups all checklists together - including 1. Can be bulky on aircraft with a largethe "Abnormal" and "Emergency" number of lengthy checklistschecklists

2. If properly tabbed, makes it easy to findany needed checklist

B-2

APPENDDC C

Summaries of ASRS Special Requests 1403 and 1417

C-l

AIRCRAFT OCCURRENCETYPE

1. LRG No nose wheel steering, had to betowed off runway

2. SMA Gear up landing

3. SMT Unauthorized entry onto runway

4. MLG Unauthorized runway crossing

5. MLG Possible traffic conflict, early turn toSID heading

6. LTT Aborted takeoff

7. LRG Departed 10,000 lbs. light on fuel,returned to airport

8. MLG Unable to pressurize after takeoff,emergency declared

9. WDB Altitudeexcursion and request forimmediate turnaway from weatherbecause of loss of F/O altimeter,flight instruments, and radar

10. WDB Deviation from assigned SID, startedto fly the wrong SID

11. LRG Crossed hold-short line but didn'tquite hsve a runway incursion

12. MLG Abnormal lightson takeoff, enginefire warning after takeoff, crewcontinued to destination

13. MDT Altitude overshoot in emergency

14. LTT Total electrical failure withemergency battery activation, spoilerswere deployed and would not retract,diverted to longer runway for landingand blew main gear tires on landing

C-2

CAUSE

Use of emergency and normal checklists - missedone item on the "descent" checklist

No written checklist availablepilot-passenger

interruption from

Busy finishing checklists and misheard "clearanceon request" 'for "cleared on course"

Busy running checklists, poor crew coordination

Reading checklist instead of paying attention toSID, poor crew coordination

Didn't turn on water injection system for takeoff,poorly designed checklist item, lack ofunderstanding of standard procedures

Busy doing checklists and no one verified theproper fuel loading - lack of clear procedures forfuelers to use and crews to verify proper fueling

Pack switches not on, checklist item notaccomplished, also not caught by the F/O on thequickcheck prior to declaring an emergency, foundsubsequently

F/O flying, Capt. and S/O doing an abnormalelectrical checklist, one part of the procedureknocked off the F/O instruments and radar at thetime they were to penetrate a line of weather

Confusion during time of reading checklists priorto takeoff and receiving runway and SIDassignment changes without programming in theFMS

Too busy with short taxi distance, unfamiliaritywith taxi route, and amount of checklist to beaccomplished

Engine fire bell went out and all engine indicationsnormal, had been prior work on and abnormallights for bleed air problems, did "air cond. supplytemp hi" checklist, later maintenance found at"hole in the engine due to starter reengaging

Loss of pressurization, emergency descent, tryingto control cabin altitude and do emergency andabnormal checklists and get clearance from center,"1,000 ft. above" didn't get called

Bad freon air-conditioner installation resulting inpower loss, used emergency procedures

AIRCRAFTTYPE

IS. LRG

QCgJRftENCE

16. SMA

17. WDB

18. SMA

19. SMA

20. LRG

21. WDB

22. SMT

23. MLG

24. MLG

25. WDB

26. WDB

Hydraulic problem after takeoff,dumped fuel, declared an emergencyand returned to land

Aircraft lost partial power on takeoff,hit powerline and made gear-uplanding on grass area of airport

Aborted takeoff due to enginedisintegration with associated firewarning

Gear retraction during takeoff roll,aircraft dropped to runway

Gear up landing

Runway incursion on rollout causingaborted takeoff by a MLG

Pilot not flying shut down bothengines in improper response to awarning light, aircraft was between1,200' and 1,500' AGL after takeoff, able to restart engines andcontinue

Altitude overshoot on departure

Aircraft returned to land, nose gearpin installed

Altitude alert activated in cruise,descent begun and oxygen masksused

Landed wrong runway from an ILSapproach

Unable to control cabin altitude,made a descent to control it

C-3

CAUSE

"A" system hydraulic failure on takeoff, subsequentmultiple abnormals due to air conditioningproblems, emergency declared with return toairport, equipment standing by and tow to the gate

No time for emergency checklists, cause of loss ofpower under investigation

Aborted, performed emergency checklist, checkedby fire crew, taxiing to gate fire crew noticedfurther engine fire which they extinguished, taxiedto the gate

Failure to follow proper checklist, instructor givingdual instruction gave pilot improper instructionsregarding a short field takeoff and the properpositioning of the gear handle

Pilot extended flaps on final instead of gear anddidn't use a checklist to assure gear down, ignoredwarning horn assuming it was a stall warning nearthe ground and of no consequence

Called for after landing checklist on rollout,misunderstood "hold short" instructions which hadbeen acknowledged by the F/O, started acrossrunway, too much confusion

No use of checklist, highly experienced Capt. triedto do an abnormal procedure without reference tothe checklist and without coordinatingwith the F/Owho was flying

PIC flying, check-pilot in the right seat acting asF/O and known for not encouraging checklist useor altitude callouts, aircraft sometimes flown as asingle pilot operation, poor coordination and noclear direction from the PIC as to procedure to befollowed

Nose gear pin installedduring tow to gate, duringchecklist the crew checked for gear pins, felt twoand thought it was three

Crew did not turn on the pressurization switcheswhen doing the checklist, thought they had butmissed them

Crew busy changing frequencies, doing checklists,etc., aircraft had been flown fully automated, oncrosscheck with raw data found improper ILSalignment, automatic go-around mode engaged,Capt. called for correction on ILS, took overaircraft and landed on the wrong runway in poorvisibility

Found air conditioning packswitches off, the restof thechecklist had been performed properly butthose had been missed

AffiCMFT OCCURRENCETYPE

27. WDB Initiated a go-around at 500' AGLbecause of gear not down

28. MLG Poorly designed and potentiallydangerous checklist

29. SMA Aircraft moved forward after startandhit the nearby fuel pump

30. MLG Go-around due to GPWS activation at500*

31. SMA

32. SMA

33. SMT

34. LRG

35. LRG

36. MLT

37. SMA

38. SMA

Gear up landing

Gear up landing

Misuse of transponder codemisleading center controller withpossible altitude conflict

Altitude overshoot

False fire warning, causinguse ofemergency procedures and evacuationof aircraft after landing with minorinjury to passenger

Aircraft made inadvertent slatsextended and flaps upT/O, no serious consequences

Aircraft landed gear up

Aircraft landed gear up

C-4

CAUSE

Crew had not fully configured theaircraft forlanding by extending the gear and final flaps,missed those items on the checklist and sot theGPWS at 500'

"Generic" checklist used for an entire fleet, has nological flow pattern and requires a PAannouncement on final in contravention of the FARsterile cockpit rule, has been approved by the POI

Pilot used aircraft checklist which called forthrottle to be pulled out 1/2" on start, regardless ofwhether warm or not, aircraft parked close to fuelpump, unable to control

Cockpit confusion due to monitoring close trafficon parallel approaches, gear handle not fully indown detent, when fully in detent GPWS continuedto sound, turned off pax 02 instead of GPWSbecause of proximityof switches in nonstandardcockpit configurations of the same model aircraft

Gear was not down and locked despite the use of achecklist, pilot also did not utilize his normalGUMPS check .

Used checklist but missed the gear, CFI in theaircraft didn't GUMP the aircraft, but ownerclaimed to have done that twice

Sloppy use of the checklist in entering transpondercode

Poor crew coordination, disregard of CRM andproper procedures by Capt. (on one takeoff thechecklist was just finished about 10 kts. prior toVr)

After checking, there wasno apparent fire, crewhad used emergency checklist and fought supposedfire, declared an emergency and evacuated theaircraft

Flaps had been programmed when checklists weredone, flaps raised when taxiingin proximity of alarge pile of dirt, flaps never extended, T/Owarning horn not programmed to sound withoutflaps since flaps retracted-slats extended T/O is oneconfiguration for that aircraft

Pilot forgot to extend gear, didn't use normalchecklist procedure with a GUMP backup due tofatigue, inop circuit breaker for gear warning horn

Pilot didn't do GUMP check, inop gear hom,distraction in the pattern

AIRCRAFT OCCURRENCETYPE

39. MLG Aircraft departed on wrong runway

40. MLG Incorrect V speeds set and not caughtuntil during the T/O roll

41. LRG Aborted T/O due to flaps not set

42. MLG Altitude overshoot on SID

43. MLG Aircraft took off with gear pininstalled, returned to land

44. SMT Aircraft landed gear up

45. LTT Overweight landing

46. SMA Aircraft landed gear up

47. LRG Aborted T/O, flaps not set for takeoff

48. MLG Engine failure and separation duringclimbout

49. SMT Gear not down for landing, minordamage from runway contact duringa successful go-around

50. WDB Aircraft off course by 20 miles or so

51. SMT Red gear warning lighton approach

C-5

CAVSB

Unexpected aircraft change with subsequent rushingand half-done job of checklists, poor crewcoordination, hearing clearance but not monitoringCapt.'s taxiing, Capt. late starting second engineafter single engine taxi with rushed and incompletechecklist and subsequent confusion

Operating rushed, late at night and fatigued andgave standard checklist response rather thanthorough check

Had read checklists and responded but the flapsweren't set, disrupted diurnal rhythm - crew hadflown late sequences all month and this trip had allearly checkins

During abnormal start procedure premature pullingof external electrical power caused automatic bugand altitude reminder resets, improper bug set wascaught on the checklist, altitude reminder was not

Gear pin flag removed and stowed in cockpit bycontract ground personnel, pin still remainedinstalled, crew on doing checklist counted three redflags but didn't check to make sure that a pin wasconnected to each

Crew preoccupied with approach to unfamiliarairport, didn't do final check, gear horn soundedjust at the flair with power reduction

Crew fatigued and rushed, improper fueling notcaught prior to departure, no mention of fuel loadon any of the checklists

Only used checklist partially, checklist difficult toread at night, busy monitoring traffic at busyairport, neither pilot nor instructorcaught the error

Fatigued crew with other distractions neglected toextend flaps and didn't read the taxi checklist

Cause unknown at present, emergency checklistperformed, emergency declared, landingwithoutfurther incident

Pilothad gear down early in the approach, raised itbecause of windshear encounter, with bad weatherand other distractions, did not extend gearagain,poor instrument scan, lack of checklist or GUMPuse

Using automated systems and Omega, both FMSand Omega had gross errors, both systemspreviously written up in the log for maintenanceaction

Unable to extend gear normally, used emergencyprocedure and checklist

AIRCRAFTTYPE

52. MLG

OCCURRENCE

Failure to shut down right engineprior to leaving aircraft

53. MDT Flaps not fully retracted afterlanding, flaps damaged by passengerbus driving under the wing on theramp

54. LRG

55. MDT

Aircraft had to level during climbdue to cabin altitude warning horn toallowcabin to catch up and topressurize

Engine fire with return to departurepoint and emergency declared

56. MLG Aircraft left with less than requiredfuel, no serious consequences

57. SMT Aircraft landed gear up

58. LTT Aircraft made go-around during anILS approach, anomalies ininstrument readings

59. MDT

60. SMT

61. SMA

62. MLG

63. LTT

64. MLG

Aircraft departed with incorrect fuelload, had to divert to alternate to getfuel

Aircraft landed gear up

Aircraft landed gear up

Complaint of passengers smoking inthe aisles and seatbelt sign off priorto completion of flight

Inflight engine shutdown due to lossof oil pressure and quantity,emergency declared

Altitude excursion on final approach

C-6

CAUSE

Crew claims to have used shutdown checklist, alsowent to belly baggage bin before leaving and didn'tnotice engine running

High demands on crew by ATC on rollout to clearthe runway quickly, during after landing checklistthe F/O was interrupted many times and didn'tretract flaps fully, SILENT checklist without othercrew monitoring

Too short a time period during taxi to accomplishall items satisfactorily, including checklist, missedthe air conditioning pack switches, should havedelayed to accomplish everything

Used engine fire emergency checklist, looked forsingle engine landing checklist and couldn't find,checklists in the process of revision with conflictsbetween some lists, FAA aware of the problemsbut no action to date

Distracted attention in the cockpit during thereading of checklist

No checklist, gear warning horn did not operate

Crew fatigue, missed proper settings on navreceivers, no items on checklist to cover this

Distraction in the cockpit at the time the checklistwas being read, holding for fuel to be loaded, rushto make schedule, fuel last item on the crewacceptance checklist and not on any other checklistfor a crosscheck

No checklist, task saturation at low level, gearhandle used but gear didn't extend, gear warninghorn inop, didn't confirm gear green lights

Pilot monitoring hot air balloons and other traffic,sun in his eyes, lowered flaps instead of gear,didn't get warning hom due to high manifoldpressure because of ATC-requested high speed onapproach

Crew not using checklist correctly and notmonitoring passenger conduct

Crew had a low oil pressure warning and ignoredit because of previous transducer failures on thisaircraft type, lowoil quantity and pressure causeda flame-out, did emergency checklist

Aircraft stall warnings systems activated, crewfollowed stall procedures including lowering thenose to pick up speed for configuration, system hadfailed, aircraft was not in a stall

AIRCRAFT OCCURRENCETYPE

65. MLG Aircraft aborted T/O due to highwind noise around Capt.'s window

66. LRG Didn't make required log bookentries

67. MLG

68. WDB

69. LRG

70. SMA

71. MLG

72. MDT

73. LTT

74. MLG

75. LRG

76. MLG

77. LRG

78; SMT

Gear doors didn't retract on raisingthe gear, damage to doors onsubsequent landing

Aircraft unable to pressurize,descended with special handling

Emergency descent due to loss ofpressurization

Aircraft landed gear up

Cabin altitude hom sounded, unableto control cabin altitude, emergencydescent with altitude overshoot

Aircraft took off with cockpit dooropen and flight attendant still stowing

Aircraft lost right engine cowling andhad right engine failure at 1,000' inclimb

Aircrafthad smoke in the cockpit andpressurization problems, descendedand continued to destination

Go-around due to no gear extensionand GPWS warning

Aircraft landed with the cabin notsecured and with flight attendantsnotin assigned landing positions

Possible health hazard to groundpersonnel from operating radar

Aircraft aborted takeoff from 40' inthe air resulting in aircraft damage

C-7

CAUSE

Window design such that the handle appearedproperly in place but the securing dogs weren'tproperly in place, window is not a checklist itemor it might have been noticed

Had an asymmetric flap procedure on landing, usedabnormal list and normal, during the confusion andsubsequent relief of being on the ground, theyforgot

Crew did the checklists required for unretractedgear doors, used all published procedures

Switch not in proper position to allowpressurization, was answered for on the before-taxi checklist but not properly checked

Failure of door seal, used all appropriatechecklistsand landed without incident

Busy watching traffic ahead on final, didn't extendgear or do GUMP check

Improper altitude put in altitude reminder whileF/O was busy trying to do the checklists and talkwith ATC

Flight attendant supposed to close cockpit door,inadequate flight attendant training, cockpit doornot on any checklist

Latches to the cowl are supposed to be checked onpreflight, pilot claims he did, all emergencyprocedures followed, uneventful landing

Did the electrical smoke or fire checklist, isolatedthe problem, continued to destination and landedwith the emergency equipment standing by on theground

Crew got behind the program'with an approach inthe weather and a change of runways duringapproach, missed the geur on the checklist

Checklist still reflects the use of a call button toalert the flight attendants at the time the no-smokesign was turned on - with the new smoking regs,the no-smoke sign is on all the time for this airline- checklist or operating policy should be revised

Aftera demanding flight the crew did the properchecklistsand thought they had turned the radar tostandby • radar had different switching than whatthey were used to and may not have been turned tostandby

Pilot took off with the control lock on the yoke •didn't use checklist to back up flow pattern

AIRCRAFTTYPE

79. LRG

OCCURRENCE

80. MLG

81. MLG

82. MLG

83. MLG

84. LTT

85. SMA

86. MLG

87. MLG

88. LRG

89. MLG

90. MLG

Aircraft depressurized requiring useof rapid depressurization andexplosive depressurization checklistsand diversion to a nearby field

Aircraft declared an emergency onclimbout and returned to land

In climb the aft cargo door lightilluminated, unable to pressurize,continued to destination and landed

Aircraft unable to controlpressurization, horn sounded, masksdropped, emergency declared

Didn'tcontrol cabin altitude, gotpassenger oxygen masks, recoveredpressurization, continued todestination climbing above 25,000'illegally (due to no availability ofautomatic oxygen mask presentation)to avoid weather

Near mid-air collision, took evasiveaction

Aircraft landed gear up after anaborted landing and go-around

Loss of pressurization and emergencydescent

Jetway shifted causing minor aircraftdamage, blamed on aircraft rolling

Near overtemp on starting engine #1

Aircraft rolled forward on enginestart, brakes applied suddenly causingflight attendants to fall with twosustaining minor injuries

Damage to aircraft tow bar duringpushback

cause;

Cracks in the cabin in thewheel well area probablydue to aircraft age

Engine loss on climbout with useof emergency andnormal checklists

Cargo door light not noticed during pre-takeoffchecklists, continued due to below landingminimums at departure point

Loss of pressurization, cause unknown, usedemergency checklists and procedures, continued todestination at lower altitude

Bleed switches not on and not noticed out of theproper position on the checklist

Busy doing checklist for descent and both hadheads inside the cockpit, although under positivecontrol, the controllerdidn't point out the traffic

Too much float on a hot day, went around. Didn'tput gear down for second approach, did a GUMPcheck and missed the gear, gear horn didn't workbecause of high approach power setting

Lostboth packs simultaneously, used emergencychecklists and descent, donned oxygenmasks, bothpacks came back on the line, continued todestination, cause unknown

Brakes were set per the securing checklist

At a stop on a through flight maintenance had beenworking on a thrust reverser problem, start levershad been left in idle rather than cutoff during thework, this was not caught prior to start since "startlevers to cutoff" is not on the before start checkliston a through flight

Brakes not set during checklist, chocks pulled byground crew without informing cockpit crew, nonstandard procedure for use of parkingbrakes priorto engine start

Abnormal start due to APU electrics inop, nospecific checklist to cover, used normal flowpattern during an abnormal start

SECOND GROUP OF REPORTS FOLLOWS ON PAGE C-9

C-8

AIRCRAFT OCCURRENCETYPE

1. MLG Aircraft landed without clearancefrom the tower

2. MLG Aircraft overshot altitude in descent,on autopilot

3. MLG Aircraft overshot altitude in climb

4. WDB Aircraft overshot altitude on SID

5. MLG Aircraft overshot altitude on descent

6. MLG Aircraft emergency evacuationleaving the ramp

7. MLG Aircraft overshot altitude in climb

8. MLG Runway incursion during taxi

9. MLG Altitude excursion, aircraft onautopilot

10. MLG Near mid-air collision, took evasiveaction

11. MLG Emergency descent made andemergency declared, couldn't controlcabin altitude

12. MLG Near mid-air collision, no time forevasive action

13. WDB Aircraft overshot turn to final

14. WDB Aircraft aborted T/O

15. WDB Questionable descent clearance

C-9

CAUSE

Two-man crew, very busy trying to locate anunfamiliar airport, doing checklists, etc., didn'tswitch frequencies

Autopilot sensing taken off F/O altimeter whichwas set 1 inch too high (30.79' vs. 29.79")

Aircraft on test flight, two-man crew, pilot flyingnew on aircraft, pilot not flying overly busy withextensive test flight checklist and didn't call 1000'before the altitude

Preoccupation with the checklist and no call for1000' before the altitude

Two-man crew fairly new to the airplane, busyrunning checklists and other duties, knocked offaltitude hold by mistake and didn't catch it untilafter descent below assigned altitude

Alleged right engine fire, ran emergency checklistsand did emergency evacuation

Didn't reset altimeters at 18,000' and didn't catchit on the checklist

Crew busy doing checklists and briefing

Crew busy doing checklists and other duties, didnot catch the fact that the autopilot had gone toanother mode and started to climb

Aircraft level, crew busy changing radio and doingchecklist, looked up to see small aircraft very closeat the same altitude, no mention by the controller

Did emergency checklists, auto pressurization lost,regained controlwith manual pressurization,continued to destination

Aircraft in level flight under positive control, didoutside check, dropped eyes to checklist, lookedback up to see an aircraft within 150' crossing atthe same altitude, no mention by the controlleralthough the controller did say afterwards he hadthe aircraft on radar

Crew busy programming die FMC and doingchecklist, got behind the airplane and didn't getinto the slot until 1000'

F/O sliding window came open on T/O, notlatched properly, item not on checklist for positivecheck

Crew busy doing checklists, handling multipleradios, etc., got a descent clearance from onecontroller, a frequency change, and the followingcontroller questioned the altitude

AIRCRAFT OCCURRENCETYPE

16. MLGA Aircraft undershot crossing altitude

17. MLG Altitude overshoot on descent,aircraft on autopilot

18. MLG Altitude overshoot on climb

19. MLG Altitude overshoot on climb

20. WDB Altitude overshoot in climb

21. LRG Aircraft missed crossing restriction

22. MLG Momentary application of heavy autobrake on landing, resulted ina verynoticeable lurch during rollout

23. MLG Aircraft several thousand feet high oncrossing restriction

24. MLG Probable needless engine shutdown inflight, emergency declared with aprecautionary landing short of thedestination

25. MLG Altitude overshoot on climbout

26. MLG Altitude overshoot on climbout

27. WDB Altitude overshoot on descent

28. MLG Altitude overshoot on descent

29. MLG Speed deviation on STAR

30. WDB Altitude undershoot in climb

CAUSE

Crew busy getting ATIS, working radio, doingchecklists, tuned wrong VOR frequency, and didn'tmake crossing restriction

Captain busy with checklist, F/O programmed theautopilot wrong and knocked off altitude hold

Maximum performance climb, light aircraft, tiredcrew, busy doing checklist and working radio,didn't reset altimeter soon enough and wentthrough the assigned altitude

Late at night, long flight sequence, light, fastclimbing aircraft, multiple frequency changes,doing checklist, didn't catch it

Crew didn't reset altimeters to 29.92" at 18,000',distracted from the checklist by turbulence

Due to multiple frequency changes and looking fortraffic climb checklist was never done, andaltimeters weren't reset

While doing the landing checklist the F/Oinadvertently programmed the auto brake for T/O,due to darkness and having to do a 360 degree turnon final, the error was not caught

Poor crew coordination, inexperience on theaircraft and that portion of the route structure forthe captain, running the checklist

While performing the checklist for an electricalabnormal, captain mistook an APU low oilpressure light for an engine low oil pressure lightand shut down the engine, poor crew coordinationwhile doing electrical abnormal and F/O wasstarting the APU

Captain had called 1000' before the altitude andgot busy doing something else, F/O looked away todo something that wasn't called for on the checklistat that point and went through the altitude

Very short flight, frequency changes (bothcompany and ATC), auto throttles not operating,doing checklists, overloaded two-man crew

Busy two-man crew, set improper altimeter andovershot by 1000'

Two-man crew doing checklists and other duties ondescent for landing, altitude capture not set onautopilot, no altitude warning on the aircraft,caught by the crew after they had overshot

Captain handflying aircraft for practice, F/O doingchecklists, handling radio, etc., both missed thespeed restriction on the STAR

Crew neglected to reset altimeters to 29.92" at18,000', missed it on the checklist

C-10

AKRCTATTIX£g

OCCURRENCE

31. MLG

32. MLG

33. MLG

34. MLG

35. WDB

36. MLG

37. MLG

38. WDB

39. MLG

40. MLG

41. MLG

42. MLG

43. MLG

Near mid-air collision on arrivalroute, took evasive action

Altitude overshoot in climb

Altitude overshoot on descent

Altitude overshoot on descent

Altitude overshoot on descent

Aircraft almost aligned with thewrong runway for landing, FAAcheck airman on board made nocomment, caught the error in time

Aircraft almost departed on a runwaywhich was too short for their weight,caught by the company and relayedby the tower controller

Crew returned to ramp to have anextended spoiler fixed, spotted bycrew of a following aircraft

Aircraft almost departed with seatbelt sign off and correct takeoffpower settings

Altitude overshoot in descent

Aircraft crossed runway hold lineduring taxi after instructions to holdshort, potential conflict

Aircraft landed without clearancefrom the tower

Altitude overshoot on climbout

C-ll

CAUSB

Crew doing checklists and crosschecking settingson instruments as per company policy, just missedother aircraft crossing the arrival route, no warningfrom the controller

New capt., new copilot, new airplane, new airport,very rushed, rushed the checklists (missing anitem), unfamiliarity with autopilot resulted inovershoot

Forgot to reset altimeter leaving 18,000' in the

Read in range checklist completely at 24,000 andmissed the altimeter reset at 18,000', prematurecompletion of the list

Two-man crew, between 310 and 180 had fivespeed changes and two hdg. changes, one altimetergot reset, the one of the pilot flying did not; inaddition, after the overshoot there were three morespeed changes, two more hdg. changes and threerunway changes (the last one taking place at 400'on final), THIS IS RIDICULOUS

Capt. busy looking for airport, running checklistsand helping recent upgrade copilot

Runway was the longer of the two and into thewind, but had a terrain restriction, crew was busydoing checklists and tending to a passengerproblem and didn't actually check the performancecharts for the runway

Taxiing withoneengine shutdown, holding off onchecklist, takeoff position advanced by controller,rushed to complete everything and missed indicatorlight for partially extended spoiler

Rushed turnaround, trying to beat a curfew, rushedchecklists and missed items, caught on the taxi forT/O

Training flight, instructor busy doing checklists andinstructing, autopilot lost the altitude hold andneither pilot caught it until after the overshoot

Two-man crew doing challenge and responsechecklists and required PA announcements andmissed holding short

Heavy traffic, a great deal of maneuvering closein, busy doing checklists, didn't switch over fromapproach to tower

Pilot flying new on the aircraft, pilotnot flyingbusy with communications, traffic watch andchecklists, pilot flying did not reset altimeter and itwas not caught on the checklist

A1BSArr OCCURRENCETYPE

44. MLG

45. MLG

46. MLG

47. MLG

48. WDB

49. SMA

50. WDB

51. SMT

After liftoff a door light came on andaircraft could not be pressurized,returned to land

Minor overshoot on descent

Altitude overshoot on descent

Altitude overshoot and excessivespeed

Altitude undershoot on climb andmissed altimeter on approach

Possible near miss

Left engine running after the securingchecklist and leaving the aircraft

Altitude overshoot, possibleconflictwith other traffic

52. MLG Aircraft landed with considerable fuelimbalance

53. LRG Aircraft overshot approach course,corrected for normal approach andlanding

54. MLG Altitude overshoot on descent

55. MLG Flight departed with less than plannedfuel load

56. MLG Altitude overshoot on descent for ILS

57. MLG Aircraft would not pressurize inclimb

58. MLG Partial hydraulic loss, manual gearextension

CAUSE

On door light checks on the checklists on theground the door light was not illuminated

Contributing factors were preoccupation withchecklist and PA

New capt. getting lineoperating experience, doingchecklist, changing frequencies, getting ATIS, de-icing airplane, autopilot did not capture properly,also no altitude alert on this type of aircraft whenit is on all the rest of the fleet, nonstandardization

Light aircraft witha fast climb, crew busy doingchecklists, frequency changes, etc., gotway behindthe airplane, attempting mixed use of autothrottleand manual control unsuccessfully

Sloppy use of checklists

Pilot had been in contact with approach, had beengiven a discrete code and cleared below the LAXTCA, approach did not passon info to LAX,passed near inbounds to. LAX thatapparently didnot see him

Did not physically check that fuel control switcheswere in cutoff, fuel control switch positions easilyconfused

Crew busy doing arrival prep such as PA, ATIS,checklists, etc., misunderstood altitude cleared toand descended too low

Crossfeeding taking place, did not reinstate properfuel pump configuration before landing, should bean item on the checklist for fuel pumpconfiguration

Unintelligible controller instructions, interruptionsof checklist, missed proper inbound course settingon resumption of checklist

Descent on autopilot, checklists in progress,autopilot failed to capture altitude, recoveredmanually

Aircraft not fueled, did not properly check the fuelload on the pre-engine start checklist

Aircraft programmed for automatic ILS approachcapture, while crew was busy doing the beforelanding checklist the FMS intercepted the localizerand began a premature descent, corrected manually

Cabin altitude control lever in the wrong position,missed on checklist

Used appropriate abnormal hydraulic checklist

C-12

AIRCRAFT

XXEEOCCURRENCE CAVSB

59. LRG

60. MLG

61. LRG

62. WDB

63. MLG

64. MLG

65. WDB

66. MLG

67. MLG

68. LTT

69. MLG

70. MLG

71. MLG

72. MLG

Altitude overshoot of 1000' indescent

Altitude overshoot on climbout

Altitude overshoot on descent

Aircraft declared an emergency,smoke in the cockpit, diverted to landshort of destination

Aircraft lost comm on an activerunway, caused a go-around

Altitude overshoot in descent

Aircraft landed without clearance

Aircraft landed without clearance

Altitude overshoot on descent

Near collision on a runway, aircraftcleared into position to hold on arunway where another aircraft hadbeen cleared for T/O

Altitude undershoot in climbout

Altitude undershoot in descent, wentbelow crossing restriction

Aircraft didn't makecrossingrestriction

Altimeter set incorrectly by 1", not caught on twochecklists

Distracted by radio, setting instruments, andchecklists, didn't make 1000' before altitudecallout, altitude reminder sounded

Crew busy getting ATIS, doing descent andapproach checklist, setaltimeter improperly,altimeter setting not checked with that issued byATC

Various annunciator warnings, smoke in thecockpit, used oxygen masks, ran normal checklistsbut no emergency checklists were mentioned

Crew busy doing checklist and final items for T/O,didn't notice a comm switch in the off position

Fatigue, descending in bright sunlight,hydraulicpump activation caused a voltage spike knockingoff the autopilot altitude hold, also making PAannouncement, crew did not notice autopilotnotengaged when running checklist

Approach during rough weather, crew busycontrolling aircraft and doing checklist, dialed inwrong frequency and didn't catch it until on theground

Approach control didn't switch the flight over totower, crew busy running checklist, etc., didn'tcatch it until on the ground

Doing checklist, reset altimeter for local pressurewhen only cleared to 18,000', altitude alert is onlytriggered by captain's altimeter, not both, so didn'tsound

Crew busy doing checklist but did hold short tocheck runway as everyone should, saw otheraircraft rolling and held short

Altimeternot reset, crew busy running checklistsand handling aircraft in bad weather, NEWCHECKLIST PROCEDURE HAS ALTIMETERSRESET FROM OFE TO ONH AT 10.000' - TOOLATE FOR ACCURATE USB WHENASSIGNED ALTITUDES BELOW 10.000'

New capt., low light level, high workloadincluding running checklists, misread DME forcrossing restriction, other pilot did not recheck onhis chart

Two-man aircraft, high work load includingchecklists, controller confusion as to a priorrestriction

Altitude undershoot in descent,missed crossing restriction

Pilot flying busywithaircraft in turbulence andicingconditions, non-standard crossing restriction,pilotnot flying out of the loop doing the checklist

C-13 and didn't catch the error

AIRCRAFT OCCURRENCETYPE

73. MLG Altitude overshot in climbout

74. MLG Altitude overshootin climbout,aircraft would not pressurize

75. WDB Wild autopilot oscillations in flight,corrected by going to manual control

76. MLG Passed hold short point on a taxiwaycutting off another aircraft

77. MLG Altitude overshoot in climbout

78. MLG Altitude overshoot in climbout, notcaught by controller

79. WDB Altitude overshoot in descent

80. MLG Possible missed crossing restrictionon both altitudeand speed

81. MLG Altitude overshoot in climbout

82. MLG Altitude undershoot in climbout,missed crossing restriction

83. MLG

84. WDB

85. MLG

Complaint concerning close parallelapproaches

Aircraft experienced multipleelectrical failures, declared anemergency and landed short ofdestination

Runway incursion

CAUSE

Pilot not flying busy doing checklist during a highrate climb at low level, altitude alert nonstandardfrom other aircraft in the fleet, pilot flyingdistracted temporarily

Inadequate preflight and checklist use didn't catchlocked open outflow valves, aircraft wouldn'tpressurize and momentarily distracted crewattention from the altitude

Crew didn't turn on pilot heat, didn't catch it onthe checklist, pitot tube iced up causing airspeedindication loss which sent incorrect speed to the airdata computer resulting in rudder inputs for lowerspeeds when aircraft was at high speed

Two-man crew busy doing checklists and workingSround and company radio, capt. misunderstoodle taxi instructions and F/O didn't monitor closely

enough because of other duties

Pilot not flying reading the checklist, failed to call1000' before the altitude, ACARS message cameacross at the same time as they hit the assignedaltitude

Crew busydoing checklist and other duties, wrongaltitude set in the altitude reminder, overshot andin the overshoot received a clearance to higheraltitude

Two-man crew busy in arrival procedures in busyarea, bad weather, copilotbusy doing comm, etc.,capt. flying aircraft, programming the computerand doing checklists, missed altimeter reset at18,000*

Aircraft developed a pressurization problem indescent, crew busy doing abnormal procedure andflying aircraft missed crossing restrictions, but atthe same time the controller gave them new altitudeand heading which cancelled prior restrictions

Lower altitude assigned thanoriginal clearancewhen aircraft was almost at the new assigned andat a highclimb rate, also distracted doing thechecklist and altimeterdidn't get reset

Changes in altitudeclearance by departure, crewbusy doing checklist and other departure duties andturned prematurely resulting in lower altitudeatcrossing point

Reporter suggests staggering aircraft, in addition tobeing alarming to passengers it distracts fromchecklist and other duties

Properuse of abnormal, emergency and normalchecklists

Aircraft had been cleared to hold short, F/O busydoing checklist and not listening, capt.misunderstood clearance

C-14

AIRCRAFTTYPE

OCCURRENCE

86. MLG

87. WDB

88. MLG

89. WDB

90. MLG

91. MLG

92. MLG

93. MLG

94. MLG

95. WDB

96. MLG

97. LRG

98. MLG

Engine oil leak caused further engineproblems resulting in shutdown, othergenerator didn't pick up the lost load

Aircraft had to return to land due totwo cargo doors open

Altitude undershoot on climb

Aircraft took off over weight on alimited runway with antiskid inop

Cabin altitude climbed above 10,000'with no altitudewarning hom,passengeroxygen masks deployed,returned to departure point

Aircraft overshot altitude on profiledescent

Altitude undershoot in climb

Altitude overshootduring STAR

Unauthorized landing

Aircraft tookoff with gear pinsinstalled and had to return to land

Aircraft tookoffwithnose gear pininstalled and had to return to land

Aircraft overshot altitude in climb

Altitude overshoot during descent,less than standard separation withother aircraft

CAUSE

Confusion in the cockpit due to nonstandardizationof fleet, compounding problems, controller queriesduring a busy time, DIFFICULTY IN LOCATINGTHE EMERGENCY CHECKLIST

Glass cockpit airplane, CRT wiped clean during thefire test in before starting engines checklist,misconception from training concerning recall ofitems to the CRT after start led to not seeing doorsopen light (crewmember had been led to believethat information wasautomatically displayed onpower change over after start when it had to berecalled manually)

Reset of altimeter at 18,000' is not on the checklistand the crew forgot it

Rushed departure after maintenance delay workingon antiskid, very short taxi with rushed checklistsand engine start, message on weights to checkdispatcher for reduced VI speed, dispatcherreferred them tomanuals, manuals poorly set up toget info, two-man crew in busy environment unableto find info readily

Proper use of appropriate checklists, inop cabinaltitude warning hom and auto pressurization

Aircraft on autopilot with altitude holdengaged,pilot not flying doing checklist, altitude warning 'horn did not sound and autopilot did not capturealtitude

Altimeters not reset, didn't catch it in the checklist,low flight crew experience level, fleetnonstandardization

Flight crew distracted doing checklist

Crew given poor vectors to final and then turnedon for a short, steep descent for landing, thoughtthey heard a clearance which was for anotheraircraft - this aircraft uses a mechanical checklistwith two blanks for "cleared for the approach" and"cleared to land" - thinking he hadheard that, thecopilot moved the slides indicating to thecapt. thatclearance was received

Crew distracted by maintenance while reading thechecklist and missed the gear, pins

F/O distracted on walkaround by new hireaccompanying him, missednose gear, PIC can'tsee gearpins in the cockpit as on other aircraft inthe fleet, missed on the checklist

Aircraft in heavy weather, pilot flying called forthe climb check, aircraft sustained a lightningstrike, misread autopilot annunciators, and changedautopilot settings resulting in an overshoot

Two-man crew inbusy environment, runningchecklists, etc., and altitudealert didn't sound

C-15

AIRCRAFT OCCURRENCETYPE

99. LTT Aircraft landed without clearance

100. MLG Altitude overshoot in descent

101. MLG Aircraft landed on the wrong runway

' 102. MLG Aircraft flew wrong radial ondeparture

103. MLG Aircraft missed crossing restriction

104. MLG Altitude overshoot of 1100' onclimbout

105. MLG Aircraft missed crossing restriction

106. LRG Altitude overshoot on short final

107. MLG Altitude overshoot on climbout

108. LRG Altitude undershoot at top of climband in cruise, not noticed untildescent for landing, controllerdidn'tcatch

109. MLG Aircraft experienced loss ofpressurization, made emergencydescent and declared an emergency

110. MLG Engine flame-out at altitude from fuelexhaustion, emergency declared, gotengine relight atTower altitude

111. MLG Altitude deviation during approach

112. MLG Altitude overshoot on climbout

C-16

CAUSE

Busy airport, crew monitoring heavy incloseproximity for the parallel runway, doing checklist,didn't contact tower

Crew didn't reset altimeter at 18,000', caught laterwhen they ran the checklist after the overshoot

Being vectored for one runway, confusion overcontroller comments concerning another, busyrunning checklist

Not set properly in nav instruments prior todeparture and not caught on checklist

Concern over airport below minimums, discussingalternate plans, busy running checklist

Automated cockpit set to altitude capture withautodirottles set, crew doing checklist, autopilot didnot capture

Crew busy doing checklist items, clearancemisunderstood by the pilot flying and not caught intime by the other pilot

Doing checklist in turbulence, pilot flying altimeterset off 1", multiple approach control course andspeed changes, mistake not caught until GPWSsounded and approach control altitude alert sounded

Crew busy looking for traffic anddoing checklist,new crew to aircraft in both seats, Ughperformance climbwith a 2000' assigned altitude

Crew new to the airplane, both used to three-mancrew, now on a two-man aircraft, missed settingaltimeters at 18,000' and didn't catch it on thechecklist

Appropriate checklists used

Ran the main tanks dry with a lot of fuel in thecenter tank, didn't have all the boost pumps on anddidn't catch it on the checklist

Two-man crew, very busy environment with manyheading and speed changes, frequency changes,ATIS, reading the checklist - one pilot thought hebeard a clearance and started down, clearance notconfirmed because of frequency congestion

Due to loss of partial aircraft systems and transferof aircraft control and subsequent abnormalchecklistsaltimeterwas not reset at 18,000', thetransition level altimeter reset is not on a checklist

AIRCRAFTTYPE

OCCURRENCE

113. WDB Aircraft failed to pressurize, returnedto point of departure

114. WDB

115. MLG

116. MLG

117. MLG

118. LRG

119. LTT

120. MLG

121. WDB

Aircraft landed without clearance

Engine flamed out, single attempt atrestart unsuccessful, landed short ofdestination

Aircraft taxied into positionon anactive runway, possibly withoutclearance

Near mid-air collision

Aircraft landed without clearance

Aircraft crossed an active runwayafter instructed to hold short

Aircraft filled with smoke at 37,000',declared an emergency and landedshort of destination

Partial runway incursion, caused ago-around

122. WDB Deviation from assigned SID

123. MDT Altitude overshoot on climbout

124. MLG Aircraft had abnormal lights prior toVI, continued T/O, had engine firewarning at V2, lights went out andthey continued to destination

125. WDB Engine disintegrated at about VI,crew aborted, residual fire put out byemergency crew

126. MLG Engine not shut down prior to exitingaircraft

CAUSE

Neither air conditioning pack was operating, nochecklist for that abnormal procedure, returned andfound a start arm switch in the wrong position,didn't catch it on the checklist after startingengines, the only checklist for packs inop is foundunder the expanded checklist for rapiddecompression (Trl)

Crew busy with tight approach and doing checklist,didn't contact tower until after rollout, tower didn'teven know they had landed

Used all appropriate checklists, abnormal,emergency, and normal

Confusion as to controller instructions, capt. calledfor last items on the before takeoff checklist whichare normally done only when cleared into position

Aircraft on approach, on autopilot andautothrottles, crew was busy changing frequenciesand doing the checklist, when they looked up theother aircraft was crossing 300' above and about700' out

Making acoupled approach for an autoland, doingchecklists, fatigue, forgot to shift frequencies

Copilot got instructions, assumed captain had them,started to do the checklist heads down and didn'tcatch the crossing, poor crew coordination

Used appropriate checklistsand procedures

Crew busy doing checklist, misunderstoodclearance to taxi up to and hold short, taxiedbeyond the hold short point

During taxi aircraft received runway changes,changed SID in FMS, runway reassigned, in doingthe checklist and other duties, SID didn't getchanged again

Crew busy dodging thunderstorms on departure,changing frequencies, flying the aircraft, doingchecklist, no altitude warning on tiie MDT whenCapt. had been flying an airplane that had one

Poor procedures, did an abnormal checklist for anairconditioning supply temp high, whenmaintenance checked the aircraft they found a 1"hole in theengine where the starter had reengaged

Crew followed proper procedure and usedappropriate checklists

Stressful flight, stress resulting from merger, poorcrew coordination, lack of use of checklist

C-17

AIRCRAFT OCCURRENCETYPE

127. WDB Inaccurate navigation, deviation fromassigned track

128. MLG Altitude overshoot during descent

129. WDB Both engines shut down at 1500' inclimb, restarted and continued flight

130. WDB Altitude overshoot during approach

131. MLG Engine failure in cruise, declaredemergency, landed at the nearestsuitable airport

132. MLG Cargo compartment fire, emergencynot declared since aircraft was onfinal for landing, did declare anemergency on the ground with apassenger evacuation

133. WDB Engine flame-out when throttles wereretarded for descent

134. WDB Unable to control cabin altitude,descended to control

135. MLG Aircraft departed with incorrect fuelload, had to make a fuel stop

C-18

CAVSE

FMS programmed improperly, should have beencaughton review of programming for checklist

Crew busy handling communications with companyand ATC, doing PA announcements, runningchecklists, set wrong altitude into the altitudereminder

Capt. did not use the checklist for an abnormalannunciator light, used the wrong switches to solvethe problem, no crew coordination

Controller cleared the aircraft to 3000', thought hehad cleared them to 4000', they got busy doingchecklists and other duties and descended to 2600'

Shutdown due to high EGT and low EPR, usedappropriate checklists

Illegally shipped hazardous cargo, crew indicatedthat with a two-man crew in this type of situation,trying to fly the aircraft, do checklists andeverything else, one person is "outof the loop"trying to get information on the problem and theother person is left to do everything else

Proper checklists used includingrestart checklist,successful restart, problem caused by bad bleedvalve which is in the process of modificationfleetwide

Engine start switch in the wrong position for packoperation, should have been caught on the afterstarting checklist

During predeparture checklists the crew wasdistracted by on board FAA inspectors, didn'tcheck fuel properly

APPENDDC D

Form and Results of ALPA Survey

D-l

CHECKLIST SURVEY <9S returns for Kft m^ilffl)

fSlows^0118 f°r ^ SUrVey "* fivefold' Each raason **" hmt ite own •* of questions. The reasons are as

1) Identify layout and other design characteristics ofchecklists that inhibit or promote easy use;

2) Determine what aspects of flight operations interfere with checklist use, and identify the phases offlight during which these distractions are most likely to occur;

3) Detennine the degree to which checklist procedures are defined in the pilot handbook;

4) Identify variations in checklist use that can be attributed to crewmember characteristics;

5) Identify procedures or design changes that could be used to promote error-free checklist use.L LAYOUT AND DESIGN OP CHECKLISTS

1.1 Types ofchecklists you have used (please check types used and circle type currently used)...currently used

a. Paper checklist 25. Yes 74 No 21b. Laminated card(s) 52 Yes 82 Nollc. Electronic (CRT) 1 Yes fi No 8$

* Does the display replaceanother display, such asweather radar Yes 4 No 1

d. Mechanical scroll 1 Yes 33 No 62e. Mechanical pointer ~ Yes J Nogf. Mechanical slide Yes £ No 86g. Toggle switch/annunciator light

combination Yes 0_ No£lh. Have you used, or do you now use,

a mix of the above (i.e., - paperchecklist & mechanical slide) Yes 12 No 74.

• If "yes," are the "normal"checklists segregated from the"emergency" and "abnormal" lists Yes M No 6.

(please explain in what way)

i. Do you see an advantage to a mixof checklist types? Yes H No XL

(please explain)1.2 Does the "silent" checklist have a

place in airline cockpits? Yes 71 No 23_

1.3 Of the following checklists, which do you feel should be "challenge/response" and which should be"silent"? ^

challenge/response silent

• Airplane acceptance 32 3JJ• Before start 71 5.• Before taxi £1 J5• Before takeoff 76. 0.• Climb 16, 6Jj• Cruise 12 64• Descent/In range 42. 22• Before landing 73. 2• After landing 2± £§• Securing 5J. %4

D-2

1.4 The following questions pertain only to those who have used electronic (CRT) checklists and paperchecklists and will attempt to ascertain the relative advantages and disadvantages of the two types. Pleasecircle the appropriate answer.

a. Easier to use in all conditions of cockpitillumination &CRT paper 0.

b. Greater susceptibility to skipping items 1 CRT paper 5c. Easier to get at and use £ CRT paper1d. Ease of use in different operating conditions

• Stationary on the ground 5 CRT paper1• Moving on the ground 5 CRT paper 1• Airborne 5 CRT paper £

e. More heads-down time required 2 CRT paper2f. Quicker to use 1 CRT paper 1g. If items are skipped and returned to (such as

in taxiing without all engines operating),which is easier to use? 4 CRT paper 2,

1.5 If a checklist response is written "as required" do you answer with

a. A known value (i.e. - flaps... 15*)? Yes 82 No 5.b. "As required"? Yes 15 No 72

1.6 Please indicate your feelings on the design of checklists you currently use.

a. List is too long Yes 12 No 69b. List doesn't cover enough Yes 10. No 2ac. Print is too small Yes 5 No 84d. Easy to skip items unintentionally Yes 25 No 54e. Dimensionsof list are too large Yes 10. No 23f. Convenient to use Yes 2Q No 16.g. Easy to use at night Yes5S No 21

• Is there sufficient supplementarylighting to make it readily visible? Yes 67 No 14

h. Organized in a manner that promotes a smoothflow pattern Yes 70 No 20.

i. Organized in a manner that reflects standardoperating procedure for the company Yes M No 2

j. Convenient place to stow the lists Yes 22 No 12k. Easy to locate "emergency" lists when needed Yes 5$ No 211. Do you feel that the checklist workload is

equally distributed among all crewmembers? Yes 70. No 1°.m. Any other comments

2. INTERRUPTIONS TO CHECKLIST USE

2.1 Please indicate on a scale of 1 to 10 (with 10 being the highest), which of the following activities tendmost to disrupt good checklist procedures. If they are particularly disruptive at one or another phase ofoperation, please indicate atwhich phase(s) - (i.e., ground, climb, cruise, descent, orapproach and landing).

(RANK)_ , score phasefsi

a. Ground personnel communications 5.05(2)b. Company radio 2*06(7) ~~~~~'c. Flight attendant requests 4^ (3) 'd. ATC communications 5^4 (i)e. Crew conversations 2^4 (9)f. Navigation requirements T4 (9)g. External taxiing distractions 4.25(4)h. Configuring aircraft for departure 2.09(10i~^__"i. External inflight distractions 2.82(8)j. Configuring aircraft for approach 3.27(6) ~~~~~~k. Aircraft abnormalities 4.06(5)1. Anyothers ""*

D-3

2.2 Do you feel there are times when the use of achecklist is disruptive to good operating procedures? Yes 22 No 58

(If "Yes," please explain)

2.3 What percent of the time is the 'Sterile Cockpit" concept, below 10,000 ft., adhered to by your airline'screws?

a. 100% of the time 21b. 75% of the time Hc. 50% of the time 1$d. less than 50% of the time 10

3. DEGREE TO WHICH PROCEDURES ARE DEFINED IN PILOT HANDBOOKS

3.1 Is a standardized method for the use ofchecklists spelledout in your companyoperating manual? Yes 88 No 6

3.2 If so, do most of the crews adhere todie prescribed method? Yes 85 No 7

3.3 Do you think the prescribed method couldbe improved upon? Yes 42 No 44

• How?

4. VARIATIONS IN CHECKLIST USE ATTRIBUTABLE TO CREWMEMBER CHARACTERISTICS

4.1 Do the individual crewmembers have any influenceon the manner in which a checklist is performed? Yes 6J> No 2j£

4.2 If so, does this result in variations, from onecrew to another, in the way in which the checklistsare performed? Yes 52 No 24

4.3 Does the influence of the individual crewmemberssometimes result in the checklists not beingperformed, or being performed in other than theprescribed manner? Yes 41 No 52

4.4 Any comments

D-4

5. IDENTIFY PROCEDURES OR CHANGES THAT MIGHT PROMOTE BETTER CHECKLIST USB

5.1 Do you have a personal "must check" list thatyou check regardless of how the formal checklistsare accomplished (such as the old "GUMP" list)?

• When do you use it?

5.2 Do you feel this sort of list would be useful toall front-end crews?

5?3 Do youhave specific checklists to cover undone items(such as for starting engines after a single-engine taxi)?

5.4 If 5.3 is 'No," what do you use for memory jogs to assurecompletion of checklist items?

• Coffee cup over the flap handle• Checklist between the throttles• Go through the list again• Other (please specif

5.5 Are your checklist procedures such that you findyourself reading checklists during periods ofotherwisehigh workload (i.e., taxiing in ORD,given a runway change in the middle of a tightapproach, etc.)?

5.6 If 5.5 is "Yes," do you

• Stop the list until it becomes less busy?

• Press on and hope that nothing gets missed?

5.7 Do crews for the different aircraft types in yourairline's inventory follow the same standardprocedures for checklist use?

• Under what conditions do they not?

D-5

Yes $5 No 22

Yes 44 No 42

Yes 25 No $?_

Yes 14 No 5|Yes 28 No 2ftYes 4ft No 28

Yes ft No 2$

Yes 42 No 14(some answered "yes"

to both)Yes 18 No3J

YesSfi No 5

nlaalWS^ii^J?™™™* f^.y™!"^ ™™™™"' wchecklisthelnfal ANJUZSE Please cJieck "Yes" or "No." YouV^lded Jomm^ts below each sactioTTwould ha

6.1 Create a core checklist, to be usedindustrywide, with variations by aircrafttype and operating environment Yes 22 No 52

6.2 Use ofautomated checklists wherever possible Yes 44 No 21

6.3 No use of checklists on the ground whenthe aircraft is moving Yes 27 No i

6.4 Use of color coding for easyidentification ofchecklists Yes 22 No 15

6.5 On paper checklists, use larger printor better letter spacing, or both Yes £2 No 22

6.6 Use a mechanical marker to markchecklist progress Yes 24 No 55

D-6

7. If you have anv suggestions or comments for improving checklist presentation, or a means of assuring thatchecklists are done in their entirety, please explain them.

8. BACKGROUND INFORMATION (Average data shown)

The following information will be used anonymously to help the survey team evaluate the data received.

8.1. Experience flying transport aircraft

a. Tvpes _i82b. Hours in typec. Seats flown

8.2 Experience flying other sophisticated aircraft

a. Tb. Hours in type_c. Seats flown

8.3 Hours in each seat collectively

a. Captain 4140b. First Officer 5570c. Second Officer 2910 (of these, 22 had no 2nd officer time.)

8.4 Aircraft and seat currently flown

8.5 Age 45.78 (ranged from 31-66)

8.6 Sex Male 24 Female 1 (32 yr. old DC-9 Capt.)

8.7 Visual correction

a. None Yes Nob. Nearsighted Yes Noc. Farsightod Yes Nod. Other

e. Do you use corrective lenses whileflying Yes 2$ No 51

• single focal Yes No• bifocal Yes No• trifocal Yes No• top-and-bottom focal Yes No

8.8 Does your company have a specific policyon cockpit resource management? Yes ffl No 22

8.9° If so, do most of the Captainsadhere to the policy? Yes 52 No 12

• If not, do they basically adhereto Captain's autonomy? Yes 22 No 2

THANK YOU FOR YOUR TIME

Q_7 -ttVS. COVCRNMENT HUNTING OFTICCl KM •MUtt/llUI

;

.t

I