Military Aviation Incident Reporting from an HTO Perspective

Military Aviation Incident Reporting from an HTO Perspective Master Thesis in the Ergonomics and HTO program

Photo by: Peter Liander Copyright Saab AB

AnnaKarin Rosén

Sara Thor 2009

Supervisor: Prof. Kjell Ohlsson

Department of Management and Engineering

Division of Industrial Ergonomics

LINKÖPING UNIVERSITY

ISRN LIU‐IEI‐TEK‐A‐‐09/00661—SE

1

Abstract

The purpose of this study is to find out if the Swedish Air Force and the military aviation industry, Saab Aerosystems, use an HTO perspective in incident investigations. The research objectives are to explore existing organizational and accident models, analyze the reporting system using the models, and suggest possible improvements to the existing incident investigation system. Fishman’s (1999) model for pragmatic studies was used to describe the study’s theoretical approach. Triangulation by literary review, case study and interviews were used to ensure validity and reliability.

Three models were chosen: Rollenhagen’s HTO model (1997), Leveson’s STAMP (2002), and Shappell and Wiegmann’s HFACS (2000). The models were further tested in the case study, revealing that their usefulness depends on type of incident, and that they leave a lot up to the investigator.

Interviews were conducted with key individuals at Saab Aerosystems as well as at the Swedish Armed Forces’ Headquarters. The interviews showed that the incident reporting system in question is a well functioning one, mainly due to the blame‐free culture, multidisciplinary investigative teams, good feedback and a generally good knowledge of the HTO concept. Difficulties within the organizations exist due to organizational boundaries between the manufacturer and the operator, and recurring structural reorganizations within the Swedish Armed Forces. Saab acknowledges the emphasis on technical issues. The Flight Safety department at the headquarters is currently working with implementing HFACS into the Flight Safety Database.

Our recommendations for Saab Aerosystems and the Air Force include:

For Saab Aerosystems:

• Evaluate and learn from existing HTO work within the organization, and incorporate the results into existing processes and instructions for incident investigations.

• Make use of the theoretical models, where applicable, in the processes as well. • Focus on HFACS in order to synchronize with the Air Force. • Consider employing an HTO specialist.

For the Air Force:

• Consider influences from other HTO models, like Rollenhagen’s or Leveson’s models, while still in the development phase of implementing HFACS.

• Perform risk analyses regarding the effects on flight safety when doing reorganizations.

For both Saab Aerosystems and the Air Force we recommend:

• Use Leveson’s model STAMP for complex and serious incidents, comprising of organizational, human and technical aspects.

Keywords:

accident model, aviation safety, HTO, human error, incident, investigation, reporting system

2

Sammanfattning

Denna studie har som syfte att ta reda på om det svenska Flygvapnet och den militära flygindustrin, Saab Aerosystems, använder ett MTO‐perspektiv vid incidentutredningar. Forskningsmålen är att utforska befintliga organisations‐ och olycksmodeller, att analysera rapporteringssystemet med hjälp av modellerna, samt att föreslå möjliga förbättringar på det existerande incidentrapporterings‐ och utredningssystemet. Fishmans (1999) modell för pragmatiska studier användes som teoretisk utgångspunkt. För att säkerställa validitet och reliabilitet användes triangulering, bestående av litteraturstudie, fallstudie, samt intervjuer.

Tre modeller valdes ut: Rollenhagens MTO‐modell (1997), Levesons STAMP (2002) och Shappell och Wiegmanns HFACS (2000). Modellerna användes i fallstudien, med resultatet att modellernas användbarhet bedömdes variera beroende på typ av incident, samt att de lämnar mycket åt utredarens erfarenhet.

Intervjuer genomfördes med nyckelpersoner på Saab Aerosystems och på Försvarsmaktens Högkvarter. Intervjuerna visade att rapporteringssystemet i fråga är väl fungerande, främst tack vare rapporteringskulturen, de multidisciplinära utredningsgrupperna, god återkoppling och en generellt sett god kännedom om MTO‐konceptet. Organisatoriska svårigheter är dels gränsdragningen mellan tillverkare och användare, dels återkommande omstruktureringar inom Försvarsmakten. Saab medger också att deras fokus ligger främst på det tekniska. Flygsäkerhetsavdelningen på Högkvarteret arbetar med att införa HFACS i flygsäker‐hetsdatabasen.

Våra rekommendationer till Saab Aerosystems och till Flygvapnet inkluderar:

Till Saab Aerosystems:

• Utvärdera och drag lärdom av befintligt MTO‐arbete inom organisationen, inkludera dessa resultat i befintliga processer och instruktioner för incidentutredningar.

• Utnyttja fler teoretiska modeller i processen, där så är lämpligt. • Fokusera på HFACS för att synkronisera med Flygvapnet. • Fundera på att anställa en MTO‐specialist.

Till Flygvapnet:

• Fundera på att också införa delar av andra MTO‐modeller, som Rollenhagens eller Levesons modeller, under implementeringsfasen av HFACS.

• Genomför riskanalyser av påverkan på flygsäkerhet vid omorganisationer.

Till både Saab Aerosystems och Flygvapnet rekommenderar vi:

• använd Levesons modell STAMP vid mer komplexa och allvarliga incidenter som innefattar organisatoriska, mänskliga och tekniska aspekter.

3

Contents Abstract ...................................................................................................................................... 1

Keywords: ................................................................................................................................... 1

Sammanfattning ......................................................................................................................... 2

1 Introduction ....................................................................................................................... 7

1.1 Background .................................................................................................................. 7

1.2 Problem Description .................................................................................................... 9

1.3 Research Objectives .................................................................................................... 9

1.4 Delimitations ............................................................................................................. 10

2 Research Approach and Methods.................................................................................... 11

2.1 Theoretical Research Background ............................................................................. 11

2.2 Triangulation ............................................................................................................. 13

2.3 Case Study ................................................................................................................. 14

2.4 Interview Methods .................................................................................................... 14

3 Frame of Reference ......................................................................................................... 17

3.1 HTO Background ........................................................................................................ 17

3.2 Accident Theories and Safety .................................................................................... 18

3.3 Human Error .............................................................................................................. 21

3.4 Teamwork .................................................................................................................. 22

3.5 Accident and Incident Reporting Systems ................................................................ 22

3.6 Organizational Models .............................................................................................. 24

3.6.1 Eklund: Extended HTO‐framework .................................................................... 24

3.6.2 Porras and Robertson: Organizational Development ........................................ 25

3.6.3 Rollenhagen: HTO from a Safety Perspective .................................................... 26

3.7 Accident Models ........................................................................................................ 28

3.7.1 Sklet: HTO‐analysis ............................................................................................. 29

3.7.2 Leveson: STAMP ................................................................................................. 30

3.7.3 Shappell and Wiegmann: HFACS ....................................................................... 33

3.8 Comparative Analysis of Models ............................................................................... 35

3.8.1 Conclusions ........................................................................................................ 40

4 The Reporting System for Military Aviation in Sweden ................................................... 43

4.1 Introduction ............................................................................................................... 43

4.2 The Reporting System in the Air Force ...................................................................... 44

4.3 Incident Reporting and Analysis at Saab Aerosystems ............................................. 46

4.3.1 Daily Product Meeting (DPM) ............................................................................ 47

4

4.3.2 Airworthiness Board (AWB) ............................................................................... 48

4.3.3 Product Safety Board (PSB) ................................................................................ 48

4.3.4 Fault Hazard Analysis (FHA) ............................................................................... 48

5 Case Study ........................................................................................................................ 51

5.1 SK60 Canopy Burst .................................................................................................... 51

5.1.1 What Happened ................................................................................................. 51

5.2 Rollenhagen ............................................................................................................... 51

5.3 Leveson ...................................................................................................................... 54

5.4 Shappell and Wiegmann ........................................................................................... 56

5.5 Conclusions ................................................................................................................ 58

6 Interviews ......................................................................................................................... 61

6.1 The Interviews ........................................................................................................... 61

6.2 Interview Analysis and Conclusions .......................................................................... 61

6.2.1 HTO Issues: General Views and Practical Applications ...................................... 62

6.2.2 Incident Investigation Methods and Models ..................................................... 64

6.2.3 Incident Reporting Systems and Reporting Culture .......................................... 65

7 Discussion and Recommendations .................................................................................. 67

7.1 Discussion and Conclusions ....................................................................................... 67

7.2 Recommendations .................................................................................................... 70

7.3 Methodological Discussion ....................................................................................... 71

7.4 Future Research ........................................................................................................ 71

8 References ....................................................................................................................... 73

8.1 Publications ............................................................................................................... 73

8.2 Internet Sources ........................................................................................................ 74

8.3 Reports ...................................................................................................................... 75

8.4 Documents supplied by Saab Aerosystems AB ......................................................... 75

9 Appendix .......................................................................................................................... 77

9.1 Appendix 1 ................................................................................................................. 77

9.2 Appendix 2 ................................................................................................................. 78

9.3 Appendix 3 ................................................................................................................. 83

5

List of Figures

Figure 1: Fishman's model for pragmatic studies, from Fishman (1999, p. 11) ...................... 12 Figure 2: Illustration of the planned iterations in this study. .................................................. 13 Figure 3: Types of accidents from Rasmussen (1997, p. 197) ................................................. 20 Figure 4: Heinrich’s “Safety Iceberg” from Johnson (2003) (p.23). ......................................... 23 Figure 5: Roles that contribute to operation of reporting systems from Johnson (2003) (p.90‐104). ......................................................................................................................................... 24 Figure 6: Eklund’s extended HTO framework (Eklund, 2003). ................................................. 24 Figure 7: Porras and Robertson: Factors constituting the organizational work setting (Dunette & Hough (1992), p. 729). .......................................................................................... 26 Figure 8: Structural model (Rollenhagen 1997, p. 17). ............................................................ 27 Figure 9: A suggested HTO framework model (Rollenhagen & Kahlbom, 2001, p. 3). ........... 28 Figure 10: HTO analysis worksheet (Sklet, 2002, p. 52). ......................................................... 30 Figure 11: Leveson: General model of socio‐technical control structure for both system development (on the left) and system operation (on the right) (Leveson, 2002, p. 62). ....... 31 Figure 12: The relationship between mental models (Leveson, 2002, p. 33). ........................ 32 Figure 13: The “Swiss cheese” model of human error. From Shappell and Wiegmann (2000), and from Reason (1990). ......................................................................................................... 33 Figure 14: Eklund. .................................................................................................................... 37 Figure 15: Porras and Robertson. ............................................................................................ 38 Figure 16: Rollenhagen. ........................................................................................................... 38 Figure 17: Sklet. ........................................................................................................................ 39 Figure 18: Leveson. .................................................................................................................. 39 Figure 19: Shappell and Wiegmann. ........................................................................................ 39 Figure 20: Overview of RML structure. .................................................................................... 43 Figure 21: Overview of the reporting structure in the Swedish Air Force. .............................. 44 Figure 22: Overview of incident reporting for the Swedish Air Force, including distribution of reports. ..................................................................................................................................... 46 Figure 23: Overview of incident reporting for the Swedish Armed Forces, FMV and Saab Aerosystems (Source: Saab Aerosystems; internal document). .............................................. 47 Figure 24: Illustration of what happened during the SK60 incident in accordance with Rollenhagen description. ......................................................................................................... 51 Figure 25: Illustration of why the SK60 incident happened in accordance with Rollenhagen description. .............................................................................................................................. 52 Figure 26: Illustration of work and barrier analysis for the SK60 incident in accordance with Rollenhagen’s description. ....................................................................................................... 53 Figure 27: Illustration of work and barrier analysis for the SK60 incident including management and system level in accordance with Rollenhagen’s description. ..................... 53 Figure 28: Rollenhagen’s description (1997, p. 17). ................................................................ 54 Figure 29: Rollenhagen’s Venn diagram applied to the SK60 case. ......................................... 54 Figure 30: Hierarchical safety control structure for SK60 case. .............................................. 55 Figure 31: Control structure for canopy changing in the SK60 case. ....................................... 56 Figure 32: First step in HFACS analysis, time line. ................................................................... 57 Figure 33: Continuation of HFACS analysis. ............................................................................. 57 Figure 34: HFACS, classification of acts. .................................................................................. 58

6

List of Tables

Table 1: Statistics of accidents in the Swedish Air Force. Source: Anders Hägg, Saab Aerosystems (2008). ................................................................................................................ 21 Table 2: Criteria for accident model evaluation, after Hendrick and Benner (1987), with our new add‐ons. ............................................................................................................................ 36 Table 3: Our criteria for accident model evaluation. ............................................................... 37 Table 4: Rating of models according to the new evaluation criteria. ...................................... 37 Table 5: Severity categories for occurred incidents (Source: Saab Aerosystems; internal document). ............................................................................................................................... 49 Table 6: Probability levels for occurred incidents (Source: Saab Aerosystems; internal document). ............................................................................................................................... 49 Table 7: Hazard Risk Index (HRI) for occurred incidents (Source: Saab Aerosystems; internal document). ............................................................................................................................... 50

7

1 Introduction

The first chapter starts with the background and motivation for this master thesis. Following the background are the problem description and research objectives. The delimitations of the thesis conclude the chapter.

1.1 Background In today’s high‐tech society we see many examples where technology moves toward a higher level of integration of several systems, which in turn increases the system’s complexity dramatically. Earlier it has been sufficient to first analyze design propositions and then test the design solution in order to find a design, which is safe for use. Today it is difficult to see through all the combinations of failure modes or how a specific failure can propagate throughout the system.

Also, data shows that incidents and accidents, at least within the aviation industry, have changed so that instead of a majority being solely attributable to mechanical failure a majority is now instead attributable to some kind of human error (Wiegmann & Shappell, 2001). New methods for safety analysis and accident prevention are needed.

An important source for prevention of future accidents is the lessons learned from accidents and incidents that occur with systems in operation. Accident models are used to explain how accidents occur. It is questioned whether the system safety accident models of today, which have their roots in industrial safety, really do find the appropriate measures to ensure that the accidents do not happen again in these new complex and integrated systems (Leveson, 2002).

Carl Rollenhagen (1997) considers the interaction between Human, Technology and Organization (HTO) to be conclusive for safety. He summarizes:

Overall the HTO field can preliminary be defined as a perspective of safety whose purpose is to study how man’s physical, psychological and social prerequisites interact with different technologies and forms of organizations, and from this knowledge work for increased safety.

(Rollenhagen1, p 10, 1997)

Within military aviation in Sweden the importance of learning from incidents has been acknowledged for a long time. There is a working routine for analyzing occurred incidents in order to prevent the recurrence of flight safety critical events. This working routine includes the end users in the Air Force as well as the manufacturers. For many military aerial vehicles, including the Gripen system, the manufacturer in Sweden is Saab Aerosystems.

1 Translated from Swedish: Övergripande kan MTO‐området preliminärt definieras som ett perspektiv på säkerhet vars syfte är att studera hur människans fysiska, psykologiska och sociala förutsättningar samspelar med olika teknologier och organisationsformer samt utifrån denna kunskap verka för ökad säkerhet.

8

When really serious incidents or accidents occur, such as for instance loss of aircraft, the investigation is carried out by the Swedish Accident Investigation Board (sw. SHK, “Statens haverikommission”). SHK was established in 1978 and investigates civil as well as military aviation accidents. According to the SHK web site (Statens Haverikommission, 2008a) they also investigate an incident if it is believed that the incident could have lead to a serious accident. An accident investigation shall always answer the following three questions:

1. What happened? 2. Why did it happen? 3. What can be done so that a similar incidence/accident won’t happen again?

When doing accident or incident investigations different models can be used. The challenge for today’s practitioners is to use a model which takes into consideration all the changes that are stretching the limits of current accident models and other safety engineering techniques. In her draft book, System Safety Engineering: Back to the Future (2002), Nancy Leveson mentions changes she considers to be main drivers for a new and updated model (p. 4):

• Fast pace of technological change; both regarding the amount and significance of the technological changes and also regarding the time that is allowed from idea to realized product.

• Changing nature of accidents; caused by more digitalized systems that make approaches that worked on electromechanical components ineffective in controlling accidents that arise from the use of digital systems and software.

• New types of hazards; as in for instance, information systems causing the potential for loss of information or incorrect information leading to unacceptable losses.

• Increasing complexity and coupling; we try to, and do, build systems that are beyond our ability to intellectually manage.

• Decreasing tolerance for single accidents; at the same time we are building systems that can cause harm to an increasing number of people and impact future generations through environmental pollution and genetic damage.

• More complex relationships between humans and automation; humans are increasingly sharing control of systems with automation and moving into positions of higher‐level decision making with automation implementing the decisions.

• Changing regulatory and public views of safety; the responsibility is changing from the individual to the government.

Leveson thinks that event‐based accident models very well describe and explain physical phenomena, but that they are inadequate to explain accidents involving organizational and social factors as well as human decisions and software design errors especially in highly adaptive, tightly‐coupled and interactively complex socio‐technical systems (p. 40).

In order to succeed with a correct analysis of the occurred incident and thereby prevent similar flight safety critical events it is therefore important to have both a systems and an HTO (Human, Technology and Organization) perspective. In the aviation world the human in the system includes both the pilot, who has an extremely complex and cognitively demanding work situation, but also all technical personnel that handle the aircraft, for instance for maintenance reasons, repairs, clearing of the aircraft before a flight etcetera. The organization in this thesis includes the manufacturer, Saab Aerosystems, and its design

9

organization, but also the end users in the Air Force and the Swedish Defense Material Administration (FMV, Försvarets Materielverk). The technology is the aircraft with its support equipment such as external loads, pilot equipment (helmet/mask, g‐suit etc.), test benches for maintenance work and so on.

One incident that shows the need for different perspectives on accident investigation is the military exercise that was conducted in 2005 with a Swedish military rescue helicopter (HKP10). The helicopter collided with the surface of the sea and was completely broken down. The crew of six all survived, but one of the crew members was injured. There were no signs of technical failures in the helicopter and the following investigation concluded that the main causes of the accident were “deficiencies of the organization, competence, quality management and resources within the Armed Forces as regards to implementation, management and supervision of military aviation….”2 (Statens Haverikommission, 2008b)

This incident is not the only one of its kind in the Armed Forces in Sweden3. This begs the question if something can be done to alleviate such problems, and further improve overall safety awareness

1.2 Problem Description There is no doubt that it is valuable to use an HTO perspective when doing incident investigations, but it is maybe easier said than done. It might be that in some parts of the chain for reporting incidents, consideration is taken to Technology, Human and Organization, but in other parts of the chain it is not.

The overall problem description of this master thesis is the question whether the Swedish Air Force and the military aviation industry, mainly Saab Aerosystems, use an HTO perspective when investigating incidents, either partly or throughout the whole reporting chain.

1.3 Research Objectives The research objectives of this master thesis are to:

• theoretically explore and analyze organizational models and accident models that can be suitable for incident investigations from an HTO perspective, and

• analyze the incident reporting system in the Swedish Air Force using the selected models and

• suggest a model that the Swedish Air Force and Saab Aerosystems can work with, or suggest improvements of already used accident and incident models in the Swedish Air Force and at Saab Aerosystems.

An improved management and analysis of occurred incidents will lead to a lower probability that similar incidents and accidents will happen again in the future. By making sure that this is done with an HTO perspective, we hope to contribute to make military aviation even safer that it is today.

2 Translated from Swedish. 3 See for instance Statens Haverikommission (2008c).

10

1.4 Delimitations This master thesis analyzes incidents that occur within the Swedish Air Force or within Saab Aerosystems’ aviation operations. For aircraft crashes or serious incidents the Swedish Accident Investigation Board (SHK) is involved and that type of events and their analyses are not included in this master thesis.

There are a great number of HTO and accident models to choose from when performing an incident or accident investigation. We have analyzed the models that were introduced to us during the university HTO course 2007/2008 and also the models that we found most relevant during our overview of the literature in this field.

The Swedish Defense Material Administration (FMV, Försvarets Materielverk) and the Military Flight Safety Inspectorate (FLYGI) plays an important role for incident reporting and analysis in the Swedish Air Force. In this thesis their role is however only described peripherally and not fully included in the analysis and discussion, due to lack of time and resources.

11

2 Research Approach and Methods

This chapter begins with a description of the epistemological standpoint we have chosen for this thesis, explaining both the pragmatic perspective as well as the iterations we go through. Within the framework of pragmatism we make use of triangulation through a literature review, a case study and interviews. These research methods are described in this chapter.

2.1 Theoretical Research Background This study has a pragmatic epistemological approach. Theoretical orientation, case studies and interviews are used in order to fulfill the research objectives. We, the authors, believe that our work to a great extent aligns itself with the model for pragmatic inquiry presented in the book The case for pragmatic psychology by Fishman (1999). Fishman describes a working model (p. 155ff) that is problem oriented and that makes use of case studies as well as contextualized knowledge.

The pragmatic perspective constitutes a middle road between the positivist paradigm and the hermeneutic paradigm. In its most typical form a positivist study starts with a theory, and tests this theory by means of hypothesis testing, while a hermeneutic study starts with a conceptual problem, a point of view or a subject, but lacks a testable hypothesis.

A pragmatic study, according to Fishman, is similar to a hermeneutic study in that it starts out with a problem or a point of view and has an idea of where it will end rather than a testable hypothesis. But unlike a hermeneutic study a pragmatic study does not view all knowledge as conceptualized and as an object of interpretation by all participants.

Fishman’s model is developed for psychological research, and is perhaps foremost a way to describe the interaction between the psychotherapist and the patient. It does, however, at the same time describe a framework for an epistemological paradigm situated between the two traditional paths; interpretivism and positivism. The pragmatic approach lends attributes from both paradigms, but stands on its own ground. We find it to be a good way to describe our work.

A pragmatic study, according to Fishman, can be visualized as in Figure 1. It starts off with a client (A) who presents a problem. The client can be an individual, a group or an organization. The problem is then assessed (D) by the researcher, who is using both experience and research (C), as well as a guiding conception (B), i.e. his or her assumptions about the problem and about the issues at hand. From the assessment a plan is formulated (E), including a description of the situation and a goal to be achieved. Next step is action (F), where the plan is carried out.

The outcome of the action is then evaluated (G), the new information derived from this evaluation is fed back through assimilation (J), to merge with and expand prior research and experience (C), and through accommodation (I), to expand the guiding conceptions (B).

If the outcome is not satisfactory (K), a new loop, or iteration, is started, beginning anew from the formulation stage (E). If and when the evaluation process finds the outcome satisfactory (H), the iteration ends, and a concluding evaluation (L) ensues.

12

In this study the client is represented by our wish (and the interest harbored by Saab Aerosystems and the Air Force) to find the answers to the underlying problem description. We differ, however, from Fishman, in that we plan our iterations in advance, making them the structure of our methodological approach, rather than deciding between each iteration if we have reached the goal or not.

This study’s first iteration begins with assessment of relevant research in the area of organizational and accident models, formulation of research objectives, the selection, description and finally the evaluation of the models.

The knowledge gathered in the first iteration modifies both the experience and the guiding conception, and leads into the second iteration which begins with an assessment of the models. Applicable models are selected along with a relevant case. The case study is then carried out, followed by an evaluation.

The third iteration concerns the planning, execution and evaluation of the interview study.

The fourth and last iteration in this study ties all the knowledge gathered about the reporting system in the previous iterations together, making it possible to meet the research objectives and move on to a concluding evaluation, i.e. the concluding remarks and recommendations of the study. The whole process, with all planned iterations is visualized in Figure 2.

We think that Fishman’s model is applicable and useful for understanding and describing our research methods. The only divergence, from our standpoint, is that we have planned the iterations in advance, rather than deciding as we go along.

Figure 1: Fishman's model for pragmatic studies, from Fishman (1999, p. 11)

13

Figure 2: Illustration of the planned iterations in this study.

2.2 Triangulation Triangulation can be defined as:

The use of two or more methods or techniques to investigate the same research question, or the collecting of’…information from several sources about the same event or behavior’

(Williamson, 2002, p. 334)

As can be seen in the citation from Williamson (2002), the two major types of triangulation are methods and sources. In our thesis we apply methods triangulation by using three different methods for our analysis. The methods we use are:

14

1. Literature research: for initial information about reporting systems and incident analysis methods in general as well as more specific information about the Swedish Air Force incident reporting system and incident analysis methods at Saab Aerosystems and in the Air Force.

2. Case study: to test the results from the literature research for ourselves by applying the theories on a real case, an occurred incident.

3. Interviews: to gather more information from people working with the system, to either verify or falsify our results.

The purpose of triangulation in research is to increase the credibility and validity (Bryman, A. (n.d.)) of the results. The advantage according to Williamson (2002, p. 36) of triangulation is that conclusions are likely to be more reliable if data are collected by more than one method.

2.3 Case Study There are several ways of doing research and a case study can be a useful means of investigating phenomena in their natural setting (Williamson, 2002, p. 121). Case studies, disputed as they may be, are a good method to get a nuanced view of the reality studied. They can help produce concrete, context‐based knowledge by providing good examples (Flyvbjerg, 2006).

When selecting a case one must make sure that it is a selection which is relevant to the applied discipline and industry. Either single‐case or multiple‐case designs may be used in case study research. (Williamson, 2002, p. 115). We have chosen to conduct a single‐case study. The selected case was chosen with the help of representatives from Saab Aerosystems and the reasons for choosing this specific case were:

• The incidents happened to aircraft SK60 which is a smaller jet, used for transportation and training in the Swedish Air Force. The SK60 is an older aircraft compared to the fighter JAS39 Gripen and information regarding the aircraft is generally less sensitive.

• The selected case included technical, as well as organizational and human, aspects and was therefore considered a relevant case.

• The incident happened a few years ago and reports and measures were finished and available. It was not however so old that it was forgotten by investigators.

When using case studies it is important to keep in mind that there may be difficulties in generalizing the research results. Also, the data collection and analysis processes may be influenced by subjectivity (Williamson, 2002, p.121).

2.4 Interview Methods An interview is, according to Kvale (1996, p. 6), a conversation with a specific goal. We have, as a part of our data collection, chosen to conduct a series of semi‐structured interviews with key individuals in the incident reporting system. One goal with the interviews is to get a better picture of the way incident investigations are carried out within the Air Force and Saab Aerosystems, than we would get by just looking at the documentation and the regulations. Another goal is to receive more information about the incident reporting system itself.

15

In a semi‐structured interview (Jordan, 1998, p. 68) the researcher has a set of predetermined questions, regarding the issues relevant to the study at hand. The respondents are allowed to answer relatively freely, but might get prompted to stay within relevant topics. Due to the predetermined questions, systematic analysis of the data is facilitated.

We chose the form of semi‐structured interview rather than a structured interview or questionnaire to increase the validity of the data. Jordan describes the chance of retaining validity by using a semi‐structured interview rather than a structured interview or a questionnaire. He states that "[t]he interactive nature of an interview, then, can potentially make the data more valid than that which is gathered from questionnaires." (Jordan, 1997, p.69) An unstructured interview was never deemed suitable, since we wanted a higher degree of control over the data‐collection.

Kvale (1996, p. 187) describes five basic methods of analyzing qualitative data from interviews:

• Categorization: the interview is coded into categories, giving more structure to the material.

• Condensation: the respondents' statements are rephrased into a more concise form, yielding a shorter material to work with.

• Narrative structuring: the text is organized temporally and socially in order to put it in context.

• Meaning interpretation: interpretation recontextualizes the statements in the interview, giving it broader frames of reference.

• Ad hoc methods: a collection of methods can be used, ranging from common sense‐approaches to quantitative methods, with the goal of creating meaning within the material.

Given our goals with the interviews, to find more information about the incident reporting system and investigations, and comparing that information to regulations and data from the case studies, categorizing is the best suited method.

We conducted interviews with representatives from both the Swedish Armed Forces’ Headquarters and from the aircraft manufacturer Saab Aerosystems in order to try and cover as much of the incident reporting chain as possible and receive different aspects and views of the incident analyses. The interviews are described in more detail in Chapter 0 and the results are discussed in Chapter 7.

17

3 Frame of Reference

The theoretical frame of reference for this master thesis first describes some HTO background, theories about accident theory, human error, and team work. Theories concerning reporting systems are then discussed. The chapter continues with descriptions of organizational models and accident models, and is concluded with a comparative analysis of the models.

3.1 HTO Background The concept of Human Factors originates from the first half of the 20th century. Wickens and Hollands (2000, p.3f) identifies three sources for its development:

1. Experiences during the World War II led to the conclusion that it was not enough to train operators of advanced equipment or pilots of airplanes in order to avoid accidents. The realization grew that the machines might need to change instead. This led to experimental psychologists starting to analyze the human‐machine interface, trying to find out what was wrong and suggest solutions.

2. The development in technology led to increasingly more complex systems, making it all the more important that the system was designed with the interaction with the operator in mind. Analyzing the relevant tasks, and distributing them between the machine and the operator, became highly relevant, especially as the speed of operation of machines grew.

3. At the same time the rise of information theory helped integrate humans and machines by providing terms with which to describe human behavior. With terms such as feedback, channel and bandwidth, replacing the stimulus‐response terminology of the behaviorists, human behavior could be described with the same terms as the system it was to interact with.

The Swedish HTO (Human – Technology – Organization; in Swedish MTO Människa –Teknik –Organisation) concept was coined in the 1980s in the nuclear power industry. The goal was to decrease the chance of accidents and increase safety (Eklund (2003)). The background to the introduction in Sweden was primarily the serious accident in March 1979 at Three Mile Island (TMI) outside Harrisburg in USA. (Andersson, 2000)

The term HTO is today a well established and generally known concept. It is still extensively used and further developed within the Swedish nuclear power industry, but it has also gained a wider use. Today it includes all aspects of the interaction between human, technological and organizational factors, not just nuclear reactor safety related issues.

The field of HTO can be described in many ways; in the introduction chapter we cite Rollenhagen (1997), and his definition (see section 1.1). Olle Andersson (2000) says that HTO, and especially within the nuclear power industry, can be said to include three different but connected views or areas (ibid. p. 4). These are:

1. Analytical tools: during the years different HTO tools have been developed. The first analytical tools were meant to analyze (complex) incidents, identify broken barriers and causes and suggest and recommend measures in order to prevent accidents. Preventive analytical tools have later been developed.

18

2. HTO as a field of expertise: HTO has been considered a relatively small field for

especially ergonomics and social sciences. This interpretation is according to Andersson too narrow. In order to understand the interaction between humans, technology and organizational factors knowledge from several different expert fields (such as ergonomics, Human Factors, psychology, social science etc.) is needed. HTO has developed into a multi‐scientific discipline and the collaboration between several different fields of expertise is necessary.

3. HTO as a system concept related to safety: HTO can also be said to be a perspective, a way of developing a safety culture philosophy which focuses on the entire socio‐technical system, including technological, human and organizational factors.

The boundaries between these three areas of HTO are not always very clear and there are strong links and dependencies between them. In order to understand causes and different connections when HTO analyses are performed, Andersson (2000) stresses that it is necessary to have a systems thinking.

Nancy Leveson seems to agree with Andersson and in her book Safeware (1995) she says that “accidents have begun to be viewed in terms of the interactions among humans, machines, and the environment. The components of a system are interrelated – each part affects the others either directly or indirectly.” (p. 199).

3.2 Accident Theories and Safety An accident is, according to Wikipedia, defined as:

[…] a specific, identifiable, unexpected, unusual and unintended external event which occurs in a particular time and place, without apparent or deliberate cause but with marked effects. It implies a generally negative probabilistic outcome which may have been avoided or prevented had circumstances leading up to the accident been recognized, and acted upon, prior to its occurrence.

(Wikipedia, December 1, 2008)

So, in other words, an accident is when something bad happens that could have been avoided if the circumstances had been different. Leveson has a similar definition in Safeware (1995), but she concludes that the accident is not necessarily unexpected: “an accident is an undesired and unplanned (but not necessarily unexpected) event that results in (at least) a specified level of loss.” (ibid. p. 175)

An incident (or near miss) is an event that involves no loss (or only minor loss), but with the potential for loss under different circumstances (Leveson, 1995, p. 176).

Another basic concept when dealing with accidents and incidents is safety. It is often defined as being free, being “safe” from accidents. MIL‐STD‐882C, which is a US standard that is often used for military applications, defines safety as:

Freedom from those conditions that can cause death, injury, occupational illness or damage to or loss of equipment or property or damage to the environment.

(Department of Defense, 1993, p. 6)

19

There have been different ways of viewing accidents over the years. Two main perspectives are:

1. Normal Accidents Theory: in this view it is argued that accidents are inevitable in complex and tightly coupled systems and safety is only one of a number of compe‐ting objectives (Perrow, 1984). Charles Perrow, who is one of the spokesmen for this theory, also argues that redundancy often causes accidents, since it increases interactive complexity and encourages risk‐taking. He also states that organizations cannot train for unimagined, highly dangerous operations.

2. High Reliability Theory: this perspective argues instead that accidents can be pre‐vented through good organizational design and management and that safety is the prioritized organizational objective in high‐risk operations (Reason, 1997). Sagan (1993) summarizes that four specific conditions need to be met in order for an organization to be highly reliable: 1) political elites and organization leaders place a high priority on safety and reliability, 2) significant levels of redundancy exists, so that backup units can compensate for failures, 3) error rates are reduced through decentralization of authority, strong organizational culture and continuous operation and training and 4) organizational learning takes place through a trial‐and‐error process.

Accidents do happen and Amalberti (2001, p 111) divides systems into three groups; dangerous systems, regulated systems and ultra‐safe systems:

• Dangerous systems have a risk of accident that is greater than one accident per 1000 events, 10‐3, and examples of dangerous systems are bungee jumping or mountain climbing. They usually correspond to personal quests for risk and thrills and safety measures are highly individual.

• Regulated systems have a risk of accident between one accident per 1000 events and one per 100 000 events. Driving, chemical industries or chartered flights are examples in this category. Typical safety tools are (1) regulations and procedures, (2) accident or near‐accident are often repetitions of stories of past accident or near‐accidents, (3) error‐resistant design and a reporting policy are present and (4) safety managers usually obtain results for newly implemented measures within a couple of years.

• Ultra‐safe systems have a risk of disaster below one accident per 100 000 or even one million safety units. Examples of such industries are the nuclear industry, civil aviation and railroads. The special features of ultra‐safe systems are (1) that they tend to be ageing, are over‐regulated, rigid and unadaptive, (2) accidents are different in nature, they result from a combination of factors, none of which alone can cause the accident, (3) reporting becomes less relevant in predicting major disasters and (4) system managers work for their successors, they don’t have time to obtain results from implemented measures (as for regulated systems). Therefore these systems tend to become a political rather than a scientific subject.

Rasmussen (1997, p. 198) does a slightly different categorization of accidents. He takes into consideration the frequency of accidents and also the magnitude of the loss connected to the individual accident. His categories are:

20

1. Frequent, small scale accidents (occupational safety); the hazards are related to a very large number of work processes and the level of safety over time can be directly measured by the number of LTIs (Lost‐Time‐Injuries) and casualties.

2. Infrequent, medium size accidents; safer system evolve from design improvements in response to analysis of the individual latest major accident. This category includes for instance hotel fires, train collisions, aircraft accidents etc.

3. Very rare and unacceptable accidents; design and operation of systems that fall into this category must be based on reliable predictive models of accident processes and probability of occurrence. The potential damage from these types of systems, especially large‐scale systems (e.g. nuclear power), is very large and the acceptable mean‐time between accidents so long that design cannot be guided by empirical evidence from past accidents.

These types of accidents are illustrated in Figure 3 and domain characteristics are described for each accident type.

Log. magnitude of loss from an accident

Log.

freq

uenc

y of

acc

iden

ts

Empirical strategy.

Domain Characteristics:•Complex set of hazard sources•Loosely coupled work system•Control by removing causes•Defined by statistical analysis

Frequent, small scale accidents: Occupational

accidents

Major accidents: Aircraft crashes: ferry accidents: train crashes: hotel fires

Large scale accidents: Nuclear power plant melt-down, chemical

plant run-away

45º slope

Analytical strategy.

Domain Characteristics:•Well-defined hazard•Tightly coupled system shaping

accident anatomy•Control of accident process after

release•Defenses identified by predictive

analysis

Evolutionary strategy.

Domain Characteristics:•Well-defined hazards•Loosely coupled systems•Control by removing causes•Defined by analysis of past

accidents

Figure 3: Types of accidents from Rasmussen (1997, p. 197)

For military aircraft systems the risk of disaster is measured in accidents per flight hours. A “disaster” (to use Amalberti’s term) is called a catastrophic event or mishap and it includes death of pilot, ground crew or third party person, loss of aircraft or irreparable damage to the environment. Military aviation can today be considered an ultra‐safe system, but about thirty years ago it was a dangerous system.

As can be seen in Table 1, the number of lost aircraft per 100.000 flight hours has decreased from 20.5 during the 1960’s to 3.6 during the 1990’s. It has decreased even further during

21

the last ten years. Also, the number of pilots killed per 100.000 flight hours has decrease significantly, from 12.6 during the 1960’s to less than 1.5 in recent times.

Table 1: Statistics of accidents in the Swedish Air Force. Source: Anders Hägg, Saab Aerosystems (2008).

1960‐1969 1970‐1979 1980‐1989 1990‐1999 Total number of flight hours

846,000 815,000 592,000 467,000

Total number of lost aircraft

173 103 49 17

Killed 109 48 17 7 Number of lost a/c per 100,000 fh

20.5 12.6 8.3 3.6

Killed per 100,000fh

12.6 5.9 2.9 1.5

3.3 Human Error James Reason describes human error as a planned sequence that “fails to achieve its intended outcome” (Reason, 1990, p. 9). Human error typically divides further into the following categories (Wickens & Hollands (2000) (chapter 12, p. 494ff)).

• Mistakes are the result of not understanding the situation properly. Errors like this can be knowledge based or rule based. A knowledge based mistake is a decision based on the wrong supposition. A rule based mistake arises when a person either applies a rule not applicable to the situation, or a rule that is incorrect in itself.

• Slips occur when the intended correct action is not carried out. This is likely to happen when the intended action is “captured” by a similar, well‐practiced behavior; this is called “capture error” and usually occurs when distractions take place.

• Lapses are the case when the intended action is not carried out at all, because of forgetfulness. This usually happens to actions that are part of a longer sequence of actions, with the effect that the whole sequence is faulty.

• Mode errors are actions performed correctly, but in the wrong setting. In other words, this can be a slip (performing the wrong action because of misjudging the context), or a lapse (forgetting what the context is). A good example is pressing the accelerator when the gear is in reverse, erroneously believing that the gear is in “one”.

Hollnagel (2002) means that human actions cannot be seen as either right or wrong, but it is only after the fact we can judge the outcome. He divides human actions along the following lines: (after Amalberti, 1996)

• Actions that are correctly performed, i.e. where the intended and actual outcomes correspond.

• Actions where the failure is detected and successfully recovered • Actions where the failure is detected but tolerated • Actions where the failure is detected but not recovered • Actions where the failure is not detected, for instance because the effects are latent

(Hollnagel, 2002, p. 5)

22

This view of human actions combined with Wickens and Hollands (2000) view, described above, gives us both an understanding of why human errors occur and an awareness, that it is not just the error itself that is important, but also how the failure is dealt with by the operator.

This more advanced view of human action helps us understand how to develop responses to actions that might cause harm to a system. Depending on how the analysis of an accident deals with the notion of human actions we get different answers to what caused the accident. This is further discussed in section 3.7 Accident Models.

3.4 Teamwork A team is a group with certain characteristics: Teams are structured and coordinated and all member works towards the same goal. The cohesiveness and social identity of a team makes the members more committed to work together (Forsyth, D. (1999) p. 165).

It is common when doing accident investigations that several people work together. Each member of the team is a professional who is specialized in his or her field. It may for instance be an engineer, a psychologist, an HTO expert etc. Teams put together out of professionals with different areas of specialization are called multidisciplinary. The complementary skills in such a multidisciplinary team are especially appropriate for complex tasks with interdependent subtasks (Wikipedia, March 10, 2009).

Multidisciplinary teams are, according to Firth‐Cozens (2001), likely to be better for everyone, but to keep them working well needs skill as well as recognition that this is always a long term task requiring constant attention and adjustment.

3.5 Accident and Incident Reporting Systems If occurred incidents and accidents are to be analyzed and understood in order to prevent similar occurrences in the future, they must naturally first somehow be reported. Renborg, Jonsson, Broqvist and Keski‐Seppälä (2006) found in their research analysis of reporting systems that the need for a reporting system can be summarized by:

[…] the organization will never learn from mistakes not reported.

(Renborg et al., 2006, p. 6)

Renborg et al. (2006) also states the importance of confidence and trust of the people filling out the reports of the reporting system and that the whole system has a blame‐free culture.

As an incentive for reporting incidents Johnson (2003) describes Heinrich's pioneering studies in occupational health and safety. These studies suggested an approximate ratio of one accident to thirty occurrences involving major injuries to three hundred 'near‐miss' incidents, see illustration in Figure 4.

23

300 near misses

30 injuries

1 death

Heinrich ratio (1932)

Figure 4: Heinrich’s “Safety Iceberg” from Johnson (2003) (p.23).

Reporting systems can be either:

• Open: reveals the identity of contributors. • Anonymous: contributors can entirely hide their identity. • Confidential: allows for limited disclosure of the contributor’s identities but only to

trusted parties (Johnson, 2003, p. 28‐29)

There may also be problems associated with incident reporting. When Johnson (2003) looked at the Heinrich’ ratios for General and Commercial Aviation he came to the conclusion that the protection offered to the people reporting the incidents can introduce biases. Especially pilots are more likely to report an unfavorable event if their livelihood is at risk or if they are concerned that their actions may be reported by colleagues and co‐workers (Johnson, 2003, p. 44).

The main barriers for success of a reporting system are according to Johnson (2003, p. 28‐29) the following:

• Punishment/Enforcement: Potential information providers may be concerned that company management and/or regulatory authorities might use the information for punitive or enforcement purposes.

• Public Access: In some countries public access, including media access, to information that is held by government agencies may be a problem.

• Criminal Sanctions: A problem in some countries is the fear of criminal prosecution for regulatory infractions.

• Civil Litigation: Probably the most significant problem, especially in the U.S., is the concern that the information will be used against the contributor in accident litigation.

Johnson (2003, p. 90‐104) describes a number of different roles that together contribute to the successful operation of many incident reporting systems. He states that the roles are generic in the sense that they represent key activities during the reporting, analysis and subsequent implementation of safety recommendations. These roles are:

24

Reporter Initial Receiver Incident Investigator Safety Manager Regulator

Figure 5: Roles that contribute to operation of reporting systems from Johnson (2003) (p.90‐104).

These roles can work together in different ways, depending on the level of the system (for instance local, national, international etc.) and the nature of the operation that is being monitored. The anatomy of the reporting system in the Swedish Air Force will be described in Chapter 0 of this thesis.

3.6 Organizational Models During the course in Human, Technology and Organization (HTO) for the Master program in Ergonomics / HTO we were introduced to several HTO models. The ones that we deemed interesting for accident investigation are described in this chapter4 and they are further analyzed in section 3.8.

3.6.1 Eklund: Extended HTOframework The goal of Eklund’s extended HTO framework (2003) is improvement of the efficiency of work systems. This is done by analyzing supporting and contradicting mechanisms within the system. Eklund’s model puts human activity in the forefront of the model. The central issue is the way human activity interacts with technology, organization and environment.

Figure 6: Eklund’s extended HTO framework (Eklund, 2003).

Eklund calls his smallest unit of analysis human work activity. Work activities can be summed together, forming sub‐processes, processes, main processes and operations. Each

4 Apart from the three models presented here we chose not to include a fourth model mentioned in the course. This model is called The ATOM model (Lundqvist, Björkman, Docherty, Hill and Ullmark (1997)). We chose to not include this model due to its lack of theoretical depth.

25

work activity has a goal, and that goal can either be supported or contradicted by the interactions with technology, organization and environment, see Figure 6. Depending on how the main components support or contradict the goal, the cost of performing the work activity varies. For an organization the costs can be things like quality, productivity and turn‐over, for the individual it can be things like health, safety and well‐being.

Relating this to accident investigation, one might use the model to find system processes responsible for human error, or to get a view of the whole work system, but the model would not be a very good tool for investigating more technical kinds of accidents.

3.6.2 Porras and Robertson: Organizational Development Porras and Robertson’s (1992) model is mostly a tool for analyzing and managing organizational change. It is designed to help identifying what can be changed within the organization, thus guiding the change. The underlying thought is that meaningful and persistent change can only be possible if the individuals within the organization change their way of doing their work. It is therefore important to first identify what must be changed in the individuals’ work environment in order to create the kind of behavioral change that is needed. The model is meant to help in the search for important points of possible change.

The model consists of work setting, vision and environment. Work setting can be further divided into four main categories:

• Organizing arrangements: The formal elements of the organization, coordination of people’s behavior and the interaction of the various parts of the organization. If these are consistent, they function to support the same type of behavior.

• Social factors: The humans in the organizations, how they interact, and what groups they join. This is often called the informal organization.

• Technology: Technology has a direct influence on the individual’s behavior. The more dominant the technology, the greater the influence. In this category we find things like tools, work flow design, technical systems etc.

• Physical setting: The physical environment in itself does not have much influence; however, it can support or hinder work. It includes space configuration, indoor climate, etc.

These factors affect each other; changes in one category might lead to the need for change in another. Changes in work flow design might create a need to change the physical environment or the technical equipment as well.

26

Figure 7: Porras and Robertson: Factors constituting the organizational work setting (Dunette & Hough (1992), p. 729).

In order to create a successful organizational change, it is important to shape these factors in a way so that they all work to influence the individuals in the same, consistent, desired direction.

3.6.3 Rollenhagen: HTO from a Safety Perspective As mentioned in the introduction Rollenhagen (1997) considers the interaction between the three HTO subsystems (Human, Technology and Organization) to be of utmost importance for safety. When an accident is investigated these three systems should, according to Rollenhagen, be analyzed with regards to behaviors (character analysis), lack of barriers (barrier analysis) and situational factors or deviations (deviation analysis). This is illustrated in Figure 8.

27

Figure 8: Structural model (Rollenhagen 1997, p. 17).

Character analysis – this analysis includes a study of the general principles that can be applied in order to understand the situation of interest. For humans it is studied how we process information, how the memory works and so on. Technology characteristics are for instance stress and fatigue of materials. Reliability and behavioral tendencies for organizations are for instance typical characteristics for a bureaucratic or a dynamic organization. When a character analysis is performed for a particular situation the specific conditions that were present at the time of the situation are studied.

Barrier analysis – if barrier functions are not working properly or are completely absent it can prevent failures to get caught in either the human, administrative or the technological protective nets. The barrier analysis studies these barriers or the lack of the barriers. Barriers for humans can be knowledge and training. Technological barriers are for instance physical protections, alarms, warnings and safety systems. The organizational barriers are instructions and rules as well as division of responsibilities.

Deviation analysis – if deviations from a planned performance are made it can increase the risk for behavioral tendencies to develop in the wrong direction. These deviations can be explained for human, technological and organizational aspects separately. For humans it can be analyzed how we function during stress or fatigue. For organizations it can be studied how safety is affected for organizational changes and for technology we can study how the technology functions during specific situations.

Rollenhagen (2003) sees accident investigations as an element in a larger setting of safety strategies that collectively can be labeled experience or lessons learned. He considers that a high level of complexity makes it difficult to predict all hazards when designing a system – so we also need to learn from our experiences from incidents and accidents (2003, p.27).

When performing an accident investigation from an HTO perspective Rollenhagen and Kahlbom (2001) recommends the use of an extended framework, as illustrated in Figure 9.

Deviation analysis

Barrier analysisCharacter analysis

Situational factors/ deviations

-human -technology -organization

Lack of barriers


Behavioural tendencies


28

Figure 9: A suggested HTO framework model (Rollenhagen & Kahlbom, 2001, p. 3).

Compared to the earlier HTO model presented by Rollenhagen, this framework puts an emphasis on the importance of information, which is shown in Figure 9. The organization is considered to lay the background to the other aspects; human, technology and information.

3.7 Accident Models There are many types of accident models that can be used when doing an accident or incident investigation. The choice of accident model affects the view of how the accident occurred. Depending of the frame of reference the accident model gives us different tools to interpret the chain of events leading up to the accident – as well as the accident itself. This is especially true when it comes to our view of the humans’ role in the accident. Hollnagel (2002) divides accident models in three main categories, based on three underlying base metaphors:

• Sequential accident models: The accident is described as the end point of a series of events, affecting each other in a cause‐and‐effect way. The model is also called the domino theory. While being easy to represent in a graphical way, it can be less useful in complex dynamic systems.

• Epidemiological accident models: Epidemiological theories are, as the name suggests, based on an underlying metaphor of disease. Accidents are being described as caused by the result of several factors, some clearly visible, other latent in the system. Epidemiological models give the opportunity to analyze complex accidents.

• Systemic accident models: The systemic approach takes a look at the whole system, viewing the accident as an emergent phenomenon of the system. This also means that the accident is seen as a normal part of the system, not a fault in itself, but rather expected.

The models we have chosen to take a closer look at fall into these three categories; with Sklet’s HTO model being sequential, the HFACS model epidemiological, and Leveson’s STAMP an example of the systemic accident models.

29

As we mentioned in section 3.3 the view of human actions plays a role in the analysis of accidents. A sequential accident model presupposes actions to be either right or wrong, but the epidemiological and systemic accident models allow for a more diversified view of human actions, looking to the outcome of actions rather than seeing it in black or white.

Hollnagel (2002) points out that this categorizing does not imply that any of the model types is better that the others, but rather that they might have different areas where they would be better to use. Simple as the sequential models might look, they can be very useful, and the complexity of the systemic models can both be a good and a bad thing.

In this chapter we will introduce the three accident models we have singled out, and in section 3.8 we compare and analyze the models.

3.7.1 Sklet: HTOanalysis One model that can be used for accident or incident investigation is introduced by Sklet in Methods for accident investigation (Sklet, 2002, p. 50). It is called the HTO analysis (sw. MTO‐analys) and is according to Sklet based on three methods:

• Structured analysis by use of an event‐ and cause‐diagram • Change analysis by describing how events have deviated from earlier events or

common practice • Barrier analysis by identifying technological and administrative barriers in which

have failed or are missing.

This model is, according to Sklet, mainly used by the Norwegian offshore industry and not comprehensively described. It is, however, a good example of a sequential analysis model. Sklet shows a HTO worksheet (see Figure 10) and lays out the appropriate steps to take when performing the HTO analysis. These steps are:

1. Develop the event sequence longitudinally and illustrate the event sequence in a block diagram.

2. Identify possible technical and human causes of each event and draw these vertically to each event in the diagram.

3. Analyze which technical, human or organizational barriers that have failed or was missing during the accident progress.

4. Illustrate all missing or failed barriers below the events in the diagram. 5. Assess the deviations or changes in which the accident progress differs from the

normal situation. These changes shall also be illustrated in the diagram. 6. Identify and present recommendations.

30

Figure 10: HTO analysis worksheet (Sklet, 2002, p. 52).

3.7.2 Leveson: STAMP As mentioned in section 1.1, Leveson (2002) aims to develop a new accident model, better suited to accommodate modern systems. The accident model that Leveson introduces is called STAMP – Systems Theory Accident Modeling and Processes. The main idea of STAMP is that accidents occur when external disturbances, component failures and/or dysfunctional interactions among system components are not adequately controlled, i.e. accidents result from inadequate control or enforcement of safety‐related constraints on the development, design, and operation of the system.

STAMP consists of three primary components:

1. Constraints: the constraints control the hazards so that they don’t develop into an incident or a mishap. Safety constraints can be relevant for system development, including both the development process itself and the resulting system design, as well as system operation.

31

2. Hierarchical control structures: socio‐technical systems can be modeled as a hierarchy of levels of organization with control processes operating at the interfaces between levels to control processes at the lower levels. The hierarchical control structures are different depending on the system you are analyzing. A general model of socio‐technical control structures for system development and system operation is shown in Figure 11.

3. Process models: a process model can be embedded in either a human controller or an automated one. The process model is used to determine what control actions are needed and it is updated through various forms of feedback. When the mental models of the designer, the operator and of the actual system (see Figure 12) does not match, the risk for an accident becomes much larger.

Figure 11: Leveson: General model of socio‐technical control structure for both system development (on the left) and system operation (on the right) (Leveson, 2002, p. 62).

32

Figure 12: The relationship between mental models (Leveson, 2002, p. 33).

Another concept that Leveson mentions, besides the three that were just described, as very important for accident investigation is adaptation. She says that any accident model that includes the social system and humans must account for adaptation (Leveson, 2002, p. 39). For an accident model to handle system adaptation over time, it must consider the processes involved in accidents and not simply events and conditions. She also claims that STAMP does just that.

Accident analysis based on STAMP generally includes the identification of the following:

1. Safety Requirements and Constraints 2. Controls 3. Context:

a. Roles and responsibilities b. Environmental and behavior shaping factors

4. Flaws in the controlled process 5. Dysfunctional interactions, failures and flawed decisions and erroneous control

actions 6. Reasons for flawed control actions and dysfunctional interactions

a. Control algorithm flaws b. Incorrect process, interface or mental models c. Inadequate coordination among multiple controllers d. Reference channel flaws e. Feedback flaws

So in other words, when doing an accident investigation according to STAMP, you first need to identify the hazard involved in the loss. Next, the hierarchical safety control structure related to the hazard shall be constructed and the constraints needed to control the hazard are identified for each level.

Then, starting from the technical process and using the proximate events and general knowledge about the hazard, any failures and dysfunctional interactions (including

33

communication problems) involved in the loss are identified. For each constraint, a determination is made about why it was violated. Either the constraint was never identified and enforced or the enforcement was inadequate.

Any human decisions need to be understood in terms of (at least):

• the information available to the decision maker as well as any required information that was not available,

• the behavior‐shaping mechanisms (the context and pressures on the decision making process),

• the value structures underlying the decision, and • any flaws in the mental models of those making the decisions.

3.7.3 Shappell and Wiegmann: HFACS Human Factors Analysis and Classification System, HFACS, is an accident investigation and analysis tool, used by several sectors in the U.S., including the Navy and the Air Force. The model builds on Reasons model of active and latent failures (the “Swiss cheese” model, Reason, 1990), specifying four levels of potential failure.

Figure 13: The “Swiss cheese” model of human error. From Shappell and Wiegmann (2000), and from Reason (1990).

Shappell and Wiegmann (2000) define the different levels of human errors along the same lines as Reason (1990) does. This tiered list becomes the basis for analysis in the HFACS system. The US Department of Defense (DoD) writes in its Human Factors Guide (DoD HFACS, n.d., p 1.) that “no investigator, flight surgeon, physiologist, Human Factors consultant or aviation psychologist can be expected to be fully familiar with all potential human factors”, but with this listing of every conceivable type of human error investigators have a check list as support.

34

This is the outline of the list according to Shappell and Wiegmann (2000):

1. Unsafe Acts: The action that triggers the accident and leads to an active failure. Unsafe acts can be further divided:

a. Errors: when the operator fail to do the planned act. Errors are unintentional. i. Skilled‐based errors ii. Decision errors iii. Perceptual errors

b. Violations: deliberate disregard for rules or instructions.

i. Routine ii. Exceptional

2. Preconditions for Unsafe Acts5: actions that leads to latent failures, which can lead

to unsafe acts becoming active failures.

a. Substandard conditions of operators i. Adverse mental states ii. Adverse physiological states iii. Physical /mental limitation

b. Substandard practice of operators

i. Crew resource management ii. Personal readiness

3. Unsafe Supervision: substandard leadership can lead to latent failures.

a. Inadequate supervision b. Planned inappropriate operations c. Failed to correct a known problem d. Supervisory violations

4. Organizational influences: bad organizational management affects all levels of the

organization; from supervisors to operators. Mistakes here also lead to latent failures.

a. Resource /acquisition management b. Organizational climate c. Organizational process

Each part of the list is also broken down to detailed examples which are given nanocodes, for easier classification and sorting.

5 In the DoD‐report (DoD HFACS, n.d.) this part is complemented with “environmental factors”, comprising of physical and technological environment. This is the only practical difference between the two representations of the HFACS model.

35

A similar report from the DoD (DoD HFACS, n.d., attachment 1, p. 1.) also contains a short user’s guide to investigating accidents:

1. Start with the event outcome 2. Create a time line backwards 3. For each point on the timeline determine if it is a material failure or a human error 4. For each point determined to be a human error:

a. document who committed the act b. use the taxonomy to classify the act

5. Evaluate the preconditions for the unsafe acts: a. check the categories and sub‐categories for issues contributing to the act b. check the supervisory and organizational issues contributing to the act

6. Write a short narrative for each act

Shappell and Wiegmann (2000) states that the model deals with all aspects of human error, and thus provides a comprehensive check‐list for accident investigation. The model has been used by several U.S. military and civilian organizations, such as the U.S. Navy, U.S. Air Force and the National Transportation Safety Board.

This model builds on Reason’s (1990) theories about human error, but it also corresponds to the theories of Hollnagel (2002), described in section 3.3, describing human action as neither wrong nor right, until after the fact. The HFACS model helps describe what has happened, in order to –after the fact – find out what the cause was.

3.8 Comparative Analysis of Models Hendrick and Benner (1987) present ten criteria for accident model evaluation. In this section we first discuss those criteria, condense them into five new criteria, apply them to our models and finally we aim to sort out the models best suited for our goals.

The full description of Hendrick’s and Benner’s list of criteria can be found in Appendix 1. Here follows a short run through with emphasis on those criteria we have found to be the most relevant in this case.

36

Table 2: Criteria for accident model evaluation, after Hendrick and Benner (1987), with our new add‐ons.

Original categories

Description New categorization

1 Realistic The model must represent reality, that is the model must not be too static, and has to take changes in time into account. The model should also represent risk‐taking in the work process.

How well does the model describe the world?

2 Definitive

The model must drive the investigation, not the other way around; model must use clear and definite data and values to describe the world.

How well does the model describe the world?

3 Satisfying

The model must provide a credible analysis of the organization and its actions.

How well suited is the model for the investigating agency?

4 Comprehensive The model must cover the whole course of events leading up to the incident, and help avoid ambiguity.

Does the model give an exhaustive description of the event?

5 Disciplining

The model must provide a technically sound framework for investigation, guiding the analysis.

Does the model guide the investigating work in a good way?

6 Consistent

The model must be theoretically consistent and fit well to the agency’s safety programs.


7 Direct

The model must provide direct identification of safety problem, enabling correction of said problems.


8 Functional

The model must be functional, linking the accident to the work process.


9 Non‐causal

The model must be free of causal factors, providing a full description of the events.

Does the model give an exhaustive description of the event?

10 Visible

The model must describe the events visibly, easily comprehendible and credibly, to the public, victims and investigators.

Does the model provide a visible and relevant interpretation of the incident?

As we can see, the criteria fall into five categories: how well they describe the world, how well they work with the investigating agency’s work, if they are extensive enough, does the model guide the investigation in the right direction, and last: is the model perceived as visible and relevant enough?

37

This means that we can analyze the models according to the following principles:

Table 3: Our criteria for accident model evaluation.

Category Description Descriptive How well does the model describe the world? Exhaustive Does the model give an exhaustive description of the event?Appropriate How well suited is the model for the investigating agency?Guiding Does the model guide the investigating work in a good way?Visible Does the model provide a visible and relevant interpretation of the incident?

The models will be analyzed according to these five criteria, and given points due to how well they live up to those. The value “2” means high agreement, “1” some agreement, and “0” means little or no agreement. The results will be visualized using polar diagrams. When unsure of the value we have used the higher one in the chart below.

Table 4: Rating of models according to the new evaluation criteria.

Descriptive Exhaustive Appropriate Guiding Visible Eklund 2 1 1 0 0Porras and Robertson

2 1 0 0 1

Rollenhagen 2 2 2 1 0Sklet 0 0 1 2 2Leveson 2 2 1/2 2 0Shappell and Wiegmann

2 2 1 2 �

Eklund

Eklund’s model puts human activity, operations and processes in the foreground, taking the organization, technology and context into account, and connects this to the overall activity via goals. This is not a model suited for accident investigation. A possible use would be for certain types of accidents, where human activity seems to be the primary cause of the accident. The model would however not work as the only one used by an accident investigation agency. The model is descriptive in its analysis of the world(2), somewhat exhaustive (1) when describing an event, but not very appropriate (1) for accident investigation, and it is neither guiding (0) the user nor producing a visible (0) representation of the event.

Figure 14: Eklund.

0

2D

E

AG

V

Eklund

38

Porras and Robertson

Porras and Robertson’s model encompasses large parts of the HTO system, and reaches a high degree of detail in its description (2). The model is exhaustive, but primary for an organizational change perspective, not only yielding it (1) in exhaustive, but (0) in appropriateness. The model does not give much in way of guidance (0) to the analyst, but it can produce quite a visual output, although it might not be very suitable for accident investigation (1). The model’s strength could be useful in combination with e. g. Leveson’s STAMP or Shappell and Wiegmann’s HFACS.

Figure 15: Porras and Robertson.

Rollenhagen

Rollenhagen’s model is made specifically for accident investigation and safety work, so it is indeed appropriate (2). The model encompasses all parts of the HTO spectrum, thus being exhaustive (2) and describes the world in a thorough manner (2). The model does not provide much guidance (1), and it is not visual (0).

Figure 16: Rollenhagen.

Sklet

This sequential accident model does not have a very good theoretical base, and it does not describe the world or the event especially thorough (descriptive: 0, exhaustive: 0). The model, although being a model for accident investigation, is mainly formed for the off‐shore industry, rendering it only semi‐appropriate (1) for aviation applications. It is visually good (2), but it can produce too much output, and it guides (2) the investigator well throughout the process.

0

2D

E

AG

V

Porras & Robertson

0

2D

E

AG

V

Rollenhagen

39

Figure 17: Sklet.

Leveson

Leveson’s model describes the world with a systemic approach (2), and does so in an exhaustive manner (2). The model is aimed at accident investigation (among other things) but might be a bit difficult to use due to its high demand on the analyst (appropriate: 2/ 1). The model guides (2) the user well, but it does not produce a visual result (0).

Figure 18: Leveson.

Shappell and Wiegmann

The epidemiological model HFACS is not completely appropriate (1) for accident investigation, since it is predominately a Human Factors model, not an HTO model. Being a Human Factors model, it is however both descriptive (2) and exhaustive (2), and it gives good guidance to the user (2). It does not produce a visual output (0).

Figure 19: Shappell and Wiegmann.

0

2D

E

AG

V

Sklet

0

2D

E

AG

V

Leveson

0

2D

E

AG

V

Shappell & Wiegmann

40

3.8.1 Conclusions If entirely relating to accident investigations, Eklund’s model might be useful for finding system processes responsible for human error, or to get a view of the whole work system. We do not believe, however, that the model is a very good tool for investigating more technical kinds of accidents.

Porras and Robertson’s model is mostly a tool for organizational change, to help identifying what can be changed within the organization, thereby guiding the change. If an incident has occurred during an organizational change, this model might be useful for certain investigations. It seems, however, in large quite limited for accident investigations.

As mentioned, Rollenhagen considers the interaction between the three HTO subsystems (Human, Technology and Organization) to be of utmost importance for safety. He says that when an accident is investigated these three systems should be analyzed with regards to behaviors (character analysis), lack of barriers (barrier analysis) and situational factors or deviations (deviation analysis). This model has been made with accident investigations in mind from an HTO perspective, so we consider it to be a broad and useful model for incident investigations.

Sklet’s model is a thorough and detailed accident investigation model that most likely will yield results for preventing further similar accidents. The disadvantage of this model seems to be that the worksheet becomes very big and if the accident is in a complex environment and involves several people it may be difficult to manage, to keep everything in the same worksheet. The model also puts demands on the person performing the analysis with regards to knowledge about the systems, organizations and processes involved.

The model that Leveson presents does not constitute revolutionary new work methods or ideas, but it is a different approach with the intent of capturing all causes to accidents and/or incidents. The difference from more traditional accident investigation models that are event‐based models is that STAMP allows for a broader way of thinking and analyzing. At the same time, it may be more complex and time‐consuming as well as more demanding of the skills and perseverance of the person that performs the investigation.

The HFACS model from Shappell and Wiegmann is according to the originators a model that deals with all aspects of human error, and thus provides a comprehensive check‐list for accident investigation. We agree that it is a useful tool for human error identification and is a good complement to other methods and models for incident investigations.

After analyzing the six models with the help of Hendrick and Benner’s criteria, our conclusion is that the models that seem most appropriate for accident investigation are:

• Rollenhagen’s HTO model, • Leveson’s STAMP and • HFACS, in conjunction with other models.

These models all have high agreement on descriptiveness, exhaustiveness and high, or fairly high, on appropriateness. The results from the comparative analysis with the help of Hendrick and Benner’s criteria support the opinions we reached with our own theoretical analyses of the models.

41

In Chapter 5 we take a closer look at these three selected models, applying them in a case study in order for us to form an opinion on how they work in real use. But first we describe the incident reporting system for military aviation in Sweden in Chapter 0.

43

4 The Reporting System for Military Aviation in Sweden

This chapter describes the incident reporting and investigation system in the Swedish Air Force and at the aircraft manufacturer Saab Aerosystems. The information in this chapter is based on internal documents supplied by Saab Aerosystems, if not otherwise stated.

4.1 Introduction The Swedish Armed Forces and the Military Flight Safety Inspectorate (sw. Flyginspektionen (FLYGI)) have issued rules and regulations regarding the operation of military aerial vehicles. These rules and regulations are called RML – Rules of Military Aviation ‐ and among other things they put requirements on the reporting system within the Swedish Air Force. The structure of RML is shown in Figure 20.

Figure 20: Overview of RML structure.

In the part RML‐V‐2B the following requirements regarding accident prevention programs and reporting are stated:

2.25.9 An air operator shall establish an accident prevention and flight safety program, which may be integrated with the quality system, including:

2.25.9.1 Programs to achieve and maintain risk awareness by all persons involved in operations, and

44

2.25.9.2 An occurrence reporting scheme, which is linked into the external reporting system according to V.2.B.28.2, to enable the collection and assessment of occurrence‐, incident‐, and accident reports in order to identify adverse trends or to address deficiencies in the interest of flight safety;

2.25.9.3 Evaluation of relevant information relating to occurrences, accidents and incidents and the promulgation of related information.

(RML‐V‐2B)

4.2 The Reporting System in the Air Force In order to comply with the requirements stated by RML, the Swedish Air Force has created a reporting system. Different types of occurrences and incidents require different forms and reports to be filled out and these reports are sent to appropriate instances for further accident preventive actions.

The figure below shows an overview of the different types of reports that are used.

Figure 21: Overview of the reporting structure in the Swedish Air Force.

The reports and forms, including when they are to be used and how they should be filled out, are described in the RAFT – “Rapporteringsanvisningar Flygmaterialtjänst”(Försvarets Materielverk, 2002) which is issued by FMV.

To summarize in short, the Alert Reports (sw. “Direktanmälan”) are used when the incidents is judged to be of high importance to flight and personnel safety and the process needs to be speedy. There is a template for filling out an Alert Report. However, the RAFT states that this template shall only be regarded as guidance for which type of information that normally

Incident/deviation

Alert ReportOperationalDisturbance

Report (ODR)

Materiel FailureReport (MR)

ODR BASBase &

Air Traffic Control

ODR FLYGFlight

TR/WO

ODR STRILCommand& Control

ODR SISLiaison & System

Information Control

ODR VÄDWeather

Technical Report/ Work Order

For materielfailureFor serious incidents

45

should be included in an Alert Report. It may need to be complemented or reduced, all depending on the type of occurrence.

Operational Disturbance Reports (ODRs) (sw. “Driftstörningsanmälan, DA”) are written as soon as there has been a deviation from planned activities. An Operational Disturbance is defined as any occurrence or failure of either personnel or equipment that have caused, or could have caused, a reduced serviceability of crew or aircraft, or otherwise have caused a deviation from ordered or planned flight operations or operating procedures. Some incidents can cause both an Alert Report and an ODR to be written. The reporting of occurrences is regulated by RML, as described in the introduction of this chapter. The requirements can be found in RML‐V‐2B and are:

Occurrence Reporting (DA)

V.2.B.28.2 An Air Operator or an pilot‐in‐command shall submit a report to FLYGI and when applicable also the holder of the Materiel System Clearance for the aeronautical product concerned, of any operational incident. The reporting shall be made according to RML‐V1‐D concerning occurrence reporting (DA).

(RML‐V‐2B)

Material Reports (MRs) (sw. “Materielfelsrapport, MR”) are written if some material is an essential part of or cause to the incident or disturbance. So if the incident is highly safety critical and caused by material failure, an Alert Report and an ODR and a MR are written.

A TR/WO (sw. “Teknisk Rapport/Arbetsbeställning, TRAB”) is written when a failure is suspected or a maintenance measure is needed. This ensures that all material changes in a system are kept track of. A TR/WO consists of two parts, a technical report and a work order. The data in the TR/WO are used for updating different information systems which are subsequently used for analyzing:

• failures, failure modes and actions taken to rectify these • ordered maintenance tasks • accumulated flight hours and other operational parameters of aircraft and serialized

units • future maintenance requirements

An incident can therefore result in different levels of reporting; either all four kinds of reports, a combination of them or just one of the reports.

The RAFT also makes clear of where each type of report is supposed to be sent. This is summarized in the following figure along with a short description of each report.

46

Figure 22: Overview of incident reporting for the Swedish Air Force, including distribution of reports.

4.3 Incident Reporting and Analysis at Saab Aerosystems As shown above some of the reports that are carried out in the Swedish Air Force are sent to the manufacturer. For many military aerial vehicles the manufacturer is Saab Aerosystems. Incidents can also occur at the test facilities at Saab Aerosystems or unsafe conditions affecting flight or human safety can be discovered by employees at Saab Aerosystems. Events or incidents which can affect flight safety, airworthiness, personal safety and/or system reliability for military aircraft, must according to regulating documents at Saab Aerosystems be reported and analyzed. Reports shall be made as soon as is practically possible, but at the latest 72 hours after the event was identified.

Two terms that are used for describing a present state or an occurred incident are:

Unsafe condition

Event which can affect flight safety, airworthiness, personal safety and/or system reliability, or the company´s ability to fulfill requirements from authorization or Type Certificates.

Flight safety

Operations involving military aviation materiel without faults/malfunctions or damage to personnel or materiel during flight, aircraft maintenance, development, design and production.

(Source: Saab Aerosystems; internal document)

Incident/Occurrence

TR/WO – used for reporting known or suspected failure or damage, installation or removal of serialized units, ordering maintenance tasks according to the Material Plan, modifications etc.

MR – used for reporting materiel failures or problems that have affected, or might affect, the airworthiness, serviceability or safety of aeronautical product or the safety of personnel in general.

ODR – are used for operational disturbances or to suggest changes in operational procedures to improve flight safety

Alert Report - reporting occurrences that are assessed to affect flight safety and that are of such importance that they should be known to the authorities as soon as possible

HKV GRO FV Flygsäk

DIDAS or other information systems

sent to

workshop, for order of maintenance or repair

DIDAS FLYG (for reg.)

FMV, Hkv FlygI, affected Tech. office & units

Affected manufacturers & workshops

DIDAS FLYG (for reg.)

sent via fax or reported by phone

Hkv FlygI & affected operator

Affected manufacturer (MSI)

47

So, if an unsafe condition or a flight or personnel safety issue is suspected or identified, it must be reported and analyzed so that proper measures can be taken. These incidents are reported with the same types of reports that are used within the Swedish Armed Forces, as described in section 4.2. In addition to those, a PLUA (sw. Potientiell Luftvärdighets‐anmärkning, translated “potential airworthiness remark”) is written for incidents on test aircraft that may also have an impact on aircraft in normal operation.

The following figure shows an overview of the flow of information between Saab Aerosystems as a manufacturer of aerial vehicles and the Swedish Air Force and aviation authorities regarding safety issues.

Figure 23: Overview of incident reporting for the Swedish Armed Forces, FMV and Saab Aerosystems (Source: Saab Aerosystems; internal document).

As can be seen from Figure 23 the information from the operators comes into the Saab Aerosystems’ organization mainly through two channels. One is through the Daily Product Meeting, which includes all field occurrences regardless of impact on safety and airworthiness, and the other is through the “Urgent” phone which is dedicated for safety and airworthiness issues. This is further described in the following sections as well as how the matters are taken care of in order to prevent further incidents or accidents.

4.3.1 Daily Product Meeting (DPM) The Daily Product Meeting (DPM) is responsible for the initial assessment of all matters. If one matter is considered to affect flight safety or the product’s airworthiness it is raised to the Airworthiness Board (AWB).

Matters that in the smallest way might influence personal safety, flight safety or air worthiness of the products within the responsibilities of RML (Rules for Military Aviation) shall by DPM be raised to the chief engineer as a matter of airworthiness. It is not up to DPM to make any judgments of how critical or probable the matter is. The chief engineer decides whether the matter shall be handled by AWB. DPM shall be prepared to present the matter to AWB.

Product Safety BoardProduct Safety Board

Air Worthiness Board

Air Worthiness Board

Urgent reporting about airworthiness and flight and person safety related occurrences.

Reporting of all field occurrences included airworthinessand flight and person safety related.

Development & Production(Vendors)

Development & Production(Vendors)

SwedishAviation

Maintenance

SwedishAviation

Maintenance

Authority(FLYGI)

Authority(FLYGI)

SwedishOperators

SwedishOperators

Swedish Boardof Accident

Investigation

Swedish Boardof Accident

Investigation

Aircraft Operators(Export)

Aircraft Operators(Export)

Air Worthiness -handling

Daily Product Meeting

Daily Product Meeting

ChiefEng.

Urgent013-

185525

FMV

Customer

Vendor MSI-Owner

CentralMT

48

Military Flight Safety Directives, Flight Safety Information and Alert Reports that come to the Customer and Product Support department at Saab from the authority or Swedish Air Force units shall be immediately distributed by fax to the affected department or post holder within Saab Aerosystems.

4.3.2 Airworthiness Board (AWB) The Airworthiness Board (AWB) handles occurrences on aircraft that operates with Military Type Certificate (MTC) with Saab Aerosystems as MTC‐holder and that can have an effect on continued airworthiness, flight safety and/or system reliability for the aircraft type. AWB is also a forum for Head of Design6, also chairman, to carry out and follow up his/hers decisions on the handling of occurring events as above.

In order to guarantee, as far as possible, that logical and full investigations are carried out, AWB has a composition which meets the incident’s degree of difficulty. If needed, specialists can be called in.

Air Worthiness Board is interface to aviation authorities and customers/users of military aircraft manufactured by Saab when reporting unsafe conditions. A close and informal exchange of information should be carried out between the authorities and the manufacturer in order to be able to handle the matter flexibly between concerned parties.

4.3.3 Product Safety Board (PSB) The Product Safety Board (PSB) at Saab Aerosystems is responsible for:

• management of company activities regarding occurred/feared accidents or near‐accidents for all aircraft and products where Saab Aerosystems has the design responsibility, has manufactured and/or is operating

• periodic review and analysis of the safety status for Saab Aerosystems aeronautical products

• executing product safety issues which have significant consequences for the company.

The chairman of PSB is the CEO of Saab Aerosystems and Head of Design, who is also the chairman of AWB, is one of the members of PSB.

4.3.4 Fault Hazard Analysis (FHA) When incidents or accidents occur, a risk analysis called a Fault Hazard Analysis (FHA) shall be performed at Saab Aerosystems.

The outcome of the Fault Hazard Analysis is a Hazard Risk Index (HRI), which is used to value the incidents from a flight safety and airworthiness point of view. It is the basis for further decisions as specified in Table 7. The HRI method is applicable to quantifiable hazards only.

Hazard Risk Index is calculated as the product of severity index and probability level. The severity index is based on assessments made regarding the outcome of the incident (Table 5). Probability for loss of aircraft or death of crew, personnel or third party may be included in the assessment in order to reach a severity index. The probability limits are described in Table 6.

6 The Head of Design (HoD) at Saab Aerosystems has the responsibility to make sure that all activities are carried out in accordance with the Rules of Military Aviation (RML).

49

Table 5: Severity categories for occurred incidents (Source: Saab Aerosystems; internal document).

Category Definition Environmental damage

CATASTROPHIC Severity index 4

Death or loss of A/C Major

CRITICAL Severity index 3

Severe injury or major materiel damage or failure demanding immediate action to avoid severe injury or major materiel damage.

Significant

MARGINAL Severity index 2

The failure or malfunction can generally be controlled but minor injury or materiel damage cannot be excluded.

Minor

NEGLIGIBLE Severity index 1

No injury or materiel damage. No

The probability of occurrence for the incident shall be calculated. Hazard quantification can sometimes be difficult if little data is available.

Table 6: Probability levels for occurred incidents (Source: Saab Aerosystems; internal document).

PROBABILITY LEVEL

ITEM INVENTORY PROBABILITY OF OCCURRENCE

6 Frequent Likely to occur frequently Continuously experienced *

5 Probable Will occur several times during life of item

Will occur frequently *

4 Occasional Likely to occur during life of item

Will occur several times during use

*

3 Remote Unlikely but possible to occur in the life of an item

Unlikely but can reasonably be expected to occur

*

2 Improbable So unlikely it can be assumed that the occurrence may not be experienced

Unlikely to occur but possible

*

* The actual figures for probability of occurrence can vary depending on the application.

When the severity categories and the probability levels for the incident have been determined the HRI can be achieved with the help of Table 7. The table also includes acceptance criteria and guidance for further handling of the hazard.

50

Table 7: Hazard Risk Index (HRI) for occurred incidents (Source: Saab Aerosystems; internal document).

Hazard Rate Catastrophic Critical Marginal Negligible

Probability of Risk Index

Level

occurrence 4 3 2 1

Frequent * 6 24 18 12 6 Probable * 5 20 15 10 5 Occasional * 4 16 12 8 4 Remote * 3 12 9 6 3

Improbable * 2 8 6 4 2

16‐24 Unacceptable Change Necessary/Mandatory correction

12‐15 Undesirable Reduce Failure Rate/Attempt to eliminate

9‐11 Acceptable with review In Depth Review

1‐8 Acceptable Acceptable

* The actual figures for probability of occurrence can vary depending on the application.

The Fault Hazard Analysis shall be documented in a report and the central System Safety department as well as the system Manager shall approve and agree with the report.

51

5 Case Study

In this chapter a case is described and analyzed with the help of the three selected models that have been described earlier. The purpose of the case study is to evaluate these methods. The selection of this specific case is discussed and explained in section 2.3.

5.1 SK60 Canopy Burst This section starts with a short description of what happened during the incident. The description is based on investigation reports supplied by Saab Aerosystems. The investigation that followed is then analyzed with the help of the three methods (Rollenhagen, Leveson and Shappell & Wiegmann) that were deemed most appropriate by the analysis in Chapter 0.

5.1.1 What Happened A SK60, which is a jet aircraft trainer built by Saab Aerosystems and used in the Swedish Air Force (and others), was being transported from Uppsala to Ronneby in February 2001 when the canopy “exploded”. The altitude at the time of the incident was 8550 m and the aircraft contained three persons. The altitude was swiftly reduced to about 1400 m and the speed to 250 km/h. The pilot describes violent vibrations in the aircraft after the incident, but he was able to fly the aircraft and land at the nearest airport (Kungsängen). The turbulence from the wind in the cockpit made it difficult for the pilot to read the instruments or to hear anything.

The pilot wrote an Alert Report, which was sent to the Headquarters (FLYGI) and the Swedish Defense Materiel Administration (FMV) as well as Saab Aerosystems (the manufacturer). An extensive investigation was started, which was led by the Air Force wing at Ronneby (F17). At Saab the Air Worthiness Board (AWB) led the work from the manufacturer’s point of view.

The investigation showed that the canopy ruptured due to errors done when the canopy was assembled. Instead of doing heat treatment on the canopy before mounting it on its metal frame, the heat treatment was done afterwards. Heat treatment is usually done before mounting, to lessen the tensions in the material, doing it after assembly probably made more harm than not doing heat treatment at all. No test of the tensions in the canopy was performed before taking the aircraft in operation.

5.2 Rollenhagen Rollenhagen states in Att utreda olycksfall (2003) (Translated:”To investigate accidents”, p. 215) that the investigation should start with a general and easy‐to‐read description of what happened. This description should show the events graphically in a chronological order.

For this incident the following could be shown:

Cracks incanopy

Flight at 8550m altitude

Canopyexplosion

Cracks notdiscovered

Tension incanopy (stress)

Figure 24: Illustration of what happened during the SK60 incident in accordance with Rollenhagen description.

52

Rollenhagen then states that it should be described why the incident occurred. This may have to be done in several steps.

The reasoning concerning why the incident occurred can be illustrated by adding on to the picture that showed what happened:

Cracks incanopy


Canopyexplosion


Canopy old age

Incorrect heat treatment

No or insufficientprocedures


Material failure

Outside temperature causing canopy glass to become brittle

Figure 25: Illustration of why the SK60 incident happened in accordance with Rollenhagen description.

Note that in our diagrams we have chosen to use different shapes for the possible contributing causes of the incident (ovals) and the events themselves (rectangles). Rollenhagen (2003, p.218) says that this can be done differently depending on personal preferences and accessible drawing support.

The next step according to Rollenhagen (2003) is to expand the diagram and perform the barrier analysis. The questions that must be answered are:

1. Which overarching systems and activities could have discovered and eliminated the weaknesses and negative events that have been identified?

2. Why the functions (above) did not work and/or why were they absent? 3. What barrier system could have blocked the unfolding of the events so that the

incident had been prevented? 4. Why the barriers did not work and/or why were they absent?

The overarching systems and activities are, for instance, the quality system, which regulates responsibilities and authorities, describes activities etc and the review process that shall supervise that the rules and regulations are followed. They can also consist of the system for learning by experience, system for how risk analyses are performed, Human Resource Management, overarching safety management, system for verification etcetera. The barriers are functions that could have stopped or eliminated one or more of the events that lead to the incident.

The performed work and barrier analysis can be illustrated as in Figure 26, where the barriers are shown in dashed boxes.

53

Cracks incanopy


Canopyexplosion


“Heating treatment”of the canopy

Scheduled check for cracks

Low level flights only


Change old canopies for new ones earlier

Keep track of canopy age/hours

Figure 26: Illustration of work and barrier analysis for the SK60 incident in accordance with Rollenhagen’s description.

During the investigation more information will be available and the diagram can be expanded to include plausible causes on a management and system level. This should be marked in the illustrations, see Figure 27.

Cracks incanopy


Canopyexplosion


“Heating treatment”of the canopy

Scheduled check for cracks

Low level flights only


System/management level - causes

Process deficiencies when changing canopies

Technical personnel not sufficiently trained for canopy check-ups and replacement

Figure 27: Illustration of work and barrier analysis for the SK60 incident including management and system level in accordance with Rollenhagen’s description.

When the causes have been fully understood, suitable measures to ensure that this type of incident or accident does not happen again can be recommended. Also, the report shall be written and clearly describe the findings from the investigation and the analysis.

Rollenhagen (2003) also states the importance of spreading the information and conclusions to affected parties as well as doing follow‐ups to ensure that the recommendations are being taken into consideration.

54

Figure 28: Rollenhagen’s description (1997, p. 17).

When doing the investigation including the interviews and the following analysis, Rollenhagen’s model as shown in the Venn diagram in Figure 28, should be kept in mind. If we apply this model to the SK60 canopy case it may be shown like in Figure 29.

Figure 29: Rollenhagen’s Venn diagram applied to the SK60 case.

5.3 Leveson The first thing to do when doing an accident investigation according to Leveson (2002) is to identify the hazard and system safety constraints involved in the loss.

In this case the hazard is that the canopy is broken into pieces (explodes) and that the crew and passengers (if applicable) are exposed to loss of sufficient oxygen supply and powerful winds. It is then difficult for the pilot to safely land the aircraft. Therefore the overall system safety constraint can be expressed as: the canopy must not be exposed to stress factors and conditions that will cause it to break.

Deviation analysis

Barrier analysisCharacter analysis

Situational factors/ deviations


Lack of barriers


Behavioural tendencies


55

The hierarchical safety control structure related to the hazard is constructed (at least an attempt made by the authors) and shown in Figure 30. It is assumed that canopy changing is a part of System Development and that System Operations include flight operations in the Air Force.

System Development System Operations

Air Force Wing

Pilot operations

RML

Passengers

Flight altitude allowed according to OSF/FM

Saab Management

Supplier* Management

* Supplier for the service of changing canopies

Canopy changing Process

Requirements, Documentation

Personnel work procedures

Sign off

OSF - Operationella Säkerhets Föreskrifter

FM - Flight Manual

-Safety policies

-Standards

-Work instructionsLuftfartsverket, LFV

Armed Forces’ Headquarters-Work instructions

Figure 30: Hierarchical safety control structure for SK60 case.

For each control level every constraint and the way it was violated shall be identified. For each constraint, a determination is made about why it was violated. Either the constraint was never identified and enforced or the enforcement was inadequate. Unsafe behaviors and dysfunctional interactions shall be identified and brought forward as well as flaws in the process and/or control algorithm models.

The identified control levels for this incident are:

• Saab management: requirements regarding canopy changing procedures, safety requirements, supplier on‐site reviews, control after canopy changed – roles and responsibilities

• Saab System Engineering: formulating requirements regarding canopy changing procedures, inadequate specifications

• Supplier management: requirements regarding canopy changing procedures, safety requirements, control after canopy changed – roles and responsibilities

• Supplier work force: following instructions from management, signing off incomplete work, flaws in process models, test performed incorrectly, heating procedure of canopy after it had been mounted in the steel frame instead of before

• Air Force: acceptance of the aircraft, check

56

The control levels and some of the interaction between the different parties are summarized and shown in the following picture. The analysis in the picture is not complete, but is deemed sufficient to illustrate the work principles.

System Development for canopy changing

Saab Management Supplier* Management

* Supplier for the service of changing canopies

-Safety policies

-Standards-Work instructions

Requirements

System Engineering-correct instructions for changing canopies?

-requirements expressed clearly?

Communication Flaws

- between Saab and supplier management

- between supplier management and workers performing the canopy changing procedure

- between supplier and operator/customer

Documentation

Customer – Swedish Air Force

Supplier Work Force-correct work procedures?

-requirements expressed clearly from management?

-test reviews performed? Incomplete?

-sign-off when work incomplete?

Figure 31: Control structure for canopy changing in the SK60 case.

When each control level is analyzed in this way the causes and contributing factors can be identified and hopefully corrected to prevent further incidents.

5.4 Shappell and Wiegmann As described in section 5.1.1, the investigation of the SK60 incident found that the canopy had ruptured due to tensions in the glass in combination with long duration flight at high altitude. The glass tension was the result of doing heat‐treatment on the canopy after it had been mounted in the steel‐frame, instead of beforehand, as should be done.

According to the DoD‐instruction on how to use Shappell and Wiegmann’s HFACS method, the first thing to do is to start with the outcome and work backwards in time (DoD HFACS, attachment 1, p. 1.). For each step it shall be decided if a material failure has occurred or if an individual made an error. The DoD‐report does not give any guidance on how to represent the problem graphically, except for the need for a time line. The graphical part of the analysis could therefore be done in other ways than the way we have chosen.

57

Figure 32: First step in HFACS analysis, time line.

This gives us this time line, ending with the outcome, the canopy rupture, and working itself backwards to the initial cause of the event. Of course, in a real investigation this step would take a long time and require the investigator to interview a lot of people. The canopy rupture, and its precursor, the forming of cracks in the canopy, were deemed as material failure.

Next the investigator decides who committed each act and classifies them according to the HFACS classification system. The classification system has four main categories – unsafe acts, preconditions for unsafe acts, unsafe supervision and organizational influences –as seen in the vertical axis of Figure 33. We classified each of the acts as unsafe acts.

Figure 33: Continuation of HFACS analysis.

The following step entails further investigating in order to find preconditions for the unsafe acts. This gives us the diagram in Figure 34, illustrating the incident and its precursors. Knowing exactly which categories each act falls into is difficult, as is the selection of what should indeed be in the analysis at all. For instance, to determine why the canopy was put into place without heat‐treatment, it takes a lot of investigation, and all people in the organization might not be inclined to cooperate.

58

Figure 34: HFACS, classification of acts.

Figure 34 shows an example of how the HFACS analysis might be continued; further investigation would result in a more detailed analysis.

5.5 Conclusions We have conducted analyses of the selected SK60 case with the help of three models; Rollenhagen’s HTO model, Leveson’s STAMP and Shappell and Wiegmann’s HFACS model. We conclude that the methods gave somewhat different results and the pros and cons for each model are discussed in this chapter.

Rollenhagen’s model is, in our opinion, straight‐forward and visual. The steps that he recommends are logical and easy to follow, see Figure 24 –Figure 27. The different levels of the system that are involved in the incident or accident are considered by adding on more information with each step. The model can be used for both “smaller” and more complex situations, depending on the context of the incident. The number of steps and the size of each step can be adapted so that the analysis can be brought to an end when it is deemed by the analyst and interested parties that the significant causes have been found and understood.

One of the drawbacks with the model is that it will most likely give you what you put in; i.e. the result is dependent on the people participating in the analysis and if they are inexperienced several important factors and/or causes may be missed. Also, if the incident involves many instances and complex connections the visualizations may become too large to be handled practically.

When adding the information illustrated by the Venn diagram, see Figure 28 – Figure 29, and applying that to the descriptive steps, all included parts may cause the model to be a bit confusing. However, in our view it does seem that all the factors Rollenhagen describes need to be considered in order to conduct a complete analysis.

Leveson’s STAMP model is in our view a very thorough analysis method. Focus is on identifying the hazards and formulating requirements in order to avoid incidents and accidents to develop. One advantage with the model is that it systematically works through

59

the different control levels and their contribution to the incident. The model also incorporates human and organizational aspects in a way that seems easier than with traditional event‐based accident investigation methods. If done properly, the likelihood that future similar accidents or incidents will happen should be small.

One difficulty with STAMP is to identify the proper control levels and set the framework for the analysis. Also it seems like the analysis can become overly detailed and time‐consuming, therefore becoming unnecessarily expensive. For incident investigation that includes aspects that might be difficult to identify, STAMP may however be a very valuable tool for finding all contributing factors, especially if both technical, human and organizational causes as well as different modes of operation and time factors are present.

Shappell and Wiegmann’s HFACS model is, at a first glance, easy to use and it provides the analyst and other interested parties with a nice visualization along the time line. The human error check list that is included in the model is extensive and has been used for aircraft environments during a long time, so it seems likely that all contributing human causes for this context can be found. As a pronounced Human Factors model, it is known beforehand that focus lies on human error and that it is not a complete HTO model.

One difficulty we found with using the HFACS model is to properly label the underlying factors according to the check list, especially if you are an untrained analyst. Our opinion is that this model, with its extensive check list of possible human errors, would be best used as a compliment to a more technical investigation model or other HTO models.

To summarize the analysis of the case study it is evident that each model has its strengths and weaknesses and which one that is the most appropriate depends on the type of incident that is investigated. One common difficulty for the models is that they leave a lot up to the investigator to decide; for instance how to categorize actions in HFACS, how to assess the constraints in Leveson’s model etc. An experienced analyst is very important for the outcome of the analysis, regardless of the model that is used.

61

6 Interviews

This chapter first introduces the persons that were interviewed for this thesis and also briefly describes how the interviews were conducted. This is followed by an analysis of the outcome from the interviews, with focus on areas of interest with regards to the research objectives.

6.1 The Interviews In order to gain more knowledge about how incident reporting and analysis are dealt with in real life, as a complement to what was written in the documentation received and analyzed, three semi‐structured interviews were conducted. The interviews were all recorded and they were conducted in Swedish. The interviews have been transcribed and are kept for future references.

1. The first interview took place in Linköping on the 17th of March, 2009. The interviewee was Hans Sjöblom, who is head of the Airworthiness department at Saab Aerosystems. Hans also has the role of Flight Safety Manager, whose task it is to coordinate contacts with relevant authorities and operators in connection with incidents and accidents with a product developed and produced by Saab Aerosystems. The questions in the interview guide that were covered during the interview, not necessarily in the same order, are included in Appendix 2. The interview lasted for approximately 1 hour. Hans also provided additional information during a telephone conversation a couple of days after the interview.

2. The second interviewee was Anders Hägg, who is the accident and incident investigator and technical expert, as well as independent reviewer for system safety, at Saab Aerosystems. This interview took place the following day, the 18th of March, 2009, also in Linköping. The interview was based on the same questions as were posed to Hans Sjöblom, see Appendix2, and it lasted approximately 1,5 hours.

3. The third interview was conducted at the Swedish Armed Forces’ Headquarters in Stockholm. Two persons participated, Christer Olsson who is Head of the Flight Safety department and Lars Hall who is a Flight Safety investigator and works at the Flight Safety department. The interview took place on the 3rd of April, 2009 and lasted for a little more than an hour. The questions that formed the foundation for the semi‐structured discussion‐like interview are included in Appendix 3.

6.2 Interview Analysis and Conclusions The interviews have been analyzed and categorized in accordance with the interview guides. Many interesting subjects were discussed, but we have chosen to describe in more detail the following three areas of interest:

1. HTO issues; general views and practical applications 2. Incident investigation methods and models 3. Incident reporting systems and reporting culture

62

The quotations from the interviews have been translated to English by the authors. The original quotation in Swedish is given in smaller text right underneath the translation. The interview and page numbers shown next to the quotations and in the text sections are references to the transcription pages. The quotations have been numbered in order to allow for easy reference in other chapters of the thesis.

Abbreviations that are used in the interview analysis:

ST: Sara Thor, interviewer AK: Anna‐Karin Rosén, interviewer AH: Anders Hägg HS: Hans Sjöblom LH: Lars Hall CO: Christer Olsson

6.2.1 HTO Issues: General Views and Practical Applications The view on HTO varies among the interviewees. Hans Sjöblom at Saab Aerosystems explains that his view is that the HTO concept constitutes of a combination of technical issues and human aspects, with the organization as an umbrella above it all. (Interview 1 p. 1)

Anders Hägg, also at Saab Aerosystems, regards HTO from an accident investigation point of view and he says that accident and incident investigations used to be focused mainly on the aircraft and the pilot, the HMI (Human Machine Interface). During the last six to seven years, however, he feels that focus has shifted towards trying to find all the underlying causes; technical, human as well as organizational causes. This would, according to Anders, indicate a generally greater level of HTO thinking in the investigations. (Interview 2 p. 4)

At the same time he feels that there is still more focus on HMI, rather than all elements of HTO, in the design process at Saab Aerosystems, because the organizational aspect of safe designs is not always present or visible. (Interview 2 p. 6)

Hans thinks that it is in some aspects difficult for Saab to work entirely from an HTO perspective, considering that Saab’s primary responsibility is design and technology. The organizational issues in an accident investigation are often the responsibility of other links in the chain:

Q1 HS: HT we cover, at least, that much I can say. The “O”, like I said, is trickier, since it frequently falls outside what we can control and what we can easily affect, but you are situated in a context that you must try to adapt the system to, and it’s not always possible to rebuild that context.

(HS: MT fångar vi i alla fall, det kan jag säga. O:et är ju som sagt var besvärligare, eftersom det hamnar ganska frekvent utanför vad vi har kontroll på och kan påverka sådär enkelt, utan man sitter i en kontext som man måste försöka anpassa systemet till och, det går inte alltid att bygga om den omvärlden.) (interview 1 p. 7)

Hans also says that there is a need for an organizational perspective at Saab in the design process, but mainly from a product cycle point of view, i.e. from initiation of a product change until delivery to customer. (Interview 1 p. 9)

Both Hans and Anders alike say it is a bit delicate from Saab’s point of view to handle organizational problems they might find among the users or purchasers. (Interview 2 p. 10) Hans tells us that since there is generally a good relationship between Saab and the Air

63

Force open discussions about organizational problems can take place, and Saab can also report to the appropriate authorities. (Interview 1 p. 6)

It became apparent during the interviews with Hans and Anders that Saab has done investigations with an HTO perspective, where the investigators followed the chain of events backwards and found underlying causes, which clearly were organizational in nature. We discussed this with Anders, and he agrees that it does happen, but not in a regulated fashion since there are no written procedures for it. (Interview 2 p. 20)

Hans adds, during a telephone conversation a few days after the interview, that internal process revisions at Saab have been initiated and performed for several incident investigations. These investigations were all handled by the Airworthiness Board (AWB). The process revisions were initiated by Saab with the intention to improve work processes and reduce the risk of similar incidents happening.

When we discussed the possibility to have an acknowledged HTO position at Saab, Hans points out that it may be difficult for an HTO expert to be able to be a natural part of the day‐to‐day work. He says that:

Q2 HS: …… it would be like trying to glue something on from the outside.. . which is so very, very difficult and that’s why you have to implement it in the daily work.

(HS: …det här med att försöka klistra på nånting från utsidan….. är så jätte, jättesvårt och därför måste man ju plocka in det…. I det dagliga arbetet.) (interview 1 p. 9)

We again discuss the concept of HTO during our third interview with Christer Olsson and Lars Hall at the Flight Safety department at the Swedish Armed Forces’ Headquarter. Christer and Lars say that the concept of HTO has emerged gradually, especially during the last five years. (Interview 3 p. 6)

After being reviewed on account of safety culture, the Armed Forces in general, and the Flight Safety department in particular, are very much aware of the need to work from an HTO perspective. (Interview 3 p 16)

When asked about the organizational aspect of incident investigation, Lars brought forth the problem of the organization itself being the source of the problem:

Q3 LH: ….and we have been criticized by the Accident Investigation Board that we do not perform risk analyses as prescribed in both RML and by the Work Environment Authority, is that what they’re called? Their regulations, then and we’re still not doing that, in general in the Armed Forces, before the reorganization, we do not follow it up during the reorganization and not after the reorganization.…

(LH: … och vi har ju fått kritik av Haverikommissionen då att vi inte genomför riskanalyser så som föreskrivs både i RML och Arbetsskyddsstyrelsen, heter det så? Deras föreskrifter, då, och det gör vi ju fortfarande inte, alltså […] generellt i Försvarsmakten, inför organisationen, vi följer inte upp det under organisationen, och inte efter organisationen… (Interview 3 p. 11)

64

6.2.2 Incident Investigation Methods and Models When asked about incident investigation work methods and models, both Hans and Anders (interview 2, p. 8) say that they don’t use one specific model at Saab. The work methods have instead evolved from the experience of participating and performing incident analyses, both within the Saab company, and in collaboration with the Swedish Accident Investigation Board and the Air Force. Hans calls it a kind of “best practice”. (Interview 1, p. 4)

Hans also points out that the work methods at Saab have a strong position and visibility with regards to the technical and Human Factors aspects. The organizational aspects are present at Saab, but not quite as visible. (Interview 1 p. 9)

During the interviews we also talked about visualization of the models and the work with incident analysis. Hans says that there is not a common method for visualization at Saab (Interview 1, p. 12), it is depending on both the incident in question and the person involved in the investigation. Anders agrees that visualization is not commonly used at Saab; the events are instead usually described with words (Interview 2, p.11). Hans says that event trees are sometimes used and he recommends a systematic course of action, so that all possible causes are considered. He also stresses the importance of considering that some events may be harmless in certain circumstances, but very dangerous in other (Interview 1, p. 12).

Anders agrees with Hans that work methods and investigations at Saab are influenced by the HTO work that is performed by the Swedish Accident Investigation Board. (Int. 2 p. 8)

Hans has a positive attitude towards more systematic ways of performing incident investigations and he says that without systematic procedures and check‐lists there is a risk that personal opinions will influence the investigation and analysis too much:

Q4 HS:…. the danger is usually that you end up in, you are influenced by someone’s personal opinions …. check‐lists are good, because then you have to take a position, and then they can be rather irrelevant sometimes, the questions, but… you are forced to say yes or no…

(HS: … det som är livsfarligt är ju oftast att man hamnar i, man tar till sig någon persons tyckande …. checklistor är ju bra, för att man tvingas ta ställning, då, sen kan de vara halvt irrelevanta ibland, frågorna, då men… man tvingas säga ja eller nej, då.) (interview 1 p. 13)

In the Swedish Air Force the Flight Safety department at the Headquarters is in the middle of introducing the HFACS model into the work methods for incident investigations. Christer Olsson and Lars Hall say that the choice of this particular model is based on its widespread use internationally:

Q5 CO: It is that one that‐, internationally we hear about HFACS very often, it is somewhat of a standard.

(CO: Det är ju den som‐, internationellt när vi är ute så hör man HFACS väldigt ofta, det är den som är på nåt sätt nån standard.) (interview 3 p. 4)

They are aware that HFACS as a work method does not give a complete HTO view of incident investigation (Interview 3, p. 4), and they also talk about the work imposed by introducing these new perceptions in the Swedish Air Force:

65

Q6 CO: ….if we implement HFACS in the FSD [Flight Safety Database], then it will involve a larger work effort, it will require a larger work effort at the wings, and it is especially our flight safety officers that get, them that finally get all the DAs [Operational Disturbance Reports (ODRs)] and are supposed to deal with them and maybe do that analysis on the basis of HFACS, they will receive a higher work load……. we have to have consensus within the flight safety family, that is that the flight safety officers must believe that this is good and the right way to go, since it is them that will have to deal with the negative, and so will we ….

(CO: …. om vi implementerar HFACS i FSD:n, då kommer det innebära en större arbetsinsats, det kommer kräva en större arbetsinsats på förband, och framför allt är det våra flygsäkerhetsofficerare som får, dom som tillslut får alla DA:na framför sig, och ska hantera det, och kanske göra den där analysen utifrån HFACS, dom kommer ju få en ökad arbetsbörda……… vi måste ha konsensus inom flygsäkerhetsfamiljen, alltså flygsäkerhetsofficerarna måste tycka att det här är bra och rätt att gå den här vägen, eftersom det är dom som måste hantera dom negativa, vi också i och för sig….) (interview 3 p. 9)

Today, when incident investigations are initiated in the Air Force, one of the difficulties is how the incident shall be classified and this is commented by Lars, who also describes some of the work procedures involved. (Interview 3 p. 18)

6.2.3 Incident Reporting Systems and Reporting Culture The incident reporting system is specified in RML. Hans and Anders at Saab report that the system works well, but that in order to get a full understanding of an event, it is still necessary to talk to the persons involved. (Interview 1 p. 14 and Interview 2 p.17)

Anders also emphasizes the importance of using natural language in the report, not just checking in the right boxes. This is especially useful if an incident needs to be reviewed at a later stage if, for instance, a similar incident has occurred. (Interview 2 p. 16‐17)

When it comes to the quality of reporting, Anders says that most of the DAs are filled out pretty well, often because the pilots wants to tell what happened. (Interview 2 p. 17)

At the Headquarters we discuss the willingness to report. Lars Hall says that the willingness might have decreased during the reorganizations, but that it seems better at the present due to active campaigning from the Flight Safety department. This campaigning has included, among other things, visits to the squadrons for supply of information regarding the necessity of reporting, especially human errors. (Interview 3 p. 19)

The importance of a blame free culture was discussed. It is often said to be a prerequisite to report readiness:

Q7 CO: and above all we have the right kind of culture within the Air Force, with reporting readiness, and personnel not afraid to report and it is not being penalized, and so on, which always was an example of when it does not work, but I would say that people are fearless, and we have a non‐blame culture, and like everything is in place.

(CO: och framför allt har vi ju rätt kultur i Flygvapnet, med rapporteringsvillighet och man är inte rädd för att rapportera och man tror inte att man blir bestraffad och så vidare, vilket alltid var ett exempel på när det inte funkar, men jag vill påstå att folk är orädda, och vi har en non blame culture, och liksom allting finns på plats.) (interview 3 p. 12)

66

The Flight Safety department tries to gather incident reports that are interesting or of pedagogical value, to distribute them, along with comments, back to the air wings, Saab and FMV, among others. This is done to encourage reporting, but also for the feedback itself. (Interview 3 p. 19)

The Flight Safety department has chosen to implement HFACS in the reporting system. Implementing HFACS into the reporting system, and into the Flight Safety Database (FSD), is difficult:

Q8 CO: the hard part is, you could develop a new DA system, a perfect one, but the problem would be that you lose the history. And then you have to work in two databases, and we don’t want that, but we try to develop a way to expand the existing one,

(CO: det som är jobbigt med det här, man skulle ju kunna ta fram en, ett DA‐system som var helt nytt, och som var perfekt, problemet är att när man gör det tappar man hela historiken. Och då måste man ha två databaser som man jobbar i, vilket vi inte vill, utan vi försöker hitta ett sätt att utveckla det som finns,) (interview 3 p. 8)

67

7 Discussion and Recommendations

In this section we bring together what we learned from the literature research, the case study and the interviews. We will address the research objectives and end with recommendations for Saab Aerosystems and the Swedish Air Force. Questions about our methodological choices will also be addressed, as well as pointers towards possible future research.

7.1 Discussion and Conclusions The pragmatic research approach has guided our work process through a series of iterations, and we will now use that notion as a handrail when we summarize the process. At the same time we will address our thesis objectives, which were:

1. theoretically explore and analyze organizational models and accident models that can be suitable for incident investigations from an HTO perspective, and

2. analyze the incident reporting system in the Swedish Air Force using the selected models, and

3. suggest a model that the Swedish Air Force and Saab Aerosystems can work with, or suggest improvements of already used accident and incident models in the Swedish Air Force and at Saab Aerosystems.

We started our work by exploring and reviewing previous research in the field, formulating our research objectives and choosing six models to be studied more closely. The models, three organizational models and three accident models, were then evaluated with the help of the criteria for accident model evaluation that we had developed, based on Hendrick’s and Benner’s (1987) original list of ten criteria. We found that some of the models, although useful in other situations, were not so useful for the purpose of incident investigation, either for being too focused on organizational change (Porras and Robertson, 1992; Eklund, 2003) or for generating too much output (Sklet, 2002). Of the remaining three, Rollenhagen’s and Leveson’s models took a true systems and HTO view, and Shappell and Wiegmann’s model covered the field of human error in a way that makes it a good complement to any technology‐focused model for investigation. All three models were deemed to be descriptive, exhaustive and appropriate for accident investigation. This constitutes the first iteration.

The second iteration comprised of choosing a case that was deemed appropriate for our ends. According to the findings in the literary review and the analysis of the models, Rollenhagen’s, Leveson’s and Shappell and Wiegmann’s models were chosen for the case study. The case, the canopy burst of the aircraft SK60, was chosen in collaboration with representatives from Saab Aerosystems. It represents an incident with technical, organizational and human aspects, and since it was fully investigated already, we had access to all relevant information. The SK60 case was then analyzed with the help of the three models and by that we could further evaluate the models as well as the incident reporting system in the Swedish Air Force.

The case study revealed that the three models worked very differently when applied to the incident at hand. Rollenhagens’ HTO model was very useful, giving a thorough analysis in three steps, although it gave little advice on how to graphically represent the data.

68

Leveson’s STAMP yields an extensive analysis, albeit demanding a lot of work from the analyst. Although useful, it might be too big to take on in some cases. Shappell and Wiegmann’s HFACS proved easy to use at the first glance, although the difficulty to properly label the underlying factors according to the check list might pose a problem for an untrained analyst. With its extensive check list of possible human errors, it would however be a good compliment to a more technical investigation model.

The third iteration involved the planning, execution and analysis of the interview study, designed to shed new light on the reporting system and the possible use of models in incident investigation. We will return to the findings of the interview study in the second part of this chapter.

With the new knowledge gained from the interviews and the previous iterations, we arrive into the fourth, and last, iteration –this chapter. The conclusions drawn in this chapter ties the three parts of the triangulation – the theoretical study of the models, the case study and the interviews – together, giving us the possibility to form recommendations on possible improvements to incident reporting and analysis system at Saab Aerosystems and the Swedish Air Force.

As Renborg et al. (2006) states in their report, as described in section 3.5, that an organization will never learn from mistakes not reported. The Swedish Air Force has a well‐functioning incident reporting system that has been operational for many years. The reporting system, including how reported incidents are handled and analyzed at the manufacturer Saab Aerosystems, was described in Chapter 0. The reporting system was further discussed during semi‐structured interviews with representatives from both Saab Aerosystems and the Swedish Air Forces (Headquarters).

The Air Force reporting system is in theory an anonymous reporting system, using Johnson’s (2003) definitions of reporting systems, since the person filling out the report form does not need to state his or her name. In real life, however, the system can be regarded as more of a confidential reporting system since the Air Force in Sweden is a rather small community and anonymity is hard to achieve. During our interviews, both Hans Sjöblom and Anders Hägg from Saab Aerosystems supported this when they said that a phone call to the involved parties, as well as technical information from the black box or aircraft computers, is usually very useful when understanding what really happened during an incident or accident. It may not be enough to read the report from the incident.

The statistics for military aircraft indicate that the reporting system is indeed helping in the prevention of accidents and incidents, see for instance, Table 1 for JAS39 Gripen data. Contributing to that is most likely that the main barriers for a successful reporting system; punishment/enforcement, public access, criminal sanctions and civil litigation as described in section 3.5, are either not present or otherwise not strong in Sweden.

Another factor that may be contributing to the successful incident prevention system in Sweden is the use of multidisciplinary teams, described in section 3.4. People from the operator, the Air Force, are used to working together with people from the manufacturer Saab Aerosystems as well as specialists, when needed. This is confirmed by Anders Hägg who says that Saab often works together with the Air Force and the Accident Investigation

69

Board for serious incidents. The roles that Johnson (2003) describes (section 3.5) are all present in the Swedish Air Force Reporting system:

• the reporter can be either the pilot, the technician, the engineer etc, i.e. the person filling out the report;

• the initial receiver is the customer support department at Saab Aerosystems that then forwards the information to the Daily Product Meeting (DPM) and the Airworthiness Board (AWB) or the equivalent receiving desks at the Headquarters and FMV;

• the incident investigator can lead the investigation or help out and there are usually incident investigators at both Saab Aerosystems, the Air Force (often the Flight Safety Officer (FSO) at the air wing where the incident occurred, according to Lars Hall) and FMV;

• the role as safety manager can be interpreted differently, and at Saab Aerosystems the Safety Manager is responsible for leading Saab investigations and coordinating with the Air Force, and FMV when applicable, whereas the safety manager in the Air Force may have other responsibilities.

• FLYGI is the authority and has the role of regulator.

Interviews ‐ HTO issues

The interviews that were conducted showed that there is a general knowledge, both at Saab Aerosystems and the Air Force, about the concept and meaning of HTO (Human, Technology and Organization). From the manufacturer’s point of view it may sometimes be difficult to fully apply (Q1) an HTO perspective to incident investigations, since their responsibility is aircraft design and technical solutions. Some organizational and human aspects must be dealt with by the operators of the aircraft, for instance educational issues and routines in the day‐to‐day operation of the aircraft. For several incident investigations at Saab, however, company internal process revisions and other organizational analyses have indeed been performed, with good results, but these work methods are not regular procedures that have been written down.

Interviews ‐ Reporting systems /culture

The big recurring structural reorganizations in the Swedish Armed Forces, mainly due to changes in defense decisions, constitute in themselves an organizational aspect (Q3) of incident investigation. The Flight Safety department at the Headquarters considers that even if the reporting system is working well, and the safety culture in the Air Force is good with a blame‐free approach to reporting (Q7), the big organizational changes do affect flight safety and should be better analyzed.

In order to encourage reporting and providing feedback to the operators, the Flight Safety department at the Headquarters selects incident reports that are especially interesting or of pedagogical value. These reports are then distributed, along with comments, to the air wings, Saab and FMV, among others.

Interviews ‐ Models

Incident investigations at Saab Aerosystems are not performed in accordance with one specific theoretical accident model, the work is rather a combination of methods that has evolved from experience and influences from working together with the Air Force, the

70

Accident Investigation Board and FMV. The focus is often on the technical aspects of the incidents and much can be gained from looking at human and organizational factors as well, which Saab is aware of. There is a great deal of experience with Human Factors with regards to the interface and the interaction between the pilot and the aircraft, and to some extent the technicians and the aircraft, but general knowledge of human errors can be expanded.

Rollenhagen’s incident and accident investigation model is mentioned during the interviews and his methods are being taught at courses for investigators in Sweden. Leveson’s model STAMP is known of, but has not really been used by Saab or the Air Force for incident investigations or other applications. The Flight Safety department at the Headquarters is currently working (Q6) with implementing the HFACS model by Shappell and Wiegmann in the Flight Safety Database. Later on the forms that the operators fill out after an incident will be updated, to better incorporate HFACS in the entire reporting system.

From the accumulated knowledge we have gained by our work, as described and summarized in this discussion, we will in the next section give our thoughts on measures that could be taken by either Saab Aerosystems or the Air Force or both in order to obtain more of an HTO perspective in the incident reporting and investigation system.

7.2 Recommendations As a result of the information gathering and analyses performed in this master thesis, we have the following recommendations to make.

For Saab Aerosystems, we recommend:

• Saab does incorporate Organizational and Human Factors in some of their incident and accident investigations, but the methods for this work is not regulated by processes or written down. It is dependent on the investigators experience and knowledge. In order to obtain a better and more complete HTO perspective in their investigative work, Saab could learn from their experiences and evaluate the HTO work that already has been performed. The results should then be incorporated in existing processes and instructions for incident investigations to assure that these factors will be considered for future incident investigations.

• Besides learning from experiences, Saab could also make sure that the, for them, most useful ideas from the theoretical models (Rollenhagen, Leveson, Shappell & Wiegmann) are included in the processes and written instructions as well.

• Since the Air Force is currently applying Shappell and Wiegmann’s model HFACS to their Flight Safety Database, Saab could give some extra attention to that model in order to be in synchronization with the Air Force.

• One way for Saab to achieve more attention to Human Factors and HTO issues from a safety point of view is to employ someone who is given the task to focus especially on these issues. One challenge with an “HTO specialist” is however to implement the issues in the daily work, so that the HTO work methods really become a natural part of the engineering activities.

For the Air Force, we recommend:

• Since the Air Force’s Flight Safety Database is being updated with Shappell and Wiegmann’s model HFACS, and the reporting system including report forms is planned to be updated as well, it may be a good time to also consider influences

71

from other HTO models, such as for instance Rollenhagen and Leveson. HFACS is used internationally in the military aviation community, but it is limited to Human Factors issues and in order to obtain a broader HTO perspective in the reporting system, other models could be considered.

• When the Armed Forces, including the Air Force, are exposed to major reorganizations, risk analyses regarding the effects on flight safety should be performed.

For both Saab Aerosystems and the Air Force, we recommend:

• Use Leveson’s model STAMP for more complex and serious incidents, where human and organizational aspects as well as technical ones are present. STAMP may also be helpful if the context surrounding the incident includes different operational modes and/or changes with time.

7.3 Methodological Discussion Looking back during research work there are always things that could have been done differently, in a better fashion or maybe other things that ought to have been done, but were not. When evaluating research the two main questions concern the validity and the reliability of the work:

Validity pertains to the quality of the measurement; did we measure what we set out to? We think that the use of methods triangulation in order to increase the validity of our results was fruitful. The interview study gave us a valuable addition to our other two methods, the literature research and the case study, with its angle from the users’ points of view.

The reliability of our work, i.e. is our study performed in a correct manner, is a little more difficult to assess, due to the fact that this study is pragmatic in its nature. How do we find out if the study is correctly done? We believe that our use of interview guides while conducting the interviews, the fact that we tested all three models on the same case and the application of Hendrick and Brenner’s evaluation criteria in the analysis of the models makes the study reliable. This is, however, a social sciences study, making the question more difficult to address. For instance, what is a correctly performed interview? We believe that our choice to use triangulation strengthens not only the validity, but the reliability as well. The three study methods aim to answer the same questions, using different techniques, hopefully giving higher reliability to the results.

7.4 Future Research Possible routes for future research might include interviews with the Swedish Defense Material Administration (FMV), Flight Safety Officers (FSO) and pilots at the air wings as well as the Military Flight Safety Inspectorate (FLYGI). That might generate both a deeper and a wider view of the reporting system in its entirety in Sweden.

Other interesting possibilities could for instance be studying the upcoming process of implementing HFACS into the reporting system of the Air Force. The implementation process will be extensive, and will be well worth a study of its own, interesting not only from a safety or HTO point of view, but from an organizational perspective too.

73

8 References

8.1 Publications Amalberti, R. (1996). La conduite des systèmes à risques, Paris: PUF.

Amalberti, R. (2001). The paradoxes of almost totally safe transportation systems. Safety Science 37, 109‐126.

Eklund, J. (2003). An extended framework for humans, technology and organization in interaction, In: Luczak, H. and Zink, K.J., (Eds.). Human Factors in Organizational Design and Management ‐ VII. Re‐Designing Work and Macroergonomics – Future Perspectives and Challenges. IEA Press, Santa Monica, California, 47‐54.

Fishman, D. B. (1999). The Case for Pragmatic Psychology. New York and London: New York University Press.

Forsyth, D. (1998). Group Dynamics, 3rd Ed. Wadsworth Publishing Company, USA.

Hendrick, K. and Benner, L. Jr (1987). Investigating Accidents with STEP, Occupational safety and health; 13, Marcel Dekker Inc., New York, NY.

Hollnagel, E. (2002). Understanding Accidents ‐From Root Causes to Performance Variability, IEEE 7th Human Factors Meeting, Scotsdale, Arizona.

Jordan, P. (1998). An Introduction to Usability. Taylor & Francis Ltd. London.

Kvale, S. (1996). Interviews: an introduction to qualitative research interviewing. Thousand Oaks: SAGE.

Leveson, N. (1995). Safeware – System Safety and Computers. University of Washington. Addison‐Wesley Publishing Company Inc. USA.

Perrow, C. (1984). Normal Accidents. Princeton University Press. Princeton NJ, USA.

Porras, J. and Robertson, P. (1992). Organizational Development: Theory, Practice, and Research, in Dunette, M. and Hough, L. (eds.). Handbook of Industrial and Organizational Psychology, 2nd Ed, Vol 3, pp. 719‐741. Consulting Psychologists Press, Inc. Palo Alto, California, USA.

Rasmussen, J. (1997) Risk management in a dynamic society: a modelling problem. In Safety Science, Vol. 27, No. 2/3, pp. 183‐213. Elsevier Science Ltd. Great Britain.

Reason, J. (1990). Human Error. New York: Cambridge University Press.

Reason, J. (1997). Managing the Risks of Organizational Accidents. Ashgate Publishing Ltd. Great Britain.

Rollenhagen, C. (1997). Sambanden människa, teknik och organisation – en introduktion. Studentlitteratur, Lund, Sweden.

Rollenhagen, C. (2003). Att utreda olycksfall. Teori och praktik. Studentlitteratur, Lund, Sweden.

Rollenhagen, C., and Kahlbom, U. (2001). Towards a method for the assessment of safety activities and their associated organizational context. The 4th International Workshop on Human Error, Safety and System Development, 11‐12 June, 2001, Linköping, Sweden.

74

Sagan, S. (1993). The limits of safety: organizations, accidents and nuclear weapons. Princeton, N.J.: Princeton Univ. Press

Sklet, S. (2002). Methods for accident investigation. ROSS (Reliability, Safety, and Security Studies) at NTNU, Norwegian University of Science and Technology.

Wickens, C. and Hollands, J. (2000). Engineering Psychology and Human Performance. Third Edition. Prentice Hall, Upper Saddle River, NJ, USA.

Wiegmann, D. and Shappell, S. (2001). Applying the Human Factors Analysis and Classification System (HFACS) to the Analysis of Commercial Aviation Accident Data. Presented at the 11th International Symposium on Aviation Psychology. Columbus, OH: The Ohio State University.

Williamson, K. (2002). Research methods for students, academics and professionals – Information management and systems. Centre for Information Studies, Wagga Wagga, Australia.

8.2 Internet Sources Bryman, Alan (n.d.) Triangulation, in Encyclopedia of Social Science Research Methods,

retrieved April 25 2009, from: http://www.referenceworld.com/sage/socialscience/triangulation.pdf

Department of Defense (1993) Military Standard System Safety Program Requirements, (MIL‐STD‐882C), retrieved April 27, 2009 from: http://www.system‐safety.org/Documents/MIL‐STD‐882C.pdf

DoD HFACS: Department of Defense Human Factors Analysis and Classification System; A mishap investigation and data analysis tool. (n.d.) Retireved October 23 2008 from: http://safetycenter.navy.mil/HFACS/downloads/hfacs.pdf.

Firth‐Cozens, J (2001). Multidisciplinary teamwork: the good, bad, and everything in between. In: Quality in Health Care 2001;10:65‐66; retrieved April 16, 2009 from: http://qshc.bmj.com/

Flyvbjerg, B. (2006). Five Misunderstandings About Case‐Study Research, in Qualitative Inquiry, Vol 12, No 2, pp. 219‐245. Sage Publications. Retrieved at May 5, 2009, from: http://flyvbjerg.plan.aau.dk/Publications2006/0604FIVEMISPUBL2006.pdf

Johnson, C. (2003) Failure in Safety critical Systems: A Handbook of Incident and Accident Reporting. Glasgow University Press. Retrieved February 14 2009 from: http://www.dcs.gla.ac.uk/~johnson/book/

Leveson, N. (2002). System Safety Engineering: Back To The Future. Aeronautics and Astronautics. Massachusetts Institute of Technology. Draft book retrieved September 3 2008 from: http://sunnyday.mit.edu/book2.pdf

Renborg, B., Jonsson, K., Broqvist, K. and Keski‐Seppälä, S. (2006). Hantering av händelser, nära misstag. SKI rapport 2007:16. Retrieved November 17, 2008, from: http://www.stralsakerhetsmyndigheten.se/Global/Publikationer/SKI_import/070514/84a47ac0b5305268da2cb2fe2bedd5d0/web_2007‐16.pdf

75

Shappell, S.A. and Wiegmann, D.A. (2000). The Human Factors Analysis and Classification System –HFACS. Retrieved September 4, 2008, from: http://www.faa.gov/library/reports/medical/oamtechreports/2000s/media/00_07.pdf.

Statens Haverikommission (2008a). Retrieved November 16, 2008 from: http://www.havkom.se/index.html.

Statens Haverikommission (2008b). Olycka med en HKP10, nr 401, i havet söder Lindö, K län, den 1 november 2005, (Rapport RM 2008:03), retrieved November 16, 2008 from: http://www.havkom.se/virtupload/news/320077576_rm2008_03.pdf

Statens Haverikommission (2008c). Militär skjutolycka på Skövde skjutfält, O län, den 5 december 2007, (Rapport RM 2008:05), retrieved November 16, 2008 from: http://www.havkom.se/virtupload/news/2234680_rm2008_05.pdf

Wikipedia (December 1, 2008) Accident, retrieved December 1, 2008 from: http://en.wikipedia.org/wiki/Accident

Wikipedia (March 10, 2009). Team, retrieved March 10, 2009 from: http://en.wikipedia.org/wiki/Team

8.3 Reports Andersson, O. (2000). MTO – Tillämpning inom svensk kärnkraftsindustri. Forsmarks

Kraftgrupp AB, Östhammar, Sweden.

Lundqvist, Björkman, Docherty, Hill and Ullmark (1997) Företagsperspektivet, en analys av företagens behov av kunskap om samspelet mellan: Affärsidéer, människa, teknik, organisation. Nutek (Närings‐ och teknikutvecklingsverket), Stockholm.

8.4 Documents supplied by Saab Aerosystems AB Förvarets Materielverk (2002) RAFT: Rapporteringsanvisningar Flygmaterialtjänst.

Internal incident investigation reports regarding the SK60 incident, Saab Aerosystems AB

Internal process regulating documents, Saab Aerosystems AB

RML‐V‐2B: Regler för Militär Luftfart; Flygdrift – Rules for Military Aviation; Aircraft Operations

77

9 Appendix

9.1 Appendix 1 Criteria for Accident Model Evaluation

From Hendrick, K. and Benner, L. (1987) p. 413.

Realistic Model must represent reality, e.g., the observed nature of the accident phenomenon; model must represent both sequential and concurrent events and their interactions with time; model must permit representation of the risk‐taking nature of work processes in which accidents occur.

Definitive Model must define nature and sources of data required to describe the phenomenon; model must drive the investigation and analysis methods, rather than be driven by those methods; model must use definitive descriptive building blocks.

Satisfying Model must contribute to demonstrable achievement of an agency’s statutory mission and not undermine that mission because of technical inadequacies or inability to satisfy agency performance and credibility demands.

Comprehensive Model must encompass the development and consequences of an accident; model must define the beginning and end of the phenomenon being investigated and lead to complete description of events involved; model must help avoid ambiguity, equivocation, or gaps in understanding.

Disciplining Model must provide a technically sound framework and building blocks with which all parties to an investigation can discipline their investigative efforts in a mutually supportive manner; model must provide concepts for testing the quality, validity, and relationships of data developed during an investigation.

Consistent Model must be theoretically consistent with or provide consistency for agency’s safety program concepts; model must provide guidance for consistent interpretation of questions arising during an investigation and for consistent quality control of work products.

Direct Model must provide for direct identification of safety problems in ways that provide options for their prompt correction; model must not require accumulation of a lengthy history of accidents before corrective changes can be indentified and proposed.

Functional Model must provide functional links to performance of worker tasks and work flows involved in an accident; model must make it possible to link accident descriptions to the work process in which the accident occurred; model should aid in establishing effective work process monitoring to support high‐performance operation.

Noncausal Model must be free of accident cause or causal factors concepts, addressing instead full description of accident phenomenon, showing interactions among all parties and things, rather than oversimplification; model must avoid technically unsupportable fault finding and placement of blame.

Visible Model must enable investigators and others to see the relevance of model to any accident under investigation easily and credibly; interactions described should be readily visible, easy to comprehend, and credible to the public and victims as well as investigators.

78

9.2 Appendix 2 Interview Guide: Saab Aerosystems

The interview guide was originally in Swedish, but is shown here translated to English.

Date: Interviewee:

1 HTO as a concept How do you view the issue of HTO, its meaning and if it is used in practice (or on paper) at Saab?

2 Investigation methodology

2.1 Do you work according to any specific model?

2.2 Do you use any kind of visualization?

2.3 How do you know how far back you should go? How do you decide when the investigation is complete?

79

3 HTO in an investigative context

3.1 Is there an "HTO expert" employed at Saab? Or engaged, if necessary?

3.2 To what extent do you think you look at the organizational and human causal factors in investigations?

3.3 Does one settle with a ”practical” error, if one could find one, or does one look at the HTO aspect even when it is not directly obvious?

3.4 Who carries out the investigation? How to ensure that the interviewing people have insight on MTO?

3.5 Do you train other personnel to understand problems related to HTO and organizational perspectives?

80

4 Functionality of the incident reporting system. 4.1 Is there room for improvement/change, in your opinion?

4.2 Does the forms for reporting function in a good way?

5 Case study Specific questions on the SK60 incident, causes of the event.

81

6 Accident or incident

6.1 What is the difference between accident investigations (SHK) and incident investigations?

6.2 Do the regulations reflect reality in this respect (both concerning HTO and investigations)?

6.3 Is there a big difference in the methods SHK use?

6.4 Do you conduct your own investigations in the cases were SHK takes over?

82

7 Relations Saab – Swedish Armed Forces 7.1 To what extent do Saab and the Armed Forces (and FMV) collaborate in the

development of working procedures for incident reporting and investigation?

7.2 Do they work in the same way?

8 Other questions

83

9.3 Appendix 3 Interview Guide: Swedish Air Force, HQ

The interview guide was originally in Swedish, but is shown here translated to English.

Date: Interviewee:

1 HTO as a concept How do you view the issue of HTO, its meaning and if it is used in practice (or on paper) in the Armed Forces?

2 Investigation methodology (provided that they are involved in the investigations)

2.1 Do you work according to any specific model or method for investigation?

2.2 Is it correct that the Armed Forces plan to implement HFACS?

2.3 How will it be coordinated with existing data bases and report forms?

84

2.4 Is there an HTO expert at the Armed Forces, FlygSäk?

2.5 To what extent do you think you look at the organizational and human causal factors in investigations?

2.6 What is the difference between accident investigations (SHK) and incident investigations?

3 Case study Specific questions on the SK60 incident, causes of the event.

85

4 Functionality of the incident reporting system. 4.1 Is there room for improvement/change, in your opinion?

4.2 Does the forms for reporting function in a good way?

5 Relations Saab – Swedish Armed Forces 5.1 To what extent do Saab and the Armed Forces (and FMV) collaborate in the

development of working procedures for incident reporting and investigation?

86

6 Reporting 6.1 How do you handle the reporting in the Armed Forces? I. e. how do you work

to ensure that safety critical incidents are not repeated?

6.2 What criteria are there for what events to report?

6.3 Do you provide feedback to the persons reporting in?

6.4 Is there centrally administered education (or encouragement) on how to use the reporting system? Or is that handled at wing level?

6.5 How has the reporting frequency varied over time?

87

6.6 Is it your opinion that everything that ought to be reported really gets reported? Or is there a hidden proportion?

6.7 The forms for reporting have not changed much since they were first created around 50 years ago, have you identified any need for revisions?

7 Other questions.

Military Aviation Incident Reporting from an HTO Perspective

Documents

Transcript of Military Aviation Incident Reporting from an HTO Perspective