Development and evaluation of a flight collision avoidance ...

Development and Evaluation of a Flight

Collision Avoidance System in a Free-flight

Environment

By

Yakubu Ibrahim

MSc (Aeronautical Engineering, Poland)

A dissertation submitted in fulfilment of the requirements

of the degree Doctor of Philosophy

Faculty of Science, Engineering and Technology

Swinburne University of Technology

Melbourne, Australia

July, 2015

i

Abstract

There is an urgent need to update the current air traffic system to enable Air Traffic

Controllers (ATCO) and pilots to cope with the rising amount of air traffic. As such, a

new air traffic management (ATM) and airborne display system is required for the

monitoring of aircraft movements and the maintenance of minimum separation

standards. At the centre of the ATM is the automatic dependence surveillance broadcast

(ADS-B) which is used to alleviate flight level congestion, mid-air collisions (MAC)

and to simplify the complex, and cognitively demanding tasks on ATCO. The proposed

ATM represents a step towards a free-flight environment.

In a free-flight environment, the management of separation between aircraft is delegated

to the pilot by ATCO. Self-separation is a new task for pilots and, therefore, the

responsibility of pilots is likely to expand to include a new set of cognitive demands

associated with separation tasks. Indeed, tasks change as an aircraft moves from a

controlled environment to a free-flight environment. Susceptibility to pilot performance

also changes, as the types of knowledge and cognitive demands vary with the tasks. A

pilot’s mental model and situation awareness are vulnerable to problem complexity and

conflicting information such as relative distance, relative speed and flight path. This

relevant information to avoid collision is contained in one variable, the conflict angle.

By relying on this variable, a pilot’s complex decision-making process can be made

more efficient and consistent with his or her mental models.

Here, the study explored the usefulness of an ecological interface design (EID)

approach in a free-flight environment. Specifically, the study examined the performance

of pilots using the EID (experimental group) versus the conventional display (control

group) in potential collision situations. Applying an EID framework to Flight Collision

Avoidance System (FCAS) design is novel with regard to supporting pilots through

visual evaluation of the aircraft relation function as well as the protective cone function,

which brings to view the violation constraints. The interactive nature of these functions

is likely to lead to improved situation awareness and a reduction in cognitive workload

when manoeuvring to avoid air conflict.

ii

Data was collected using a mixed research method (i.e., qualitative and quantitative

method). A quantitative method was used to assess pre-and post-questionnaires and

flight simulation data. The questionnaires assessed the demographics, experiences,

perceived effectiveness and ineffectiveness of the FCAS. The flight simulation data was

used to assess the strategies the pilots used to manoeuvre to avoid conflict. A qualitative

method (i.e., interview) was used to assess pilots’ comments. The quantitative and

qualitative data were then statistically analysed to confirm the validity and reliability of

the research findings. A number of findings which emerged from the study have

practical implications for the free-flight environment. Pilots’ inconsistencies in avoiding

conflict with the Starboard1 approach have also been identified. Specifically, the results

of experimental evaluations of FCAS revealed that pilots benefitted from the improved

situation awareness, reduced mental demand, improved performance and decision-

making to successfully and safely cope with a pairwise mid-air collision over the

conventional display. The contribution of this analysis has yielded three meaningful

constructs, mental demand, performance and situation awareness differentiate between

the groups. It also was evident from the pilots’ comments that presenting a protective

cone and relative vector function significantly improves pilot decision-making over that

associated with the conventional display. Therefore, the FCAS provides a new level of

conclusiveness, completeness, clarity of goals and perception of collision, to help pilots

attain situation awareness, improve decision-making and reduce mental demand.

Further studies are required to develop appropriate procedures in avoiding conflict

when an Intruder is approaching from Starboard and pilots’ cognition activities. In

summary, this research is the first attempt at using relative velocity vectors and

protective cone functions to support pilots’ self-separation in a free-flight environment.

1 Starboard approach, an Intruder approaching off the right wing of Ownship. Ownship refers to the pilot’s own aircraft.

iii

Refereed Conference Papers

Higgins, P.G., and Ibrahim, Y. (2014) “Flight Collision Avoidance System for Self-

Separation”. Proceedings of the 2014 European Conference on Cognitive

Ergonomics, September 01-03, 2014, Vienna, Austria. ACM New York, NY,

USA. http://dx.doi.org/10.1145/2637248.2637264.

Ibrahim, Y., Higgins, P., and Bruce, P.(2014) “Development and Evaluation of a

Collision Avoidance Display for Supporting Pilots’ Decision Making in a Free-

flight Environment” Paper to be presented for the 4th Industrial Engineering and

Operations Management (IEOM2014), 7th - 9th January, Bali, Indonesia.

Ibrahim, Y., Higgins, P., and Bruce P. (2013) “Development of a Horizontal Collision

Avoidance Display to Support Pilots’ Self-Separation in a Free-flight

Environment” Paper presented for the 2nd International Conference on

Interdisciplinary Science for Innovative Air Traffic Management (ISIATM), 8th

- 10th July, Ecole Nationale de l’Aviation Civile, France

http://dx.doi.org/10.1145/2637248.2637264

iv

Declaration

I certify that this thesis contains no material which has been accepted for the award of

any other degree or diploma in any university or other tertiary institution in my name

and, to the best of my knowledge and belief, contains no material previously published

or written by another person, except where due reference has been made in the text. In

addition, I certify that no part of this work will, in the future, be used in a submission in

my name, for any other degree or diploma in any university or other tertiary institution

without the prior approval of the Swinburne University of Technology.

I certify that the thesis is less than 100,000 words in length, exclusive of tables,

diagrams, bibliographies, appendices and footnotes.

Signed: _____________________

Date: July, 2015

SUHREC Project 2012/002 Ethics Clearance

The research SUHREC Project: Evaluation of a Flight Collision Avoidance System in a

Free-flight Environment approval number 2012/002 for this thesis received the approval

of the Swinburne’s Human Research Ethics Committee on Tuesday, 27 March 2012.

v

Acknowledgement

First, of all, I am grateful to Almighty God for his guidance to complete this project.

I would like to express gratitude to my parents and family for the support and

inculcating in me the character and dedication required to pursue a study such as this to

its completion. I would like to express my thanks to my wife Mary, children Rachael

and Lydia for their support over the past few years.

I wish to thank my supervisors Dr Peter Higgins and Dr Peter Bruce for their guidance,

assistance and feedback during the course of this research. Thank you for your

suggestions, understanding and patience. My thanks also are extended to the Faculty of

Science, Engineering and Technology for awarding me with a tuition free scholarship

and financial support for this project. To the staff at the Nigeria College of Aviation

Technology, Zaria, thank you, especially to my colleagues Mr Patrick Igbru, Hajiya

Salamatu–Aliyu Yusuf, Suleiman Mohammed and Hirse Michael.

Finally, my intellectual debt is to Dr Sylvia Mackie and Dr Karola von Baggo for giving

me constructive comments and warm encouragement. I really appreciated it. I

would like to acknowledge to all who, directly or indirectly encourage me pursue this

topic. To all of the participants who took part in the experiment, thank you for your

time. Also my appreciation goes Ms Vesna Stefanov with proofreading.

viii

Table of Contents

ABSTRACT ................................................................................................. I

REFEREED CONFERENCE PAPERS ................................................ III

DECLARATION ...................................................................................... IV

ACKNOWLEDGEMENT ......................................................................... V

TABLE OF CONTENTS ...................................................................... VIII

LIST OF FIGURES................................................................................. XII

LIST OF TABLES................................................................................. XIII

CHAPTER 1. INTRODUCTION .............................................................. 1

1.1. Rationale for Developing a FCAS ......................................................... 3

1.2. Statement of the problem ....................................................................... 3

1.3. Research Objectives ............................................................................... 4

1.4. Scope of Research and Delimitations .................................................... 4

1.5. Research Question .................................................................................. 5

1.6. Research Hypotheses .............................................................................. 5

1.7. Significant Contribution of this Thesis .................................................. 5

1.8. Structure of the Thesis ............................................................................ 7

CHAPTER 2. FLIGHT COLLISION AVOIDANCE SYSTEM: ISSUES AND CHALLENGES ................................................................. 13

2.1. Introduction .......................................................................................... 15

2.2. Airspace and Collision Resolution Challenges .................................... 17 2.2.1 Automated Systems to Support See and Avoid Limitations .............................. 21 2.2.2 Examining TCAS Current Limitations .............................................................. 22

ix

2.3. A Free-flight Environment ................................................................... 24

2.4. Cockpit Automation ............................................................................. 27 2.4.1 Mental Workload: Vigilance and performance .................................................. 34

2.5. Conceptual Frameworks for Interaction Design .................................. 39 2.5.1 Mental Models of Complex Systems ................................................................. 40 2.5.2 Conceptual Models of Complex Systems .......................................................... 53

2.6. Situation Awareness ............................................................................. 57 2.6.1 System Awareness ............................................................................................. 62 2.6.2 Spatial Awareness: Mental Models.................................................................... 64 2.6.3 Loss of Situation Awareness .............................................................................. 66

2.7. Decision-making .................................................................................. 69

2.8. Automated Display System Design Approaches ................................. 74 2.8.1 The Control-Centred Design Approach ............................................................. 75 2.8.2 The User-Centred Design Approach .................................................................. 76 2.8.3 The Technology-Centred Design Approach ...................................................... 77 2.8.4 The Ecological -Centred Design Approach ....................................................... 78

2.9. Summary .............................................................................................. 86

CHAPTER 3. THEORETICAL FRAMEWORK FOR FCAS ............ 89

3.1. Introduction .......................................................................................... 91

3.2. Ecological Interface Design Applications ........................................... 93 3.2.1 Work Domain Analysis (WDA) ........................................................................ 94

3.3. Flight Collision Avoidance System Modelling ................................. 104 3.3.1 Relative Motion Function ................................................................................ 106 3.3.2 Protective Cone Function ................................................................................. 108

3.4. Flight Collision Avoidance System Mapping .................................... 116 3.4.1 Supporting Skill-Based Behaviour (SBB) ....................................................... 122 3.4.2 Supporting Rule-based behaviour (RBB) ........................................................ 123 3.4.3 Supporting Knowledge-based Behaviour (KBB) ............................................ 123

3.5. Summary ............................................................................................ 129

x

CHAPTER 4. RESEARCH DESIGN AND METHOD ....................... 131

4.1. Introduction ........................................................................................ 133

4.2. Data Collection Methods .................................................................... 135 4.2.1 Measures of Mental Workload and Task Performance .................................... 136 4.2.2 Measures of Situation Awareness .................................................................... 139

4.3. Experimental Methodology ................................................................ 142 4.3.1 Study Participants ............................................................................................ 142 4.3.2 Ethical Considerations ..................................................................................... 143 4.3.3 Experimental Environment .............................................................................. 143 4.3.4 Experimental Procedures ................................................................................. 145 4.3.5 The Scenarios and Collision Avoidance Resolutions ...................................... 147

4.4. Independent and Dependent Variables ............................................... 152 4.4.1 Measures of Independent Variables ................................................................. 152 4.4.2 Measures of Dependent Variables ................................................................... 152

4.5. Methodology for Data Analysis ......................................................... 153

4.6. Summary ............................................................................................ 155

CHAPTER 5. FINDINGS AND ANALYSIS ........................................ 157

5.1. Introduction ........................................................................................ 159

5.2. Methods .............................................................................................. 160 5.2.1 Scope of the analyses ....................................................................................... 160 5.2.2 Overview of research methodology ................................................................. 161

5.3. Results of Interaction with FCAS ...................................................... 163 5.3.1 Flying Hours .................................................................................................... 163 5.3.2 Performance measures ..................................................................................... 164 5.3.3 Mental Workload ............................................................................................. 176 5.3.4 Situational Awareness ...................................................................................... 179 5.3.5 Correlation Analysis ........................................................................................ 183 5.3.6 Pilots’ Appraisement of FCAS ........................................................................ 194

5.4. A Proposed Revision of Parker’s (1973) Model ................................ 206

5.5. A Proposed Revision of Wicken’s (1984) Model .............................. 208

5.6. Implications of Mental Models Theory .............................................. 211

5.7. Summary of Main Research Findings and Analysis .......................... 213

xi

CHAPTER 6. CONCLUSION AND CONTRIBUTION .................... 215

6.1. Conclusion ......................................................................................... 219 6.1.1 Theoretical Contributions by the Study ........................................................... 225 6.1.2 Methodological Contributions by the Study .................................................... 227 6.1.3 Practical Contributions by the Study ............................................................... 227

6.2. Limitations of the Current Study ....................................................... 228

6.3. Suggestions for Future Research ....................................................... 229

6.4. Closing Remarks ................................................................................ 230

REFERENCE .......................................................................................... 232

APPENDICES ......................................................................................... 274

xii

List of Figures

Figure 1-1 Overview of research development framework .............................................. 7

Figure 2-1 A flowchart of a Mid-Air Collision Factor (adapted from Parker, 1973) ..... 19

Figure 2-2 Self- separation assistance display (adapted from Ellerbroek et al., 2009)... 84

Figure 2-3 Autonomous operations planner interface (adapted from Dole et al., 2005) 85

Figure 3-1 Ownship in a level turn ................................................................................. 98

Figure 3-2: Nomenclature for turning flight: plan view (adapted from Hull, 2007). ..... 99

Figure 3-3: Nomenclature for turning flight: front view (adopted from Hull, 2007) ... 100

Figure 3-4: A typical 2D conflict scenario for lateral separation ................................. 107

Figure 3-5: A typical 2D conflict scenario with the protective cone ............................ 110

Figure 3-6: A typical 2D conflict resolution scenario for lateral separation ................ 112

Figure 3-7: Relative motion of aircraft on a collision course ...................................... 115

Figure 3-8: The problem (a) space is modified to provide the solution (b) space ........ 118

Figure 3-9: Decision ladder for the flight collision avoidance display ........................ 125

Figure 3-10: Flight Collision Avoidance System (FCAS) for horizontal separation ... 129

Figure 4-1: An Experimental Setup for Simulation ...................................................... 144

Figure 4-2: Overview of Conflict Resolution Manoeuvres by the Ownship for the

control group .............................................................................................. 150

Figure 4-3: Overview of Conflict Resolution Manoeuvres by the Ownship for the

experimental group .................................................................................... 150

Figure 4-4: Screen of Display Formats (Portboard Approach) .................................... 151

Figure 4-5: Screen of Display Formats (Starboard Approach) ..................................... 151

Figure 4-6: Screen of Display Formats (Head-On Approach) ...................................... 151

Figure 5-1 Overview of research methodology ........................................................... 162

Figure 5-2 Pilot (E3) Spacing performance in the experimental group ........................ 168

Figure 5-3: Pilot (E12) Spacing performance in the experimental group. .................... 169

Figure 5-4: Pilot (C6) Spacing performance in the control group. ............................... 170

Figure 5-5: Pilot (C7) Spacing performance in the control group ................................ 171

Figure 5-6 A revised Parker’s (1973) Mid-Air Collision Factors (MACF) model ....... 207

Figure 5-7 A revised Wicken’s (1984) model of pilot information processing ............ 209

file:///C:/Users/Yakubu%20Ibrahim/Desktop/Yakubu-Ibrahim_Thesis__2015_Submission_VER2.doc%23_Toc425134765






















xiii

List of Tables

Table 2-1 Overview comparison of (SRK) model between the Classical and Ecological

Interface Design Approaches ....................................................................... 80

Table 2-2 Summary comparison of two analysing information requirements techniques

...................................................................................................................... 82

Table 3-1: Airborne separation display in a free-flight environment ............................. 95

Table 3-2: Abstraction Hierarchy of Lateral flight ....................................................... 103

Table 3-3: Abstraction Hierarchy of Flight Collision Avoidance System .................... 103

Table 3-4: A system boundary for conflict resolution .................................................. 105

Table 3-5: Some selected features of domain invariants based on work domain analysis

.................................................................................................................... 120

Table 3-6: Function and conceptual advantage of the graphic forms in the ecological

interface ...................................................................................................... 127

Table 3-7: Function and conceptual advantage of the graphic forms in the ecological

interface ...................................................................................................... 128

Table 4-1: Question Contents and Levels of Situation Awareness ............................... 141

Table 4-2: The mean age of participants classified by group (N=21) .......................... 142

Table 4-3: A summary of the conflict resolutions and data required to avoid conflict for

both groups ................................................................................................. 149

Table 5-1 Presents the frequency and percentage of participants classified by the

number of flight hours (N=21) ................................................................... 163

Table 5-2 : Descriptive statistics associated with pilots’ performance on separation

measures ..................................................................................................... 166

Table 5-3 Descriptive statistics associated with pilots’ performance on heading

measures ..................................................................................................... 172

Table 5-4: Frequency and percentage of pilot preferences for the control group (N=8)

.................................................................................................................... 175

Table 5-5: Frequency and percentage of pilot preferences for experimental group

(N=13) ........................................................................................................ 176

Table 5-6: Results of the different task demands level by NASA-TLX scores ............ 178

Table 5-7: Reliability of assessment results .................................................................. 180

xiv

Table 5-8 Display mean ratings on seven subjective preferences questions for the

experimental and control groups ................................................................ 181

Table 5-9 Bivariate Correlations among separation, heading, workload and situation

awareness (N=21) ...................................................................................... 184

Table 5-10 Multivariate Tests of Significance (before to covariate adjustment) ......... 187

Table 5-11 Multivariate Tests of Significance (after to covariate adjustment) ............ 188

Table 5-12 Homogeneity of Regression slopes: Dependent Variables vs Hours, by

Group ......................................................................................................... 189

Table 5-13 Eigenvalues table ........................................................................................ 190

Table 5-14 Wilks’ lambda table .................................................................................... 191

Table 5-15 Canonical discriminant function coefficients (Unstandardised) ................ 192

Table 5-16 Unstandardized canonical discriminant functions evaluated at group means

.................................................................................................................... 193

of 303

Chapter 1. Introduction

“It's amazing how dependent we become on technology to solve all our problems when

sometimes going back to the basics gives the best results”.

Rochelle G. Williams

The future adoption of a Free-flight Environment is necessary, as Air Traffic

Controllers will be unable to handle the workload from the forecast doubling of air

traffic within the next two decades. In a Free-flight Environment, Air Traffic

Management delegates managing separation between aircraft to the pilot. The

responsibility of pilots will expand to include a new set of cognitive demands associated

with separation tasks. In managing separation, pilots adjust the speed and direction of

their aircraft, within specified constraint boundaries.

of 303

This page is intentionally left blank

of 303

1.1. Rationale for Developing a FCAS2

The ability of pilots and Air Traffic Controllers to apply the “see and avoid” technique

for self -separation has been severely challenged in detecting and resolving air traffic

conflicts in uncontrolled airspace. To perform self-separation manoeuvres in a free-

flight environment, pilots need a supportive tool that clearly shows conflict geometries,

and provides alternatives to overcome a loss of situation awareness. Pilot difficulties in

maintaining situation awareness arise from an inability to plan or arrange displayed

information in a specified form that will improve situation awareness and decision-

making. As new cockpit systems are being developed, and will continue to be

developed over the next five to ten years, it becomes increasingly critical to solve this

problem.

1.2. Statement of the problem

To allocate flight levels, currently Airspace Traffic Management (ATM) is running out

of airspace. The ATM can only assign one flight level to one aircraft at a time; no other

aircraft can fly within the buffer zone, or may depend on the navigational precision of

the aircraft instrument. As air traffic increases, both controllers and pilots will be faced

with a change in cognitive workload. New ways of maintaining separations and/or

situation awareness will be required.

Many industries, including the robotics, automotive and aviation industries, have been

investigating ways to resolve collision or conflict. Several studies have proposed

approaches for detecting and resolving conflicts in controlled or uncontrolled airspace

(a “free flight”). Kuchar and Yang (2000) have presented an extensive review of these

approaches relating to aviation collision systems to support pilots operating in a “free

flight” environment. However, these approaches, until now, have been based on

abstract, procedural or computerised systems, rather than human related approaches.

Only a few researchers have examined the application of Ecological Interface Design

(EID) to the design of an interface that includes geometric representations of vectors to

support the pilot’s flight collision avoidance manoeuvres. Therefore, there is a need for

2 Flight Collision Avoidance System

of 303

further research in applying an Ecological Interface Design approach to develop a Flight

Collision Avoidance System (FCAS) to be used in a free-flight environment. A well-

designed FCAS may improve pilot situation awareness and decision-making in such an

environment.

1.3. Research Objectives

The research objective is to understand the interaction between pilots and a collision

avoidance system in a free-flight environment, and how this understanding may to

contribute to safety. The study is partially designed to deal with issues such as pilots’

cognition (i.e., pilots’ information-processing models). However, more importantly, it

investigates how such systems are related to situation awareness, preferences and

cognitive workload. To address the study objectives, an EID approach by Vicente and

Rasmussen (1992) will be explored.

1.4. Scope of Research and Delimitations

The study investigates the potential of a Flight Collision Avoidance System to improve

pilots’ situation awareness and reduce cognitive workload in a free-flight environment.

The study uses a simulated flight-system to compare the performance of two collision-

avoidance displays (based on EID and the other not), within the bounds of the following

assumptions:

The participants in the research understand and follow the instructions about the

navigational and collision avoidance task.

The participants have at least basic flying training, physical and mental capacity,

and the ability to complete the collision avoidance task.

Both the experimental and control groups are comparable in terms of their

spatial visualization abilities in two-dimensional (2D) space.

The experimental study will be conducted in a controlled room using a desktop

computer-based simulator, with the analysis done only for the benefit of future

developers of a collision avoidance display.

of 303

The algorithm used for collision avoidance is limited to simple conflict

resolution scenarios.

The study does not model aircraft flight dynamics.

1.5. Research Question

The specific question posed by this research is as follows:

Which factors associated with the environment, aircraft constraints and

perception of collision significantly influence decision making by pilots when

manoeuvring their aircraft, such that the protected zone is not violated?

1.6. Research Hypotheses

The overarching hypothesis is that a Flight Collision Avoidance System, designed to

improve pilot situation awareness, will help pilots perform manoeuvres for avoiding

separation conflicts with aircraft that intrude into their flight path. The underlying

hypotheses are as follows. Compared to pilots that only have standard instrumentation,

a pilot using the FCAS will:

1. Have an improved situation awareness compared with the situation awareness

experienced by pilots using standard instruments (H-1).

2. Violate separation constraints less frequently than the horizontal minimum

separation standard of seven (7) nautical miles (H-2).

3. Have a lower mental workload compared with the mental workload experienced

by pilots using standard instruments (H-3).

1.7. Significant Contribution of this Thesis

This dissertation contributes to the area of airborne collision avoidance systems in a

free-flight environment. Specifically, it introduces a new flight collision avoidance

system. This dissertation establishes a path that can be used to address a particular

modelling problem of a new system. Cognitive Work Analysis (CWA) suffers from

methodological problems.

of 303

The main methodological contribution of this thesis is the combination and application

of theoretical concepts of both Attention and Cognitive Work Analysis (CWA) to

support the development of the FCAS. These two concepts have been studied

separately. The FCAS incorporates relative velocity vectors and a protective-cone

display. This system is specifically designed to provide pilots with relative aircraft

information, and has been implemented from the ground up to avoid influences from

traditional, computation-oriented display systems. In particular, it is based on a

modification of Parker’s (1973) mid-air collision factors (MACF) model. These rules

result in pilot difficulty in managing self-separation in a free environment. The FCAS

capabilities depend on displaying the relative directions and relative positions of the

traffic and obstacles in the vicinity, which have not been tested or simulated using a

flight simulator system. It is the purpose of this investigation to obtain pilots’

performance result from this system. The results presented in this dissertation suggest

that FCAS is useful both for optimising the intended flight path, as well as to ensure

pilots’ cognitive resource management is supported, thus helping to improve their self-

separation performance in a free-flight environment. The FCAS provides a supportive

ground for novel solutions, which is an improvement over current systems of self-

separation. This dissertation also contributes to a greater understanding of ways to

improve airborne conflict resolution, using an Ecological Interface Display (EID) that

could work in a realistic free-flight environment. The theoretical framework for

automation design presented by Parasuraman, et al., (2000) is revisited. The framework

suggested four stages by which pilots process information, including information

acquisition, information analysis, and decision selection and implementation. However,

in the current study, the modification of the framework is to include airborne planning

to further understand the decision-making dynamics of pilot self-separation in a free-

flight environment.

of 303

1.8. Structure of the Thesis

In Figure 1-1, the various aspects of a Flight Collision Avoidance System, based on

Ecological Interface Design are represented. These aspects are covered in the chapters

as highlighted below.

Figure 1-1 Overview of research development framework

Chapter 2 The emphasis is on a holistic approach to collision avoidance. The chapter

examines current airspace problems, and future challenges. It outlines the concepts of

airspace issues associated with collision avoidance problems in a free-flight

environment. Several examples of these problems either use ‘see and avoid’

techniques, or have problematic features in the display. The chapter provides

background knowledge and approaches related to the design of Cockpit Display Traffic

Information (CDTI) to mitigate the collision avoidance related problems.

This leads to the question of: Why do these problems occur when pilots are equipped

with a display to support them in cockpit activities? In order to address this question, a

review was conducted to examine the pilots’ cognitive limitations that often lead to

errors.

of 303

This has been recognised in the literature, and various research approaches have been

proposed. For example, one of the commonly proposed solutions has been to increase

the use of automation systems in the cockpit.

Studies, such as those conducted by Erzberger (2004), have suggested the need to

update the current Air Traffic Management (ATM) by using Automatic Dependence

Surveillance Broadcasting (ADS-B), which would represent a step towards a free-flight

environment. However, there are potential issues associated with a shift to a free-flight

environment. With a free-flight environment, the current buffer zone, which no other

aircraft can fly in, will be “eliminated” or “maximised”, as it is unnecessary, nor does it

depend on the aircraft’s navigational capabilities. Instead, it depends on the relative

motion.

To ensure that these problems are addressed, an ecological interface design approach is

considered with the goal of providing traffic information that supports a pilot in

collision avoidance situations. In order to contextualise the design approach taken in

developing the EID display, the chapter addresses pilots’ cognitive abilities and

interactions with automated display systems. The chapter considers pilots’ cognitive

ability in seeking to ensure that flying tasks, functions, conflict information, and

environmental issues are compatible.

This chapter also reviews the theoretical knowledge of mental models with the

introduction of two important ideas related to mental models: situation awareness and

cognitive workload. To examine what is meant by a mental model, the current study

begins by providing a definition for the concept of a mental model. Pilots’ mental

models have been analysed. In particular, pilots use mental models to understand how a

system works, and the environment in which the system operates. Mental models refine

detailed understanding of a system. Pilots’ mental models are vulnerable to both

problem complexity and conflicting information. These are crucial when pilots make

critical decisions. Cognitive processes affect these decisions. Further to this, the chapter

reflects on human limitations and their effect on the fidelity of mental models. The

discussions also point out potential accidents which are caused by pilot error.

of 303

The review provides background knowledge, and suggests methods for designing an

effective collision avoidance system.

The effectiveness of pilots’ mental models will help to support a conclusion through the

functionality of this new system. A flight collision avoidance system that improves

situation awareness and reduces workload requires an understanding of the pilots’

mental models. By understanding pilots’ capabilities and limitations, it will be possible

to design a system that is compatible with their mental models. This is important when

pilots make critical decisions, such as collision avoidance in a free-flight environment.

However, the current study has also focused on attributes such as mental workload,

situation awareness, capturing mental models, and decision-making that may have

contributed to the already stated problems. This led to the choice of the ecological

approach conducted in Chapter 3 to support the development of FCAS. This approach

allowed the affordance to be perceived and presented in FCAS, in which flight

constraints were displayed in the form of geometric representations of vectors visible to

pilots. FCAS features, such as the protecting cone and relation display are incorporated

to predict the time and distance to reach first loss of separation during collision

avoidance in a free-flight environment.

Chapter 3: Attentional theory and the theoretical framework of Cognitive Work

Analysis (CWA) are presented to support the development of FCAS. This chapter

explores attention as is related to the design of the Flight Collision Avoidance System

(FCAS). The decision ladder, one of the tools of CWA, is presented. Throughout the

thesis, the focus is on how to improve situation awareness, maintain minimum

separation standards, and reduce mental workload in a free-flight environment. The

FCAS that is developed and applied in an experimental setting is an interactive display

of constraints- associated with changes in heading, relative distance, and relative speed-

that pilots can use to avoid collision.

This chapter also presents an abstraction hierarchy as part of the Ecological Interface

Design (EID) approach. The objective of this approach is to make environmental

constraints visible to pilots. The visibility of these constraints should reduce pilot

mental workload, errors and improve situation awareness.

of 303

Thus, the approach will improve pilots’ performance, and ascertain how situation

awareness and mental workload fit into the attainment of the pilots’ goals, with the

support of the FCAS, such as maintaining safe-separation in a free-flight environment.

The FCAS consists of two displays: (a) a protective-cone function, and (b) a relative

motion function. The current Ecological Interface Design displays for conflict

avoidance map the relationship between these two displays to clearly show both

geometry of conflict and operational constraints. With the aircraft constraints visible, a

pilot might be able to predict instantly the possible future state of the system in many

conflict situations. Pilots’ mental models could then be captured and externalised to

improve situation awareness.

Chapter 4: The characteristics of three research methodologies (qualitative methods,

quantitative methods, mixed methods) and their importance are described. The chapter

uses a mixed method in ways that support rigorous inquiry. The chapter presents a

review of questionnaires, interviews, audio recordings and numerical simulation as

research tools for data collection, followed by a discussion on the scenarios to illustrate

strengths in the research design, as well as opportunities to enhance the results.

This chapter also affords essential information on the evaluation used in experimental

studies. A low-fidelity simulator is used for the experimental study. With the support of

the FCAS, pilots choose, and make decisions concerning flight paths independent of air

traffic control. Furthermore, the chapter includes pre-testing, ethics consideration, data

storage, a sampling approach and other approaches to data analysis

Chapter 5: The main objective of this research is to examine the influence of the

protective cone on collision avoidance in a free-flight environment. This chapter seeks

to advance the understanding of a pilot’s behaviour in relation to free-flight issues as

one of the key factors in air navigation and collision avoidance. Developing and

evaluating a Flight Collision Avoidance and control System achieved this.

The chapter presents findings and analysis in relation to the evaluation of collision

avoidance concerning the usefulness of the proposed FCAS. As a result, the

effectiveness of FCAS design is a fundamental issue in collision avoidance in a free-

flight environment.

of 303

The research problem and the experimental results are viewed from environmental and

aircraft constraints, as well as the perception of risks (i.e., collision). The interpretation

of the results is considered from multiple perspectives, such as mental workload, mental

models, and performance and situation awareness. This is fundamental because the

pilot’s effectiveness in maintaining separation standards will depend upon improved

situation awareness, lower mental workload and larger minimum separations standards.

This, in turn, will depend upon display enhancements of the traffic situation.

Chapter 6: Summary, conclusions, contributions to research and some closing remarks

will be presented. The goal of a free-flight concept is to ensure that pilots have the

freedom and responsibility to maintain separations, unlike the current ATM. The

chapter covers a holistic approach in relation to collision avoidance resolution in a free-

flight environment. The chapter presents contributions based on an integration of the

study findings, analysis and interpretation by using static flight simulations,

questionnaires and interviews. A number of findings will emerge from the study that

will have practical implications for the free-flight environment. Finally, this study

defines what sort of information is needed in order to draw conclusions about a pilot’s

behaviour and performance. The chapter concludes by stating some of the limitations of

the current study and suggests possible directions for further research. Enjoy!

of 303


of 303

Chapter 2. Flight Collision Avoidance System:

Issues and Challenges

We are what we repeatedly do. Excellence, therefore, is not an act, but a habit.

Aristotle

The purpose of Chapter 2 is to provide background knowledge and approaches related

to the design of the Flight Collision Avoidance System (FCAS). The chapter examines

current airspace problems and future challenges. Several studies have suggested a need

to update current Air Traffic Management (ATM) by using Automatic Dependence

Surveillance Broadcasting (ADS-B), which would represent a step towards a free-flight

environment. This chapter outlines concepts of airspace issues associated with “see and

avoid” problems in collision avoidance (Parker, 1973). “See and avoid” is an expression

by which pilots must be in conformity with each other, regardless of operating

Instrument Flight Rules (IFR) or Visual Flight Rules (VFR).

To ensure that the problems related to collision avoidance in a free-flight environment

are addressed, an Ecological Interface Design (EID) approach is considered (Rasmussen

and Vicente, 1992), with the aim of providing traffic information that is transparent and

that supports a pilot in collision avoidance situations in a free-flight environment. In

particular, pilots’ cognitive abilities constitute a mental model, mental workload,

situation awareness (Endsley, 1988), and decision-making. A possible solution would

be to reduce the number of alternatives available to pilots. For example, a pilot needs

information, such as where another aircraft on a conflicting path is coming from as well

as its relative distance, relative speed and direction to determine whether the aircraft is

on a collision course (Doble, et al., 2005). This confict-avoidant relevant information is

contained in one variable, the conflict angle. By relying on this variable, a pilot’s

complex decision-making process can be made more efficient (i.e., the conflict situation

becomes consistent with his or her mental models).

of 303


of 303

2.1. Introduction

The 1956 Grand Canyon mid-air collision shaped the way aviation systems are

integrated. This air crash brought about the installation of radar surveillance technology

and the setting up of minimum separation standards. The current Air Traffic

Management (ATM) uses this technology to calculate aircraft positions, speeds,

headings, and ranges of moving objects by interrogating aircraft transponders (Vortac

and Edwards, 1993). However, aircraft detection, and tracking radar returns may be

delayed. The delay (i.e., blind spot) in broadcasting traffic data may lead to inefficient

scanning periods by ATCO. Because of this delay, the ATCO has no way of precisely

knowing traffic positions and it becomes difficult for ATCOs to keep track of aircraft

movements at all times. Therefore, ATCOs lose the ability to project the future of air

traffic positions (Endsley, 1998), resulting in radar monitoring errors (Shorrock and

Kirwan, 2002). AS a result, aircraft may have to fly within confined boundaries to

maintain separation standards, as constrained by ATM. The “see and avoid” technique

is a basic technique for avoiding mid-air collision under visual flight rules (VFR)

(Parker, 1973). However, the application of the “see-and-avoid” technique to avoid

mid-air collision in both controlled and uncontrolled airspace has failed to support both

pilots and ATCOs in a conflict situation (Morris, 2005).

Air traffic is predicted to double in the next two decades (Sheridan, 2009). The constant

stream of aircraft can overwhelm the ability of even the most experienced ATCOs.

ATM manages both controlled and uncontrolled airspace (Morio, Lang and Tallec,

2012). Controlled airspace is a protected airspace (“controlled activity”) within which

controlled services, such as maintaining constant smooth traffic flow, are provided by

ATM. However, uncontrolled airspace is regarded by ATM as unattended airspace;

there is limited interaction between pilots and controllers. Illman, (1993, p. 42) defined

uncontrolled airspace as “…no air traffic control service to either IFR or VFR aircraft

is provided other than possible traffic advisories when the air traffic control workload

permits and radio communications can be established”. Unlike controlled airspace,

radio communications with pilots in uncontrolled airspace are limited. Pilots are

responsible for maintaining minimum separation standards with minimum intervention

from Air Traffic Controllers (ATCOs).

of 303

It is therefore imperative that we begin to understand the way pilots manage difficult

situations within the context of avoiding a mid-air collision in a free-flight environment.

The current study focuses on uncontrolled airspace (i.e., a free-flight environment) and

will examine the issues related to collision resolution in airspace to provide a better

understanding of what aspects of collision avoidance are important. A primary aim of

the current study is to highlight the difficulties associated with the flight environment.

Understanding the flight environment can help support the formatting of the design

concept.

The introduction of the Ecological Interface Design (EID) framework (Vicente and

Rasmussen, 1992) is designed for interfaces to improve overall pilot performance and

reduce mental workload by externalising mental models onto the abstraction hierarchy

to make the environmental constraints visible to pilots. The EID provides specific

guidelines for the design of a new system, as compared with conventional approaches

such as the User-Centred Design. The fundamental principle of the EID approach is

Work Domain Analysis (WDA), Skill, Rule and Knowledge (SRK) model, and the

decision ladder. The new cockpit automated display systems being developed will

replace the current Traffic Collision Avoidance System (TCAS) and will give pilots the

capability and ability to change flight trajectories at any given time. For example, with

the support of a Cockpit Display Traffic Information (CDTI) system (Thomas and

Rantanen, 2006) and/or ATCO, pilots will be able to choose and make flight path

decisions independently, supporting situation awareness (Endsley, 1998) and decision

making (in Cognitive Work Analysis: Rasmussen, 1986).

of 303

2.2. Airspace and Collision Resolution Challenges

In 1956, under visual flight rules (VFR) conditions, 128 people on board a DC-7 and a

Lockheed Constellation lost their lives in a mid-air collision over the Grand Canyon

(ASN, 2010). As with any major accident, several causes were involved. These causes

included poor visibility, pilots and ATCOs’ mental workload, deviation from the

assigned heading, and the same flight levels. Normally, pilots use radio to communicate

with ATCO to maintain separation. Air safety is embedded in this use of radio

communication; the radio communications convey standard flight information to pilots

for maintaining separation and this flight information includes identification of aircraft,

speed, flight level and position. As communications channels and flight levels become

congested, there is potential for human error (Reason, 1990) and the likelihood of mid-

air collision increases. With increasing numbers of aircraft, knowledge of air traffic

operating procedures become more important for pilots. ATCO have a mental “picture”

of air traffic movements based on their training and experience (Niessen and Eyferth,

2001; Shorrock and Isaac, 2010) and they use this reliable “picture” of air traffic

information to guide pilots around potential air problems, such as mid-air collisions.

Air Traffic Management (ATM) currently control, manage and provide services in

airspace (Morio, Lang and Tallec, 2012). These services include clearances, weather

reports, and maintenance of separation standards. In managing separation standards,

pilots change speed and direction of their aircraft to avoid collisions. However, besides

adhering to rules and procedures issued by the ATM, pilots are also responsible for

applying the “see and avoid” technique to ensure minimum separation standards are

maintained for safety purposes. “See and avoid” is a basic technique for avoiding mid-

air collision under visual flight rules (VFR).

The ”see and avoid” technique is an important factor in conflict resolution and still

remains one of the basic flight principles of avoiding mid-air collisions with minimum

intervention from ATCOs (Colvin, Dodhia and Dismukes, 2005). Mid-air collisions

often occur because the pilots “failed to look”, “looked but failed to see”, and “could

not see” the incoming traffic (Parker, 1973).

of 303

As argued by Parker, pilots spent fifty per cent of their time scanning for other traffic

outside the cockpit window, and forty per cent searching for traffic within the airport

vicinity. To address the question of whether there would be relative improvement in

standard patterns approach with bank angles limited to less than 15 degrees at an

altitude of approximately 1000 ft, Packer presented a model of evaluation of mid-air

collision hazards in uncontrolled terminal airspace. The model is set to determine the

improvement in a pilot’s ability to see another aircraft for various changes in traffic

pattern approach.

To simulate a pilot’s see-and avoid problem, pilots flew standard approaches at altitudes

near the published pattern altitude on a high simulator with actual flight data of mid-air

collisions that occurred in the uncontrolled terminal airspace. Airspace block space data

were characterised as variables such as the aircraft heading, flight path, bank angle,

altitudes and distance separation. Time of day was used to simulate various air traffic

situations. Packer reported pilots could not see each other at a close proximity due to

vision envelope restrictions. The view angle depends on those variables, as previously

stated. Parker’s results also indicated that pilots have a limited amount of time for

detecting another aircraft at critical closing speeds. Furthermore, pilots are unable to see

another aircraft for approximately one-third of the time during the last two (2) nautical

miles of closure during their approach. Other factors that would have reduced the

possibility of a pilot seeing another aircraft within the vicinity include 1) the detection

of aircraft against the sky background, 2) aircraft with a head-on profile at a closure

from two (2) to three quarter ( ¾ ) nautical miles would provide limited relative

movement in the pilot’s field of view at that critical time and 3) the pilot’s attention is

diverted from detecting aircraft between the 120 to 90 seconds time period towards the

runway in preparation for approach landing.

According to Parker, a high percentage of pilots performing non-standard entries

observed trends to support NTSB3 conclusions that lack of the ability to see each other

may be a factor for concern. Further to this, Parker has documented the categories of

breakdowns of mid-air collision, as shown in Figure 2-1.

3 NTSB - The National Transportation Safety Board

of 303

According to Graham and Orr (1970), when flying at low closing speeds, only 5 per

cent of potential mid-air collisions are avoided. However, when flying at high closing

speeds, such as above 400 knots, pilots tend to overestimate the miss distance between

the two aircraft. The effective use of the “see and avoid” technique reduced mid-air

collision by approximately 50 per cent.

Schuch (1992) conducted a study, involving skilled pilots, to examine the relevance of

pilots’ flight experience on mid-air collision risk perception. Schuch concluded that

because these pilots frequently operated flights without an incident, they become

complacent during the flight and stop scanning the sky. However, novice pilots may

maintain an unrealistically high perception of mid-air collision risk. This hypothesis can

be investigated in the context of perceived mid-air collision probabilities. Taneja and

Wiegmann (2001) reviewed mid-air collisions and found that more than two-thirds of

mid-air collisions occurred when aircraft were on a convergence course.

MID-AIR COLLISION

FAILED TO SEE-AND-AVOID

COULD NOT SEE

Vision Envelope

Traffic Pattern

Manoeuvres

LOOKED BUT FAILED TO SEE

Background

Contrast

Visibility

Movement

FAILED TO LOOK

Cockpit Duties

Runway Fixation

Distractions

Figure 2-1 A flowchart of a Mid-Air Collision Factor (adapted from Parker, 1973)

of 303

Conflict geometries of aircraft on a convergence course are not easy to detect and

resolve. For example, pilots find it difficult to resolve conflicts on a converging

collision course (Mackintosh, et al. 1998). Lack of proper use of the “see and avoid”

technique has prevented both pilots and controllers detecting and resolving conflict

(Morris, 2005). Both Morris (2005) and Rantanen et al. (2004) have suggested that the

phase of flight under consideration is not important but visual perception is critical.

That is, pilots find it difficult to detect aircraft on a converging collision course. Aircraft

on a collision course appear to pilots to be small, motionless, and to blend with the

environment. Therefore, traffic seems invisible before a collision occurs (Morris, 2005).

With respect to aircraft movement, flight dynamics of a heavy aircraft at a slow speed

make it difficult for pilots to ascertain whether the aircraft is climbing or descending

(Morris, 2005). Accordingly, Morris suggested that aircraft on a convergence course

have been cited as a major cause of mid-air collisions.

In summary, the “see and avoid” method is recognised as a common technique for

avoiding mid-air collision in a free-flight environment. The technique has serious

problems in mitigating the risk of mid-air collision, due to limitations of pilots’ visual

perception. For example, clouds may prevent the sighting of aircraft due to a blurred

image of the aircraft. This may take an inordinate time to detect and resolve and, as a

result, pilots’ evasive actions may not be successful. With these limitations, the “see-

and-avoid” technique is not expected to play an important part in a free-flight

environment.

As the “see and avoid” technique cannot, in itself, ensure air safety, Traffic Conflict

Avoidance Systems (TCAS) were developed to help pilots avoid mid-air collisions.

Parker’s (1973) model should support the design of the FCAS by taking into account

previously discussed factors, specifically, the provision of salient information to pilots

in relation to aircraft relative movement in the pilots’ field of view and attention.

of 303

2.2.1 Automated Systems to Support See and Avoid Limitations

To overcome the limitations of the “see-and-avoid” technique in resolving collisions

across many domains, several automated systems and models have been investigated in

fields including robotics, automobiles, and aviation. In particular, aviation studies have

proposed approaches for detecting and resolving air conflict. Kuchar and Yang (2000)

have presented an extensive review of these approaches. These approaches, until now,

have been based on procedural systems, in connection with the radar system and the

TCAS, to advise pilots, rather than approaches to support pilots’ perception in relation

to collision detection and avoidance. These procedural systems can be classified into

three categories (i.e., instructions, optimisation and potential field). The first category,

instructions issued by automated systems such as TCAS and the Ground Proximity

Warning System (GPWS) are based on a set of “rules of flight”. However, these rules

usually do not take into account issues such as aircraft performance and pilot decision-

making. The second category optimised the set of rules of detection and resolution. This

so-called ‘optimisation method’ is based on the mathematical determination of the

closest approach, subject to cost constraints. The advantage of this method is that it uses

a set of rules to optimise flight paths. However, pilots may find these rules difficult to

understand or use during unexpected events. Another method produced by Durand et al.

(1996), based on a genetic-algorithm optimisation of conflict resolution, applies to

horizontal resolutions. Its core component is stochastic optimisation. The advantage of

using this technique is that the model can handle multiple conflicts globally with

minimum information. However, the model is not meant to resolve pairwise conflicts.

Durand et al. further pointed out that it is important that models support pilots’ future

trajectory manoeuvres. Separation tasks associated with Durand et al’ s model are

designed to automatically handle controllers’ cognitive capabilities. However, the

controllers’ cognitive ability to withstand environmental pressures associated with

coping with the increase of traffic flow may be tested to its limit in a free-flight

environment (Pawlak, 1996).

of 303

Another study by Bilmoria (2000), using genetic algorithm optimisation for conflict

resolution, presents a different approach to Durand et al. (1996). Utilising aircraft

position and optimisation of velocity vectors, it resolves conflicts via changes to aircraft

speed and heading. Its advantage is that the trajectory changes are “apportioned” or

“shared” by the other aircraft involved in the conflict. The “shared” conflict information

changes would improve pilots’ situation awareness. Thus, the model may be suitable for

free-flight operations. However, the study did not conduct experimental studies to

validate the genetic algorithm presented.

The third category, the Potential Field approach has been investigated by Koren and

Borenstein (1991) and Schneider and Wildermuth, (2003). This approach automates

navigation systems for detecting obstacles and avoiding collisions. It is widely used in

mobile robotic applications (e.g. unmanned aerial vehicles). In this approach, the

“bodies” apply a “force” (i.e., a repulsion or attraction force) to guide the robot towards

a specific path (Koren and Borenstein, 1991). However, the method is impractical for

civil aviation operations where a continuous change of aircraft heading and speed is a

major factor in maintaining safety.

2.2.2 Examining TCAS Current Limitations

To support pilots coping with the limitations of the “see-and-avoid” technique and

errors by Air Traffic Controllers (ATCOs), an airborne Traffic Collision Avoidance

System (TCAS) was developed. The primary purpose of TCAS is to prevent mid-air

collisions and reduce errors by both ATCOs and pilots by providing a traffic advisory

(TA) of a potential threat and a resolution advisory (RA). A resolution advisory (RA)

for collision avoidance is issued as a last resort for pilots to resolve conflict (Graham,

1989; Williams, 2004). The TCAS is modeled to resolve conflict by directing them to

ascend or descend from the intended flight paths, to assure that minimum separation

standards are maintained (Williams, 2004; Kuchar and Drumm, 2007). However, mid-

air collisions involving TCAS do occur. These collisions include the 1996 Charkhi

Dadri mid-air collision between Saudi Arabian Airlines Flight 763 and Kazakhstan

Airlines Flight 1907, the mid-air incident in 2001 in which two aircraft from Japan

Airlines (Flights 907 and 958) nearly collided, and, most recently, the 2002 Überlingen

of 303

mid-air collision between Bashkirian Airlines Flight 2937 and DHL Flight 611. In the

case of both the 2001 incident and the 2002 accident, the on-board TCAS and the air

traffic controller issued conflicting instructions to the pilots and, as a consequence, the

situation awareness of the pilots and the ATCOs may not have been compatible. These

incidents clearly indicate that there is no proper cooperation between pilots, TCAS and

the ATCOs.

Another problem concerns the resolution advisories (RA) issued by TCAS to pilots for

an evasive manoeuvre. The RA may require action outside the aircraft safety-flight

envelope4 as the flight progresses (Kuchar and Yang, 2000). Importantly, the use of

TCAS as last-minute collision avoidance guidance directly to pilots may not be suitable

for the next generation of transportation systems for a number of reasons. First, the

TCAS is intended only to command manoeuvres in ascent or descent to resolve

conflicts. As a result, aircraft may burst the flight level in both cases to avoid collision.

Second, the TCAS design is based on time to go to the closest point approach (CPA).

Third, it is not designed to support aircraft relative speed with respect to the intruder

and the intruders’ intent information is not supported. The design also fails to provide

alternative guided flight paths to support pilots’ decision making, resolution

manoeuvres such as crossing astern or ahead of the Intruder, and best climb rates or

better sink rate instructions in vertical resolution. Finally, TCAS alerts may not have

considered aircraft performance constraints during the negotiation of conflict resolution

advisories (RA) to pilots and, critically, pilots have no provision to revise the TCAS

RA. Thus, at present, TCAS does not have the capability to provide pilots with the

required information for strategic planning, as compared to the proposed Cockpit

Display Traffic Information, as suggested by Thomas and Rantanen (2006). Notably,

the current conflict avoidance information provided by TCAS is not configured or

designed for strategic planning (Dowek, Munoz and Geser, 2001). For example, a major

limitation of TCAS is that the system only deals with procedural knowledge tasks, and

does not help pilots to change plans or goals for an alternative manoeuvres 4 Safety-flight envelope relates to aircraft safe performance limitations, such as

minimum and maximum operating speeds, its service ceiling, and aerodynamic

operating limits.

of 303

consideration. Kochan, Jensen and Chubb (1997) note that experienced pilots should

have the opportunity to revise their procedural knowledge or plans, and simultaneously

access multiple interpretations of a current situation.

2.3. A Free-flight Environment

Recent literature suggests that the number of aircraft will double in the next three

decades (Metzger and Parasuraman, 2001; Hollnagel, 2007; Sheridan, 2008) and current

ATM may not be able to support the growing numbers of passengers, higher and faster

aircraft while sufficiently maintaining separation standards. The constant stream of

aircraft could tax the ability of even the most experienced air traffic controllers (Prinzo,

2003). Thus, controllers’ cognitive workload is predicted to increase. The increase in

aircraft speed will also challenge the pilots’ ability to apply the “see and avoid”

technique and, therefore, aircraft with higher speed and altitude capabilities are

predicted to require an increased flow of information to pilots. This will challenge the

applicability of the “see and avoid” technique to avoid collision. The current ATCOs’

skills of monitoring flying activities and advising pilots may be insufficient to meet the

demands of information flow to avoid mid-air collisions. Moreover, the use of a radar

system places enormous responsibilities on current ATCOs and, thus, the possibility of

violating minimum standards is certain to occur which poses a major threat to air safety

(Pape, Wiegmann and Shappell, 2001). As a result, a free-flight environment must be

considered. The Radio Technical Commission for Aeronautics (RTCA) defines a free-

flight environment as:

“…efficient flight operating capability under instrument flight rules (IFR) in which the

operators have the freedom to select their path and speed in real time. Air traffic

restrictions are only imposed to ensure separation, to preclude exceeding airport

capacity, to prevent unauthorized flight through special use airspace, and to ensure

safety of flight. Restrictions are limited in extent and duration to correct the identified

problem. Any activity which removes restrictions represents a move towards a free

flight.”

of 303

A free-flight environment has previously been proposed to support pilots’ problem-

solving tasks, such as flight planning and maintaining aircraft separations in super-

density airspace (Thomas and Wickens, 2008). As air traffic volume has increased,

airports have become busy places, and air traffic movements in the current system are

beginning to overwhelm controllers (Vortac and Edwards, 1993; Andrews, 1978). It is

predicted that the free-flight environment will improve pilots’ situation awareness,

reduce mental workload, and accommodate environment constraints.

In the free-flight environment, pilots are able to choose flight paths to their destination

and maintain self-separation with the support of Cockpit Display Traffic Information

(CDTI). The CDTI supports pilots with information about the Intruder5’s intent and

visual capabilities in the environment (Thompson and Sinclair, 2008) and will enable

pilots to maintain situation awareness of the environment (Endsley, 1988; Flach, 1995).

Automatic Dependence, Surveillance–Broadcast (ADS-B) plays a central role in the

provision of collision traffic information in a free-flight environment (RTCA, 1995;

Livack et al., 2000; Prinzo and Hendrix, 2006). The ADS-B is a sophisticated Global

Position System (GPS) receiver that graphically paints a detailed picture of the

surrounding traffic. The ADS-B targets weaknesses in the current ATM, such as pilots’

constant radio communication with ATCO, the use of the “see and avoid” technique to

maintain separation standards, circumnavigating weather conditions and restricted

airspace, and airport congestion. By extending surveillance beyond non-radar airspace,

ADS-B allows the redefinition of ATM. Currently, in maintaining separation, air traffic

is restricted to operate within constrained boundaries. This restriction places the burden

on pilots to ensure that separation standards are maintained thereby applying

disproportionate responsibilities on ATCOs. The ADS-B technology that is being

developed might provide the solution to these restrictions (Prinzo and Hendrix, 2006).

In contrast to radar systems, the ADS-B system automatically allows data transmission

to the surrounding traffic simultaneously. Available information from the system

including flight level, heading, wind vector and relative aircraft position are

5 The “Intruder” is an individual aircraft, which presents the threat to the Ownship’s intended flight path.

of 303

simultaneously broadcast twice per second to air traffic within the vicinity (Thompson

and Sinclair, 2008) to support pilots with a detailed picture of any air traffic within the

vicinity to improve their situation awareness (Endsley, 1988). Pilots can identify and

monitor the flying activities of other traffic, including other traffic behaviour. This

improved situation awareness is predicted to function more effectively than in the

current ATM by enabling pilots to make decisions and manage self–separation

independently from ATC. Despite the ability of pilots to use the current TCAS to avoid

mid-air collision, according to Delzell, Johnson and Liao (1998), self-separation is a

new task. Consequently, the new environment may introduce new systems, procedures

and tasks compared to the current ATM. This new responsibility may pose a challenge

to both pilots and ATCOs in conflict situations, thus imposing a new mental workload

and previously known problems associated with pilot mental workload will change in

nature.

An automated display system that is designed to reduce ATCOs or pilots’ mental

workload and aid in maintaining separation standards does not guarantee aircraft

accidents will not happen again. However, the number of requests by pilots to report

flight parameters at a regular interval or to change flight routes to maintain separation

would be reduced (Vortac and Edwards, 1993). Additionally, under a free-flight

environment, pilots will be able to not only visualise collision traffic information, but

also to perform and maintain self-separation (Bone et al, 2002). Furthermore, with the

support of the CDTI system and/or Air Traffic Controllers (ATCO), pilots should be

able to choose a flight path, different aircraft speeds, and flight levels to destinations

that are constrained by the current ATM (Andrews and Hollister, 1997, Thomas and

Rantanen, 2006). Furthermore, the ADS-B system promotes shared situation awareness

by reducing interaction between pilot and air traffic controllers (Prinzo, 2003).

Providing cooperative conflict resolution is the main objective of the free-flight concept

and, more importantly, free-flight focuses on how to address the question of delegation

of responsibility for aircraft separation between pilots or ATCOs (Livack et al., 2000;

Erev, Barron and Roger, 2004). However, “right-of-way” rules (Erev et al., 2000) have

yet to be addressed.

of 303

Automated display systems, and not human issues, are the driving force behind the next

generation of air traffic systems (Funk, Mauro and Barshi, 2009). However, the use of

air traffic systems, such as the CDTI, to include monitoring requirements are excessive

and make situation awareness difficult to maintain over long periods in normal

operations. Accordingly, traffic search on CDTI is conducive to stress and detection

performance decreases over time. As a result, pilots may select a procedure below an

optimal level, leading to diminished performance (Funk, 2009). Indeed, Funk, Mauro

and Birdseye (2008) have emphasised that excessive use of CDTI may reduce pilots’

abilities to scan the environment for other air traffic for safe-separation. According to

these authors, issues of “trust” of automated systems, “team work” between pilots, and

pilot-ATCO “collaboration” are also a major concern.

2.4. Cockpit Automation

Whenever pilots fly an aircraft, flight safety is under the control of cockpit automation

which is believed to improve aircraft safety and reduce operational costs. For example,

flying has progressed from a non-automated “hands on” system to “hands off” control

(Funk et al., 1999). The newer generation of automated display systems usually

simplifies pilot tasks, or may even replace some of the pilot’s decision-making

activities.

There are numerous features in the cockpit, including automated display technology,

weather prediction systems, and voice communication. An automated display system

has become an integral part of complex systems, such as in aircraft design. It is

designed to cooperatively work with pilots to achieve stated objectives (Billings, 1991).

The objective of incorporating automation systems in the cockpit is to aid pilots in

performing tasks such as completing complex procedures within a short time

(Bainbridge, 1983). Bainbridge also notes that, from the designer’s perspective, human

operators are unreliable and inefficient and should be eliminated from the system.

Bainbridge further states that the designers leave it to human operators to complete the

tasks they formulate on how automated systems should work.

of 303

Cockpit tasks are performed through interactions between the pilot and the automated

system. Often these interactions lead to mode confusion, or surprises, thus keeping

pilots out of the control loop (Sarter, Woods and Billings, 1997). The interaction does

not usually make flight constraints visible to pilots. An example of such constraints

concerns safety (e.g. avoiding collision or weather conditions along the flight paths).

Pilots performing simple tasks, such as changing flight routes to avoid hazards, can be

hampered by difficulty in re-programming the Flight Management System (FMS). The

difficulty arises from the pilot requiring to key complicated sets of information into the

system. This cognitively demanding activity may lead the pilot to fail to correctly

program the system, thereby resulting in a track deviation. Importantly, such task

demands on pilots may be unevenly distributed over the flight time.

The use of automated display systems is effective during low mental workload phases

of flight however, in critical situations, automated display systems may prevent pilots

from interacting with the FMS, thus keeping the pilot out of the control loop (Endsley

and Kaber, 1997). To accommodate ATM’s request to change a flight trajectory (i.e., to

change altitude), the inherent flexibility in the FMS does not easily allow pilots to make

changes to cope with unexpected events. The flexibility usually enhances the pilot to

select the mode best suited to a particular flight situation. According to Sarter and

Woods (1995), the price of flexibility is valuable (i.e., the ability to be easily modified a

system): pilots must know precisely how and when to use which mode, configure each

mode to fly the aircraft, and monitor the active mode. These new cognitive demands are

easily increased during critical stages of a flight; stages in which pilots actually require

mechanisms to alleviate cognitive demands. Indeed, it has been argued that pilots spend

more time re-programming FMS in anticipating future events than manually flying the

aircraft (Lintern, Waite and Talleur, 1999)

Previous studies (Sarter and Woods, 1992; Degani and Kirik, 1995; Faerber, 1999) have

identified key interface issues associated with FMS. For example, the pilots’ interaction

with the system can be difficult when they complete data entry. The difficulties can be

higher when performed under a heavy mental workload, which may result in errors. The

information presented to pilots by the interface is not clearly understood, nor are the

working principles of FMS expressed thus making it difficult for pilots to understand.

of 303

Furthermore, there is a lack of consistency in the waypoints database. The findings also

reveal latent problems with FMS that affect pilot performance in emergencies.

As mentioned, it has been suggested that, with the introduction of automation display

systems, pilots spent less time flying the aircraft manually (Lintern, Waite and Talleur,

1999; Damos, John and Lyall, 2005). This creates a vacuum between the pilot and the

aircraft (Adams, Tenny, and Pew, 1995) and, therefore, pilots must be vigilant and

ready to take over control from automated display systems in emergencies, if necessary.

Bainbridge (1983) notes that to take over control from the automated display systems,

pilots must “wait”, “become aware of the current situation” (Sheridan, 2007), or

“reorient," themselves to the current situation (Endsley and Kiris, 1995).

Importantly, these studies did not state the time required to “wait” or “recall” after

disorientation, that is the pilot having to take over from the automatic control in an

emergency. However, this thesis argues that the “time” required is based on the

mapping of pilots’ mental models with the physical system in question. Past research

has documented accidents associated with the inability of pilots to take over control

from an automated display system. For example, in 1994, an aircraft crashed over

Japan. The findings from the investigation concluded that the pilot’s inability to take

over control from the automated system (Billings, 1997) was the result of inconsistency

with the pilot’s mental models (Sherry and Polson, 1999). Studies such as Sheridan’s

(2007) have in fact not made clearer the time required for operators to “wait”, “become

aware of the situation”, or “reorient” before manually taking over controls, or the time

required to process the required information.

Furthermore, the current source of difficulties in interacting with the system is that its

level of intelligence is not sufficient to support the continual, appropriate feedback that

naturally occurs among human operators (Norman, 1990; Hoc, 2007). According to

Parasuraman and Riley (1997), there are four ways pilots interact with a system:

Use—the choice to activate a specific system.

Misuse—a pilot’s failure to monitor a system occurs because of dependency on

the automation system. Flight instruments or indicators that are salient can draw

of 303

a pilot’s attention. Dependability and coherence are factors that influence misuse

of a system.

Disuse—underrating the capabilities of the systems. The user tends to ignore the

warning signs.

Abuse—designers and engineers often do not take pilots’ performance into

consideration.

Parasuraman and Riley state that to improve pilot and automation interactions, these

factors should be considered. Furthermore, an automated display system does not

always “fail” per se but often the system does not operate according to pilot

expectations.

An automated display system is often associated with a system’s problem, known as

“mode confusion”, that is pilots are occasionally confused with the system’s mode of

operations (Sarter, and Woods, 1995). A classic example of these issues is the

Bangalore air crash involving an Air India A320 in 1990. In this incident, pilots failed

to properly understand and handle the advice issued by the automated system and, as a

result, descended to below the safe glide path. With unreliable automation display

systems, pilots often do not consult them even though the information displayed is

“visible” to pilots. Moreover, the workload of pilots might change when confronted

with unexpected events, such as the need to change the aircraft’s heading by inputting

complex information, including the bank angle, altitude, speed and heading into the

system, within a short time.

A system such as FMS is designed to perform fast, accurate and complex calculations,

within a minimum time frame. The FMS therefore supports pilots with a large number

of functions and alternatives in flight tasks under different circumstances (Sarter and

Woods, 1995). However, the FMS was not designed to take into account pilots’

changing goals (Johnson et al., 2003) and, often, pilots do have multiple goals for a safe

flight, and these goals can change as the flight progresses. Flight planning is complex

and must be both economically and efficiently planned, as well as meet other

operational/commercial objectives, such as safety and costs, and to minimise the risk of

mid-air collision.

of 303

To optimise the flight, pilots must choose flight paths, maintain spatial separation and

consider environmental conditions with minimal intervention from ATCO. Changing

between goals might require planning to allow pilots to perform a set task. In order to

effectively switch between goals, pilots should be able to solve collision avoidance

problems, as they arise, through more clearly defined strategies, better conflict

resolution planning, clearer information about their needs, and more systematic

selection and implications procedures.

Several researchers have proposed different levels of automation depending on the role

humans play in human-machine interactions (e.g. Sheridan and Verplank, 1979; Kaber

and Endsley, 1997; Parasuranman, Sheridan and Wickens, 2000). These studies have

articulated function allocation for integrated systems design. Usually, the designer

decides which levels of automation are suitable for a given task requiring human-

machine interaction and these studies do not include a planning stage as part of their

model. According to Parasuraman, Sheridan, and Wickens’ (2000) model, the planning

process to change flight paths is conducted automatically by integrating information

from acquisition (stage one) and analysis (stage two). The decision automation systems

frequently supported decision makers by suggesting an acceptable decision or action

selection (stage three). This process was carried out saliently by the system in which the

“safest” path was explicitly recommended.

Muthard and Wickens (2001) have examined pilots’ planning task in space. Pilots were

asked to consider the space as a free-flight environment and the planning performance

task required them to consider all hazards. Weather information was most frequently

examined while terrain and traffic information were not usually considered. Pilots found

it difficult to monitor and evaluate the risks associated with these services because these

services are currently provided and managed by air traffic control. Similarly, Layton et

al. (1994) conducted a study to examine pilots’ difficulty in planning tasks and

confirmed that pilot planning processes may be supported by higher levels and stages of

automation when the solution space happens to be very large. Unsupported task

planning may impose a high cognitive workload. However, Layton et al., (1994)

suggest that, in addition to the system’s suggestions, it is important to design a system

that allows individuals to examine their own alternatives with “What if” scenarios, as

of 303

these individuals may differ in preferences and mental models in a given situation.

Furthermore, there is no objective way to recommend the right planning choice for each

situation. The findings of Layton and colleagues (1994) suggest that it may be possible

that the recommended path choices by automated systems are not compatible with a

pilots’ view of a situation but, any possible automated system’s recommended path

choices should be compatible with pilots’ mental models and preferences of the current

situation.

Horrey and Wickens (2001) conducted a resource allocation task study to examine how

participants will perform, supported by automation in an army battlefield simulation.

They demonstrated that while participant performances increased as the level of

automation is increased, the ability to remember highlighted features of the battlefield

decreased. However, when an automation failure was presented, only 50 per cent of the

participants were able to detect the automation failure. These results suggest that the

high reliability of automation systems often leads to an automation-induced

‘complacency’ problem (Parasuraman, Singh and Molloy, 1993). On the contrary, an

automation system is designed to be consistent and accurate in operations, reducing

users’ task demands and errors (Sarter et al., 1997). Parasuraman, et al. (1993) stated

that issues such as poor weather (e.g. turbulence, poor visibility) and physical or mental

weariness resulting from exertion, contribute to users’ overconfidence.

In contrast to FMS, pilots that are supported with a display that allows them to directly

manipulate the flight plan, combined with a preview of the effects of control action on

performance, experience reduced cognitive demand (van Marwijk et al., 2011). A study

by Van Dam, Mulder, and van Paassen (2008) argued that making these constraints

visible to the pilots is more important than using command structures. The Ecological

Interface Design of an airborne separation assistance system presented in their study is

based on the speed and heading representations (i.e., conflict representations). These

representations are designed to support pilots in airborne planning and strategically

executing manoeuvres that resolve conflict. Airborne planning refers to the cognitive

activity that a pilot directs towards expecting and predicting future events, with the aid

of a display, and handling changes in ATM operation (van Marwijk et al., 2011). The

study states two types of planning task; a spatial trajectory planning task allocated to the

of 303

pilot and a temporal trajectory planning task to automation to provide optimum flight

paths. According to van Marwijk et al., to cope with the future of ATM operation with

reduced cognitive demand, pilots should be supported with direct manipulation of

displayed information in relation to the re-planning of waypoints to avoid hazards.

As previously stated, though the Parasuraman et al.’s (2000) model has indicated that

planning tasks do place within the model, it is not clear whether the planning task is a

continuous process as the model does not provide feedback. Also, the attentional

resources are not shown or included in this model to support human information

processing. However, a similar model, presented by Wicken’s (1984) has included

attentional resources. According to Wicken’s model, attentional resources are

distributed or equally spread as a limiting factor across perception, decision making and

response selection, and working memory. Because this model is related to human

information processing only, this may suggest that the model is primarily concerned

with the spatial planning tasks of the operator (i.e., pilot). While Rasmussen’s decision

ladder may take into account both temporal planning and spatial planning tasks, as

indicated by van Marwijk et al. (2011), this thesis arguably illustrates that the

Rasmussen’s framework also fails to accommodate or shown attentional resources. For

example, in Wicken’s (1984) model, airborne planning (i.e., spatial planning tasks) may

support perception of collision (risk management); that is, a pilot can anticipate mid-

collision and how to cope with the situation rather than be surprised by it and therefore

will cope with mid-collision problems proactively rather than reactively.

Accordingly, in regards to displaying the Intruder movement on a collision avoidance

display, temporal planning tasks may bridge the gap between the Intruder’s current

heading and the future status of the Intruder. In addition, the displayed information

provides a means by which a plan may be monitored and changed, based upon up-to-

date conflict information. It can be argued that a planning task is a continuous process

(i.e., with system feedback) where airborne plans are refined and modified over time,

supported by attentional resources as a limiting factor. Therefore, a need exists to

provide pilots with a system which enables them to preview a task “on line” in order to

improve their decision-making. Thus, Wicken’s model of human information

processing should be modified to accommodate temporal planning task.

of 303

2.4.1 Mental Workload: Vigilance and performance

The philosophy behind an automation system design is primarily to reduce pilots’

mental workload and errors. However, to date, this effort has not decreased pilots’

mental workload, but simply changed its nature (Sarter et al., 1999) and, furthermore,

pilot strategies are subject to change when interacting with a new automated system. It

is therefore difficult to model task demands and mental workload from automated

systems to be used in a standard operational procedure. Loft et al. (2007) noted that it is

important to design and model a system with a strategy that addresses how both task

demand and mental workload can be controlled. This would give designers the

opportunity to analyse and evaluate different system combinations. For example, an

increase in pilots’ mental workload is observed when they are reprogramming Flight

Management Systems (FMS) during critical phases of flight (Sarter and Woods, 1992).

A change of flight path, or transitions from the cruise to approach phase of flight is a

classic example of such a problem. However, if the system was fully automated to

resolve this problem, the pilots would be placed in a passive or supervisory role

(Sheridan, 1999).

A number of formal definitions exist for the term ‘mental workload’. Young and

Stanton (2001, p. 507) state that “The mental workload of a task represents the

attentional resources required to meet both objective and subjective performance

criteria, which may be mediated by task demands, external support, and experience".

Similarly, Gopher and Donchin (1986, pp. 41-3) note that “... mental workload may be

viewed as the difference between the capacities of the information processing system

that are required for task performance to satisfy performance expectations and the

capacity available at any given time”. Lysaght et al. (1989, p. 27), on the other hand,

define mental workload as “... the relative capacity to respond, the emphasis is on

predicting what the operator will be able to accomplish in the future”. According to Loft

et al. (2007), a pilot’s mental workload is associated with the time pressure required to

carry out system changes and to make future projections. It is therefore critical that a

system’s modality allows pilots to cope with different types of mental workload.

of 303

A subjective measure of mental workload (NASA_Task Load Index-NASA_TLX) has

been used extensively to identify specific workload components that take advantage of

pilots’ information processing capacity by demanding their attention when performing

vigilance tasks (Wickens and Hollands, 2000).

Vigilance

Vigilance tasks require pilots to visually monitor displays for extended periods. These

tasks may change pilots’ performance if the frequency of system interactions decreases

their attentional resources over an extended period of time. With planning performance

tasks, however, it was found that they did not examine all hazards to include terrain and

traffic information because monitoring and evaluating the risks of these hazards in a

free-flight environment are the responsibilities of the current air traffic controls. This

suggests that if a system were to be implemented, it should be able to encourage them to

perform self-separation.

Modern cockpit displays contain a vast amount of information. Pilots must scan and

search regularly for relevant information with reaction time being crucial. A pilot’s task

in the cockpit is to cope with monitoring or detecting the system’s faults in a normal

situation, which may never occur (Adams, Tenny and Pew, 1995). However, as pilots

spend more than seven hours during long haul flights, monitoring cockpit systems

becomes increasingly difficult (Donald, 2008). For example, in the case of an accurately

programmed automated system, pilots with a low mental load may not monitor the

system adequately. These conditions may results in a low-level of vigilance (Warm and

Hancock, 1998). Vigilance tasks are related to an observer’s ability to maintain and

remain alert to stimuli over extended periods of time (Davies and Parasuraman, 1982).

Typically, performance starts to decline around 20-30 minutes after commencing a

vigilance task (Warm, Dember, and Hancock, 1996). Dynamic visual stimuli are

associated with greater performance decrements likely due to an additional ocular

accommodative demand. Indeed, when vigilance tasks are high difficulty, performance

will deteriorate rapidly and decrements in perceptual sensitivity are observable within

five to ten minutes of sustained attention (Nuechterlein, Parasuraman and Jiang, 1983).

of 303

Young and Stanton (2002) performed an investigation into vigilance task. The length of

vigil in their experimental design was approximately 10 minutes of the session. As

reported in their study, the length of vigil was only 30 minutes, which then cannot be

considered a vigilance task, as observed elsewhere (Warm, Dember, and Hancock,

1996). Both Bainbridge (1983) and Parasuraman (1987) have noted that it might not be

possible for pilots to remain vigilant while monitoring a system for more than thirty

minutes even though they are motivated. A lack of active interaction between the pilot

and the automated system effects the pilot’s monitoring and scanning skills. This

situation frequently results in loss of manual skills, a precondition for not coping with

emergencies successfully (Endsley and Kiris, 1995). As Endsley and Kiris further state,

a mode change, or inadequate information from the automated system, may keep the

pilot out of the control loop.

A vigilance task of monitoring automated systems failure, which is about 30 minutes in

duration, is usually shorter than the duration in a real world scenario (Molloy and

Parasuraman, 1996). In a highly salient environment such as in the cockpit, data-driven

events such as object movements are more likely to capture pilot attention than features

(Endsley, Bolte and Jones, 2003). According to Endsley et al. (2003), salience that is

data-driven causes distraction regardless of the pilot’s intentions to remain focused on

the current task. This problem is not due to lack of information or insufficient data, but

instead lies with the design of modern cockpits, which do not capture the pilot’s focal

visual attention, as a result of competition of information across the cockpit with flight

experiences. Indeed, pilots often miss important information displayed in the “green

box” (Sarter, 2000; Nikolic, Orr and Sarter, 2004) and, under crucial circumstances, it is

likely to require more than two different sources of information to alert the crew to

avoid a disastrous result (Sarter, 2000).

According to Warm, Dember and Hancock (1996), vigilance degradation is closely

related to: (a) important signals that are less noticeable to a pilot, (b) the lack of spatial

certainty of signals in relation to the locations, and (c) increased task demand in events.

Both pilots’ performance and the rate of the vigilance degradation are influenced by the

salience level of the signal being observed (Helton, and Warm, 2008).

of 303

There is not necessarily a loss of vigilance in a complex task environment (Moray and

Haudegond, 1998), however, in the case that such a state does exist, applying a warning

system that is not a continuous alert can be enough to improve vigilance in supervisory

control. In addition, boredom, fatigue, and motivation should be considered, when

designing a system (Young and Stanton, 2002). For example, in the event that users’

mental workload associated with the system is relatively low, errors may result, due to

loss of vigilance and boredom. A decrease in mental workload can lead to boredom,

loss of situation awareness and vigilance problems (Endsley, 1999). However, in the

event that the mental workload is relatively high, errors may result from users’ inability

to manage the critical task demands successfully (Young and Stanton, 2002).

One possible method to cope with vigilance problems is to artificially increase the rate

of the signal (Bainbridge, 1983). However, false alarms might cause unnecessary

distractions, thus making a pilot ignore important information outside the field of view

(Endsley et al., 2003). Indeed, Xiao and Seagull (1999) found that pilots tend not to

attend to warning signals, due to the fact that alert signals are combined with false

information from the surroundings. This can lead to true alerts being left unattended. It

was also stated by Huey and Wickens (1993) that the loss of detection is characterised

by a missed signal of relatively short duration, and that this gradual loss of signals over

a short period represents the start of a vigilance process. In addition, different alarm

thresholds may make it difficult for pilots to diagnose system failures.

Performance

Automated systems are designed to improve a pilot’s overall performance by reducing

mental workload (Sarter, and Woods, 1992; Endsley and Strauch, 1997). Pilot

performance and mental workload are inextricably linked and variations in mental

workload are associated with significant changes in performance. Hancock, Williams

and Manning (1995) have investigated this complex relationship between mental

workload and performance. They noted variations within pilots’ multi-tasking oriented

schemes and suggested that mental workload and performance are susceptible to

multiple characteristics of the task, and not just the immediate demand level of the task.

of 303

It is also been observed that a high mental workload lowers performance and sets an

upper limit handling capacity, which has an effect on the handling of emergencies

(Wickens, Mavor and McGee, 1997 ). For example, a pilot’s monitoring of tasks is

considered a secondary task. Thus, as the pilot’s mental workload increases, primary

tasks (e.g., maintain speed and altitude) may decrease during emergencies. In other

words, secondary task performance as indexed by transmitted information is associated

with different primary flight conditions (i.e., aircraft take-off, landing).

In contrast, Young and Stanton (2002) have proposed the ‘malleable attentional

resources theory of underload’ which states that there is a strong correlation between a

low mental workload (i.e., underload) and a decrease in attentional resources. This idea

has implications for pilots’ attentional capacity. For example, in situations of low

mental workload, pilots may not be able to reclaim control of the aircraft in an event of

automated system failures. This provides a practical prediction of pilot's performance

with automation systems.

To sum, previous research indicates that automated systems issues associated with

mental workload, vigilance and performance of pilots may significantly hinder the

implementation of the new air traffic management and cockpit technologies. For

example, the use of an automated system supports pilots in rerouting an aircraft’s flight

path to avoid obstacles and may be affected by changes in pilot mental workload, and

thus, performance.

The next section will examine the design approaches of an interactive system to support

users in decision-making processes for alternative routes, improve situation awareness

and reduced cognitive load. Thus, the section will present the definition of mental

models, some its relevant concepts and how mental models relates to the design of a

conflict avoidance system an effective tool to mitigate mid-air collision.

of 303

2.5. Conceptual Frameworks for Interaction Design

The previous section described computer-based technologies that were used to help

pilots cope with their mental workload demand in the physical world. To support a

pilot’s mental workload, situation awareness and decision making in a free-flight

environment, conceptual frameworks can be applied. In this section, two theoretical

approaches in relation to interaction design will be examined from the perspective of

cognition:

Mental models (Internal cognitions)

Conceptual models (external cognitions)

The ability or capacity to perform or act effectively in the physical world is influenced

and represented by mental models (Vosgerau, 2003). Mental models are internal

representations of reality that the user can use to understand the physical world (Greca

and Moreira, 2000). Norman (1983) had earlier suggested users’ mental models should

not be confused with conceptual models (i.e., designer’s mental models). According to

Norman (1983, p. 12), “Conceptual models are devised as tools for the understanding

or teaching of physical systems. Mental models are what people really have in their

heads and what guides their use of things.” This statement, in relation to the differences

between mental models and conceptual models, suggests that the system designer

designs a conceptual model into the physical system (“system image”) to create

something that is unambiguous and comprehensible to the user. However, a conceptual

model (i.e., the design model is a type of mental model) is a physical or environmental

model conveyed to the user, and often represents the information that a system designer

intends to communicate about the working principles of the target system in a way that

users can comprehend. Conceptual models can also inform the user, helping them to

dynamically construct a mental model (i.e., the user's model) of the target system.

Furthermore, the use of mental models as a mediator at the analysis level may enable

users, including pilots, to understand the working principles of the target system, such

as a conflict avoidance display (i.e., environment). Thus, in theory, the conceptual

model should be compatible with the mental model (i.e., in the mind of the user) of the

system.

of 303

A parallel issue is the role of the user's mental model in extracting information from

written material (i.e., text) using complex systems such as an abstract or written or

spoken language. Johnson-Laird (1989) stated that an abstract is in the mind of the

human and proposed that a reader constructs a mental model of written material being

read to explain, understand and/or simulate the processing of the “system” being

described. According to Johnson-Laird, problematic sentences in written material can

yield multiple competing mental models. However, written material that is not

problematic produces a single mental model that is easier to understand by the reader

(i.e., the material is only open to one interpretation).

The central premise of the Flight Collision Avoidance System (FCAS) is that the

system would be more effective if spatial constraints can be mapped into essential

collision avoidance constraints in an external representation in such a way that they

restrict the problem-solver in engaging different kinds of interpretation of the same

problem (e.g. reduce search in different directions). Thus, to reduce users’ cognitive

processing of information (i.e., search, recognition, inference), a direct manipulation

interface is considered, as suggested by Shneiderman (1982). For example, the

manipulation of a relative velocity vector via the flight yoke system will support a

clearer way of interacting with external representations to change aircraft heading.

2.5.1 Mental Models of Complex Systems

The first section of conceptual frameworks for interaction design reflects on human

limitations and their effect on the fidelity of mental models. The premise that humans

depend on mental models of an external reality of the world was first proposed by Craik

(1943). According to Craik, mental models can be considered as “small-scale models”

of external representations. Mental models have opened a new area of research within

cognitive science. Johnson-Laird (1983) used mental models to propose ways and

processes to explain how human reasoning can solve problems. Johnson-Laird states

that an abstract study of complex systems is in the mind of the human and that the

human may not have to process data with the interface. According to Johnson-Laird,

there are three processes of internal representations. First, mental models are structural

analogues of situations or processes.

of 303

Second, propositional representations are strings of symbols linked to each other by a

specific syntax. Third, mental images are pictures of mental models from a given

perspective. In general, humans may use all three different processes to interact with a

given system.

Gentner and Stevens (1983) state that studies on “Mental models” concern the

understanding of human knowledge about the world. Gentner and Stevens’ study

contains several causal analyses of how users understand the basic operation of a simple

physical device. According to these studies, the flow of information is from the device

to the user, creating a “mental model” and is not under the influence of the

environment. There are no dynamics involved. If the user changes the status of the

device, it will change and stay changed. The authors explored structural knowledge of

representations that is specific to the application domains. They examined, for example,

how mental models influence spatial knowledge, competence models, learning and

problem-solving. They state three characteristics of mental models implications for

users’ interaction with the physical world:

Domain research, focusing on:

i. A physical system (concrete, contextual) is usually in many conceptual

forms e.g. a conflict avoidance display

ii. Physical theories (of abstract objects, external, and contextual) are

usually in mathematical form e.g. algebra expressions.

The nature of the method, are knowledge representations based on signs and

symbols of what they represent or reveal in procedures and rules.

Theoretical approach is based on empirical studies and modelling to elicit,

analyse and represent users’ mental models.

With the advent of modern conflict avoidance systems, item number one (i) under

domain research should be the main focus of the current study in relation to mental

models. Further to this latter statement, Carroll and Olson (1987) state that users use

their mental models to know how a physical system is constructed and to explain the

system’s behaviour.

of 303

According to Carroll and Olson, a ‘model is a simplified version of a physical system, a

representation, or a surrogate system that can be descriptive or explanatory. Mental

models have predictive powers and limitations (for example, they can limit the way

users think and act, and incorrect models may lead to wrong decisions), and are subject

to change. Additionally, users' mental models are their detailed understanding of how

the physical system, such as a speed indicator, works, and why. These mental models

are also linked to the relationship between a system and the environment.

The terminology of “mental models” is widely used in the literature, but it is not well

defined (Moray, 1998). Moray pointed out that mental models are the mapping between

a task and its representations in a person’s mind. However, according to Rouse and

Morris (1985, p. 7) a mental model is defined as:

“Mechanisms whereby humans are able to generate descriptions of a system

purpose and form, explanations of system functioning and observed system states,

and predictions of future system states”.

Further to this, Rouse and Morris state that mental models are users’ explanations of

how a physical system should operate in the real environment. It is a representation of

why a system exists, how the system should perform, what the system is currently

doing, and the physical appearance of the system. Norman (1983a, p. 12) makes

“mental models” clearer by describing them in more detail: “Mental models are what

people have in their heads and what guides their use of things”. Accordingly, mental

models have the potential to help users to predict future events and to aid in the

diagnosis of a system’s problems; completing a task successfully depends on the state

of users’ mental models. Norman classifies the limitations of mental models into two

categories: First, user mental models are developed through interactions with target

systems and are constantly being modified. Second, users’ mental models are limited by

a lack of acceptable knowledge and understanding of the current problem state.

According to Norman earlier, when a user interacts with a system, he/she constructs a

mental model of that target system. Mental models that are acquired through the

interactions with the target system are constantly modified.

of 303

Mental models are an internal representation that may be flawed or inaccurate, durable,

limited, or accessible, but are still able to maintain the perceived structure of the

external system (Doyle and Ford, 1998). Another definition of mental models is

presented by van der Veer, and Melguizo, (2002, p. 60) “mental models are incomplete,

unstable, and unscientific, but they can be used for planning, execution, evaluation, and

interpretation of the system or problem the subject has to solve”. This definition is

adopted to support the design of FCAS. Pilots in a collision course should be able to

plan, evaluate and modify their flight paths to avoid a mid-air collision.

Young (1983, p 54) defines the term user’s conceptual model as ‘‘…a more or less

definite representation or metaphor that the user adopts to guide his actions and help

him interpret the device’s behaviour’’. From this definition, it can be assumed that the

user’s conceptual model (UCM) is similar to that of the user’s mental model or user’s

model as defined by Norman (1983).

Interacting with complex technology-based systems is an integral part of our daily

activities. These systems have both structural models (i.e., the complex composition of

knowledge of elements and their combinations), and functional models (i.e., how the

system operations and the consequences of using it) (Lee and Yoon, 2004). Lee and

Yoon state that a structural model, such as a state chart diagram, is well suited to

modelling the dynamic nature of a system, while the functional model, is an efficient

method for representing task procedures. According to Kieras and Bovair (1984), a

mental model of functionality is necessary for understanding the operation of a system,

for making a logical judgment on the basis of circumstantial evidence, and for problem-

solving. Young (1983) introduces a distinction between two types of user’s internalised

representations: structural models (i.e., surrogate models), and functional models (i.e.,

task-action mapping models). From structural models, surrogate models are

mechanistic models, in the sense that the models are highly simplified, and can take the

form of a scale model. The choice of the term surrogate implies that such a model can

be used in place of a larger system. Surrogate models are a useful, inexpensive way to

gain insight into the overall behaviour of the system. This may seem a desirable model

for users to hold. The drawback of this model is that for a complex system, it will take

considerable time and effort to acquire such a model.

of 303

So a surrogate is an optimisation; we would only expect to find this model in expert

users’ internal representation of complex systems. The second type of model is task-

action mapping models. Young linked these internalised representations of a system to a

real-world task. For example, in a calculator, the task-action model is a one-to-one

mapping between a task domain (i.e., addition of 2 and 4) and action domain (i.e., key

presses and display of results from the calculator). The task-action models provide an

understanding of how the user's mental model can directly interact with the real-world

(i.e., how to use a system, but not how the system works). Carroll and Olson (1987)

argued that this type of model does not explain the internal processes of the target

system to the user.

In contrast to other researchers, Wilson and Rutherford (1989) state the literature has

rejected the notion that all information (i.e., descriptive abstraction) should be organised

as a simple mental model. They further state that a system’s description needs separate

mental models with a particular form of abstract descriptions. This abstract description

should have the information on each state and function of the system. Doyle and Ford

(1998) also argued that to support decision-making, a mental model should be

compatible with the limitation of working memory (i.e., seven plus minus two “chunks”

of information). However, in the case when the information is organised in a

meaningful way, such limitations (i.e., erroneous mental models) may be overcome

with experience based on information learned (Coren andGirgus, 1980).Wilson and

Rutherford (1989) further pointed out that mental models are subject to changes in form

when a source of knowledge is acquired by means of observation, or experience is

introduced into them. According to Wilson and Rutherford, previous studies of mental

models have made assumptions without providing any empirical evidence of the

performance of users with mental models, as compared to those without a model. These

authors did not provide an explanation on how a mental model is constructed, on the

basis of individual performances; instead, their discussion was focused on mental

models in terms of the differences in knowledge and experiences associated with the

system behaviour. Furthermore, they pointed out that it is difficult to find in past

literature an example of a plain and comprehensible users’ mental model that was

applied in an interaction design.

of 303

This interaction depends on the current background knowledge of representations and

task requirements. Wilson and Rutherford also suggested that a mental model is not the

same as mental representation.

In contrast to Wilson and Rutherford (1989), Rasmussen (1990) makes a reference to

the mental representation of a physical system as a mental model. Rasmussen’s mental

representation as a physical system representation consists of five levels of an

abstraction hierarchy (AH), as part of a cognitive engineering approach. These levels

are comprised of physical form, physical functions, general functions, abstract function

and functional purpose. The AH has been used to analyse models of human-machine

systems design, and has been adopted mainly by specific work domains, such as the

nuclear power industry.

At the bottom of the abstraction hierarchy are the external representations of the

physical form such as speed indicator. Researchers use external representations as

descriptive models to analyse users’ knowledge, so that they only determine difficulties

in how the system works, or the type of information that is not clear to people (Carroll

and Olson, 1987). However, capturing an event through descriptive models requires the

observer to pay close attention to details. For example, an observer distracted from the

ongoing task can lead to human error. Moving up Rasmussen’s (1990) abstraction

hierarchy from the physical world are the physical functional representations that enable

users to interact with the physical world, such as a physical system. The physical

functional representations of the system are in the form of functional models, which are

structured representations of procedural knowledge of how the system should work and

operate (Young, 1983). An example of a physical system model is an aircraft “in-flight

entertainment system” with input and output. However, users’ functional models can

also be used to construct a similar system from existing knowledge, but are less suitable

for representing the structural models (i.e., internal working mechanisms) of a system

(Lee and Yoon, 2004).

Above Rasmussen’s (1990) physical functional level are the general functions and

activities describing the physical processes of the system. The physical processes enable

users to perform tasks at the abstract function level.

of 303

The general functions are in the form of structural models that consist of subsystems,

and causal relationships exist between these systems found at this level. At this level,

the users’ structural models define the knowledge about how a certain system should

work (Choi and Lee, 2011). The advantage of this model is that users’ procedural

knowledge of a system can aid in predicting the possible effect of any sequence of

actions taken by users. Constructing such a model in the minds of a user involves the

use of procedural knowledge. A structural model is used to describe the internal

workings of a device, which is then used to make predictions about its operation. A

common example of this is a pedestrian at a crossing with traffic signals that help the

pedestrian cross the road safely. Usually, when a pedestrian wants to cross the road

urgently, they will press the control buttons continually and wait for the lights to change

to “green”, assuming that such an action will cause the traffic light to change to green

faster. This incorrect assumption demonstrates unrealistic expectations, which are

incompatible with the traffic light system.

Above Rasmussen’s (1990) general functions of the abstraction hierarchy are the

functional structure representations that enable users to perform tasks, and how these

representations are influenced by external constraints. The representations of functional

structure are based on abstract functions, and relate to standard rules, procedures, or

laws, by which the main objective or purposes of the system can be achieved. This level

of abstraction hierarchy of mental representation can be considered as descriptive model

representations. Models represent what the users should know about the system with

which they are interacting. (Carroll and Olson, 1987). These sets of rules, procedures,

and examples are usually used as a training aid to learn about the system.

However, under these conditions, users’ mental representations are controlled by the

way they interact with the system. Fuchs-Frothnhofen et al (1996), as cited in Palmer

(1998), suggested that designers often assume how users should interact with the

system. The study argued that designing a system that conforms with individual users’

mental models is essential. For example, the same sets of rules can be applied to a

different system, and users’ mental models should allow them to reason and know the

current or future state of that system. The uppermost level of Rasmussen’s (1990)

abstraction hierarchy is the functional purpose model of a system.

of 303

This level is concerned with the reason why a system is built. For example, the purpose

of the FCAS being developed in this study is to provide pilots with a safe passage to

their destination by avoiding obstacles along their flight path. The detailed discussions

are presented in Chapter 3 in relation to the development of the FCAS.

The advances in automated systems have led to increases in the complexity of

engineering systems (Vicente, 1999). As a result, it has become more difficult for

designers to design a system that accommodates all the possible future situations within

such systems. These situations cannot be prevented from occurring in simulated

training, standard operational procedures, or further increases in an automated system.

With users’ inability to deal with the new and unfamiliar situations, safety is often

compromised (Vicente and Rasmussen, 1992). Accordingly to Vicente and Rasmussen,

Cognitive Work Analysis (CWA) is an approach to support a user in unforeseen

situations. In particular, Ecological Interface Design (EID, which will be discussed

more fully later in Section 2.8.4 and Chapter 3, supports the user by providing them

with the necessary instruments and information when dealing with unforeseen

situations. As a result, their mental workload is reduced, thereby supporting efficient

problem-solving (Vicente, 1999). This enables novice users, such as pilots, to easily

acquire advanced mental models, allowing them to become experts within a short

period of time (Burns and Hajdukiewicz, 2004). Therefore, by analysing how users

make errors, or have difficulties in interacting with the system, researchers will be able

to know how the user builds his or her mental models when interacting with it.

The above discussions have highlighted differing views on what a mental model is. This

illustrates how the human mind is incomplete, or not always consistent with perceptions

of the physical world. Therefore, consistencies in interaction between physical world

and human mind in results should not be expected. For the purposes of this study,

pilots’ mental models are an internal mechanism of their mind, used to support current

situations with reasons to access past situations or circumstances as they interact with

the environment.

of 303

2.5.1.1 Models of Mental Models with Automated Systems

In this section, the different types of mental models requiring facilitating human and

machine interactions will be discussed. Interacting with automation technology is

usually considered a cognitive process. The cognitive processes involved and the

cognitive limitations of users should be considered (Norman, 1983), and this

understanding used to inform system design. This information aims to identify and

explain the nature and causes of the problems users encounter. Accordingly, Carroll

and Olson (1987) presented three models users may use to interact with systems:

Metaphor models.

Network models.

Glass Box models.

2.5.1.2 Metaphor Models

An example of a metaphor commonly found in the literature is the typewriter model of

a text editor. This kind of model makes learning new systems easier, as metaphor

models are associated with a system’s familiarity. The model makes it easier for users

to learn, or build a target system model, but with inconsistent results (Carroll and Olson,

1987). However, improved performance with a new system can be achieved by using

this model, provided that the target system model and users’ mental models are

compatible. With this model, it is difficult to analyse users’ behaviour when interacting

with a system with similar symbols. A user’s mental model boundaries are subject to

change; therefore confusion arises when they are presented with two similar automated

display systems. This interaction often leads to cognitive interference.

2.5.1.3 Network Representation Models

The network representation model is a descriptive system. The system’s status is

represented in a graphical form that is visible via a display screen to users without

displaying details of how the system works (Kieras and Polson, 1983). The statement

suggests that graphical simulation can describe a system’s behaviour before it is fully

developed, and so system analysis can be performed using this model.

of 303

However, the model does not explain the internal process of the target system to users

as well as the surrogate models does (Carroll and Olson, 1987). The network models,

however, simplify the interaction of the working principles of the system with what the

users already know. Carroll and Olson further pointed out that the model is a useful

model for diagnosing system errors or failures, as use of this model makes it easier for

users to make appropriate decisions when such problems occur. Thus, it will impose a

minimum cognitive demand on users (Kieras and Polson, 1983). Furthermore, systems

that have complex representations will be difficult for users to learn and operate, so

simplifying this complex representation can reduce users’ high cognitive demands.

2.5.1.4 Glass Box Models

Glass box models are descriptive representations of a system (Carroll and Olson, 1987).

A glass box model is, as the name suggests, a model that is clear and transparent. A

better conceptual model requires a device to be transparent or visible to a problem-

solver. Glass box models can be useful to support users’ perspective of the target

system when they are interacting with a system, as the users’ interpretation of the target

systems’ representations are usually accurate. A glass box model is exactly the opposite

to that of a “black box” model. With a “black box” model, the users have no idea what

is going on inside the “black box”, and so to accurately interpret how the system works

can be difficult to achieve with only their conceptual knowledge. Here, compared to the

“black box” model, the glass box model can arguably be used to ensure the state of the

system is visible to the users. However, with complex information displayed, the

interaction between the black models and the glass box model could easily obscure the

visibility of the system’s state. The complexity will prevent the glass box model from

promoting the system’s transparency to users, and as a result, the glass box model can

only be transparent by name. The complexity acts as a shroud that keeps the system

from being transparent and is counter-intuitive.

Lawson (2006) investigates how people use their conceptual knowledge of an everyday

device without external representation. These devices are usually a simple and

transparent system, such as the bicycle. Lawson suggested that novice cyclists and

expert cyclists alike were not able to provide complete and accurate information about

of 303

bicycle functions. Their errors tend to cast doubt on the use of conceptual knowledge,

which is in contrast to the statement earlier of a glass box. However, the study

concluded that when people perform a task, they use their conceptual knowledge with

the availability of the system’s external representation. Therefore, the inaccurate

conceptual knowledge of the transparent system is being mitigated, thus allowing

mental models of the system to be created in working memory for problem-solving. The

possible explanation of these inconsistencies in the findings of a bicycle design

representation may be that the participants failed to pay attention to details; they may

have missed key points in relation to a specific bicycle function. Another possible

explanation is that the study may not have considered the differences in the participants’

conceptual knowledge of bicycle designs; people tend to interact with similar bicycle

designs every day.

Mostly, mental models are powerful because they determine how and what users pay

attention to, and what actions they will take to deal with current situations, which. it can

be argued, could include a change of flight path. Behind every pilot’s plan lies a random

fluctuation of their mental models, unconsciously shaping their decisions about which

issues are to be addressed and their desired outcomes. Because of the limitations of

users’ mental models, every airborne planning procedure must, at some point, be

explored and challenged under different circumstances.

2.5.1.5 Limitations of Pilot Mental Models

The limitations of users’ mental models can be attributed to mental models that are

relatively less complete, seriously constrained, and lack stability and to systems’

detailed display information that is not always easily remembered over extended

periods of time. Users’ mental models cannot be measured directly, and they can be

reluctant to change their behaviour under the same set of conditions even when

presented with alternatives due to variability in mental workload. They are

‘‘parsimonious’’, and users prefer to choose extra physical activities, instead of

reducing mental complexity. So, a pilot that has an accurate mental model of an existing

system that works, may find it difficult to learn a new system or use one in a new

environment (Carroll and Olson, 1987).

of 303

For example, users who have built their mental models of using the backspace button in

a text editor environment to undo typing errors may find it incompatible in a Web

Browser. In a Web Browser, the backspace can also be used to perform similar

functions as “to go back to the previously visited page”; however, the user’s mental

model may resist any change in this new environment. Similarly, a pilot’s “mental

models” will allow him or her to understand how different aircraft perform. Conflicting

mental models can arise as pilots’ mental models change from one model of aircraft to a

different model. For example, switching from a manual car to an automatic, or vice-

versa Or, from one with the indicator on the Left Hand Side (LHS) of the steering

column to one with it on the Right Hand Side (RHS). This type of users’ including

pilots’ behaviour often leads to disastrous results (Billings, 1997).

Previous studies, such as de Brito and Boy (1999) have shown that people's cognitive

processes have limitations that restrict their ability to continue with current tasks when

interrupted. These restrictions may adversely affect their activity at a critical time. For

example, while performing a procedural task before take-off, a pilot may make the

mistake of not completing his or her checklist accurately, if interrupted. According to de

Brito and Boy, when pilots are performing a checklist task, approximately 68 percent of

the pilots tend forget to complete the checklist when interrupted by ATC. Further,

interruption is also caused by a system failure, where 89 percent of pilots forget to

complete the checklist.

To counteract this problem, automated display systems are designed with visual and/or

auditory cues, mainly to remind or alert pilots of critical events (Sarter, 2000). Seagull

et al. (2000) argued that warning pilots of immediate dangers is not the purpose of

designing auditory or visual cues. As a result, these cues are not informative to pilots.

Most interruptions from these systems are not informative and therefore not suitable for

decision-making (Xiao and Seagull, 1999). Furthermore, designers usually do not

consider the significance of the time of the flight when these cues are presented (Xiao

and Seagull) For example; presumably pilots would want as few interruptions as

possible from top of descent to landing. Adamczyk and Bailey (2004) agreed, saying

that the timing of interruptions is important to avoid interruption overload.

of 303

The study also identified coarse, fine, breakpoints likely to influence interruption

overload, if not properly presented to pilots. The above discussion suggests that the

current systems are designed to convey signals without considering the state of the

pilot’s mental models and flight situations. Further to this, to build a mental model of

the flight environment, it is important to gather sufficient and useful data via flight

crews’ visual, auditory and tactile sensory systems as they scan the environment. The

pilots must direct their attention toward detecting important aspects of their

environment, both inside and outside the cockpit. It is an active process, knowing

where, how, when and why to look for a specific piece of information. Data, monitored

and gathered by pilots’ from inside or outside the cockpit is not noticed to be deficient

due to interruptions. Pilots can be interrupted unexpectedly by ATC communications to

attend to various kinds of activities such as requests to change course or altitude, and by

unexpected changes in aircraft state. McFarlane (2002) states that humans are prone to

make mistakes, misinterpretations and/or errors when interrupted, thus, leading to

significant safety problems.

Latorella (1998) conducted a study to find out the effects of interruption of ongoing

procedural tasks on cross modality (i.e., visual versus auditory cues) to determine

pilots’ error rate associated with each type of modality. Latorella reported that pilots

committed more “procedural performance” errors during visual tasks than when

interrupted audibly. Further to this, the study also states that:

o The overall pilots’ performance is a function of duration of flight.

o Pilots’ response time in auditory tasks is slower than visual tasks. Pilots’

performance decreases during auditory interruption while doing auditory

tasks (i.e., receiving data from radio altimeter via headset) than induced by

visual interruption while doing visual tasks.

The slower cognitive processing of questionable sentences in written material is one

field of study where the mental model concept is important to interaction design.

Interaction designers are, in particular, interested in measures of the acquired

knowledge of, skill in, and ease of use of, a system.

of 303

In this respect, the conception of mental models has applicability and interpretive power

as a tool that serves as a means for explaining and illustrating of how a system works.

Along this line, the use of mental models was widespread in both the human-computer

interaction and interaction design fields, as suggested by Norman (1988). Norman used

the term ‘mental models’ as a representation of how a system is designed and actualised

based on the designer's mental model. Comparable to the reader of the written material

previously discussed, the user establishes a mental model of how the system works,

supported by their interaction with the system. Specifically, the designer demonstrates

his/her mental model of a system design e.g. a computer system, as a medium of

communicating his/her mental model to the user, such as pilots. Therefore, according to

Vicente and Rasmussen (1992), the Ecological Interface Design approach within the

framework of Rasmussen’s (1990) cognitive control model (i.e., skill, rules and

knowledge based behaviour), will capture or isolate some of the central difficulties of

interaction design by supporting and promoting a single mental model of the systems.

2.5.2 Conceptual Models of Complex Systems

The second section of conceptual frameworks for interaction design deals with

designers’ conceptual models. Different definitions of conceptual models have been

proposed. For example, Wilson and Rutherford (1989) define designers’ conceptual

models in terms of the designer’s representation of users’ mental models. The user’s

conceptual model is the representation of the physical system while users’ mental

models describe the user’s internal representation of the physical system. Norman

(1986) stated that the design of an interactive system should be based on conceptual

models. Norman pointed out a conceptual model that is built by designers is usually

around a “system image” seen by users. This image is the physical structure, including

documentation and instructions, of the actual physical system. It should guide the

formation of the user’s mental model. That is, the mental model comes from the way

the user interprets the system image. Therefore, the designer must construct an

appropriate system image that is consistent with the design model.

According to Norman, if instructional manuals are compatible with the system's

“image”, then the user’s mental models will be accurate.

of 303

Therefore, to fulfil this requirement, for example, “Usability” is essential for conceptual

models. Usability, according to Norman, is important to ensure that the pilot’s mental

model matches that of the system’s model. Usability is defined by the clarity of the

interactions on which a system is designed and based. The clarification allows the users

to predict the system’s behaviour. If the system models are consistent with the users’

mental models, the users should be familiar with the functionality and appearance of the

physical system in question. Learning is made easier if a meaning is attached to the

information being displayed. The acquisition of skills and/or knowledge through

learning is accelerated with experience.

Stagger and Norcio (1993) argued that conceptual models of a system may not always

be consistent with the system’s “image”. However, while mental models are created as

users interact with a system, the same cannot be said for conceptual models. For

example, Greca and Moriera (2000) suggested that instructors and/or designers use

conceptual models in the form of mathematical models to communicate with users as

problem-solvers. Novice problem-solvers reason by constructing situation models and

diagrams to improve reasoning (Bauer and Johnson-Laird, 1993). The novices

manipulate their images of the diagrams to reach conclusions. According to Tversky

(2001), diagrams are most suitable for communicating structural models. Diagrams use

aspects like shape, colour and spatial relationships to communicate system features and

spatial or conceptual relationships, thus taking advantage of the user's experience in

interpreting the significance of things like spatial relationships.

With reference to Simon and Paige’s work, Larkin (1989) has also highlighted that

problem-solving is commonly performed with the support of external representations as

a valuable or useful resource. Frequently, the problem-solver may construct

mathematical equations and diagrams to help solve the problem. Similarly, Simon and

Paige (1966) note that problem-solving of mathematical representations (i.e., algebra

expressions) is more effective for participants who draw diagrams6 essential to the

solution. This suggests that diagrams can greatly shorten the time to examine and detect

physical objects, such as aircraft on a collision course, and eases mental simulations.

6 Diagrams refer to vector representations.

of 303

Larkin and Simon (1987) further Simon and Paige’s ideas, demonstrating that some

problem-solving with external representations or good diagrams has diagrammatic

external representation advantages. First, diagrams assist the process of the reasoning

involved in drawing a conclusion, and that perceptual information of the environment

can be improved dramatically with little cognitive effort. For example, with spatial

relationships, the location of objects in the three-dimensional external world do not

have to be described, as they are immediately apparent. Second, the problem-solver

may be able to substitute the rapid processing capabilities of the human visual system’s

perceptual judgment (e.g. whether two circles of the protective zone around the Intruder

and Ownership intersect) for more difficult deductive inferences. The effect of diagrams

on the process is the first practical application of mental models (Bauer and Johnson-

Laird, 1993). Therefore, a user’s (such as a pilot’s) accurate mental models require the

correct interpretation of information, the ability to bring them up to date, maintaining a

flow of information to further improve the quality of the models and accepting or

rejecting any particular model.

Cox (1999) specifies numerous advantages of using external representation construction

to assist problem-solving and processes. For example, external representations make

full use of both the verbal content and visio-spatial data components of working

memory load. External representations support the problem-solver in keeping track of

progress through the current problem state. The importance is to keep the problem-

solver in the control loop. Cox further suggested that diagrams encourage organising

information spatially with “spatial mental models”. Accordingly, task performances that

are promoted by external representations may reduce the cognitive workload by

avoiding cognitive restructuring of problems, thereby reducing the cognitive effort or

time in solving problems. This enhances the impression that the user is performing the

task, and is in control; not that the computer is responding to requests while the user

waits patiently, wondering if the computer is executing the request correctly. Novices

can learn the basic functionality quickly while intermediate users can retain operational

concepts over time, and error messages become rarely needed. Van Dam, Mulder, and

van Paassen (2008) argued that it is more important to make flight constraints visible

via ecological display systems to pilots than using computer command and control

structures.

of 303

The interaction between pilots and automated systems usually depends on a

conversation metaphor (e.g. a command driven interface), which is relatively indirect

and abstract, and will require the interactions to be processed in working memory.

Predominantly, most of the display-based problem-solving models consider analytical

reasoning as ‘in users’ heads’ to successfully solve problems, as compared to the use of

a screen in the cockpit. In particular, the problem-solver can observe many of the

essential features of the screen. Therefore, if the problem-solvers are not familiar with a

new situation, they can always use external representations to build up a picture of the

current problem state. Further to this, Larkin (1989) suggested that there is an important

interaction between the problem-solver’s internal problem-solving processes and the

external representations. An effective display is designed to show all the necessary

information of the current problem state, and makes the information visible to the

problem-solver. However, failure to represent essential information in a simple way,

which shows all the characteristics or features that the problem-solver would expect

from the external representation, will typically lead to problem-solving errors.

At this point it seems useful to state that the information provided by Simon and Paige

(1966), Norman (1986), Carroll and Olson (1987), Larkin and Simon (1987), and Cox

(1999) all encourage the use of external representations to support the development of

Flight Collision Avoidance System. A task may have an implication on how

information is represented on the external representation, especially with direct

manipulation. Actions on the interface may be represented visually and be incremental,

immediate, and reversible, to give a user the impression of acting directly in an

environment. Such direct manipulation of the relative velocity vector may require little

or no cognitive workload to promote skilled-based behaviour (Rasmussen, 1983).

Accordingly, with a direct manipulation interface, interactions are possessed by at least

three main characteristics (Shneiderman, 1982). These are include 1) Continuous

representation of objects and operations of interest; a visual representation of objects is

presented to users, in contrast to traditional command driven interfaces. 2) Physical

actions, instead of long sentences with deeply nested dependent clause structures; for

example, with direct manipulation, pilots can avoid a collision by manipulating the yoke

steering wheel and/or rudder pedals to change flight path, as compared to using

of 303

command structures with the goal to change aircraft heading. 3) Rapid reversible

operations in increments, whose impact on the object of interest is immediately visible,

for example, when pilots make a mistake such as by turning the yoke steering wheel too

sharply, they can rapidly recognise this situation and perform a corrective measure.

Further to this, pilots can immediately see in the event that their corrective courses of

actions are furthering their goals.

In conclusion, to contextualise the design approach taken in developing the Ecological

Interface Design system, systems designers may use system diagrams to plan the flow

of information that may not be immediately obvious hierarchically or comprehensible.

For example, at the physical level (i.e., at the lower level), control flight networks are

used to control aircraft performance and safety. In the case of diagrams of complex

systems, vectors can be used to demonstrate temporal sequences and the structure of the

operation of the elements to accomplish the specific goal. Furthermore, pictures or

diagrams help build mental models (Glenberg and Langston, 1992) to support situation

awareness.

The next section discusses situation awareness framework in combinations of

automated display systems, modality, mental models and environments to create pilots

situation awareness and reduce their mental workload. To complete tasks successfully

in a complex and dynamic environment, improved situation awareness is necessary.

Further to this, the section discusses the background knowledge and the implications of

losing situation awareness.

2.6. Situation Awareness

In the previous sections, issues regarding airspace, pilots’ mental workload, mental

models, and problems induced by automated display systems were discussed. This

section elaborates on a cognitive mechanism through which pilots’ situation awareness

is acquired, maintained, and then lost. These problems are often attributed to the

complexity of a system and poorly designed displays, with poor mental models keeping

the pilot out of the control loop.

of 303

There has been a growing interest in the aviation community as to how pilots maintain

awareness of complex and dynamic events. This increased interest has been mainly due

to readily available sensors in a modern aircraft cockpit, in conjunction with the pilot’s

“new” role as a supervisor (Sheridan, 1999). Different definitions of situation awareness

have been proposed. Situation awareness can be viewed in terms of both the product

and process (Adams, Tenney and Pew, 1995). The product (i.e., perceptual and

cognitive activities) is involved in reconsidering and changing the state of situation

awareness whereas the process is related to the availability of information and

knowledge. Alternatively, situation awareness can be considered as “adaption”, in that it

does not reside permanently, neither in the physical world, nor does it reside in people’s

minds. Rather, situation awareness resides in people’s behaviour and is consciously

controlled by the interaction with the environmental constraints (Smith and Hancock,

1995). This definition suggests that situation awareness is goal-oriented behaviour that

is achieved through both a pilot’s knowledge and behaviour under current

circumstances and environmental constraints. As such, this definition is mainly

concerned with how strong the interaction is between the individual and the physical

world. According to Bedney and Meister (1999), situation awareness provides the

individual with a dynamic orientation of a situation, the opportunity to reflect, not only

on the past, present and future, but also with the potential features of the situation. The

individuals’ dynamic reflection contains a logical-conceptual, imaginative, conscious

and unconscious component, required to develop mental models of external events.

Their definition focuses on the individuals’ dynamic reflection, with relation to mental

models, to understand the working principles of physical systems. In contrast, Endsley’s

(1988) definition of situation awareness pays special attention to individual perception

and understanding of the physical world, and future projections of this understanding. It

is formally defined as: "the [1] perception of the elements in the environment within a

volume of time and space, the [2] comprehension of their meaning, and the [3]

projection of their status in the near future.” This definition of situation awareness, as

presented by Endsley, is universally accepted.

Situation awareness (SA) is also regarded as a function of automated display systems to

provide the necessary information, and the modality in which the information is being

presented (Endsley, 1995a).

of 303

Environmental pressure, such as stress, cognitive workload, system complexity, and

type of information represented, may affect a pilot’s situation awareness. Thus,

Endsley’s three-level model of situation awareness includes identifying critical

elements within the environment (Level 1), interpreting and making sense of what those

relevant elements mean in relation to the pilot’s goals (Level 2) and, at the highest level,

pilots should be able to project the status of those elements into the near future (Level

3). These higher levels of situation awareness support the pilots to function in a timely

and practical manner. The three-level model of situation awareness tends to be

supported by more general models of human cognitive information processing

(Rasmussen, 1986) and provides an indication of which types of information might be

obtained from individuals when investigating problems associated with situation

awareness.

Pilots’ perception is commonly used to extract critical flight information from the

cockpit or the environment for task execution (Level 1). This information is obtained

from different locations under a specific set of conditions, characteristics, and dynamics

of essential elements, such as physical objects. However, pilots’ interactions and

responses are usually based on the perceived information through visual and auditory

cues. To enable pilots to receive a full and accurate “picture” of the environment, it is

necessary to integrate and process all the information acquired from the sensory

systems. Endsley (1995a) notes that several factors have proven to be important in

influencing the processes of acquiring and maintaining situation awareness. For

example, pilots’ cognitive abilities may differ in their ability to acquire situation

awareness. This may be influenced by the ability, experience, and training requirements

of the pilots. Furthermore, pilots may have certain goals and objectives that may

influence perception and interpretation of information acquired from the environment.

Perceptual modalities, such as visual and auditory modalities have been investigated in

detail (Hameed et al., 2009) and there is a long tradition in making use of auditory and

visual displays to present warnings and alerts in emergencies. The visual and auditory

display systems currently present warnings or alert information to pilots. In earlier

designs of lighter aircraft, for example, there was no instrument for measuring and

displaying crucial information, such as stall warning.

of 303

Because this information is critical to flight, a stall warning horn was installed to

provide a sound that is clearly audible to improve pilots’ situation awareness. However,

the sense of touch has generally been neglected, until recently. Advanced technology

allows tactile applications, such as a stall warning system, to capture a pilot's attention

through a tactile device such as a “stick shaker”, to replace the missing cues without

degrading performance on concurrent or secondary tasks (Nikolic, Sklar and Sarter,

1998; Sarter, 2006). There is less interest in tactile modality than visual and auditory

modality because both visual and auditory technology do not require special equipment

(i.e., they do not require any wearable devices) to convey any information, as compared

to tactile modality (Verrilo, 1993). However, tactile modality can be used to

compensate for a lack of auditory or visual sensory input. Woods (1995) suggested that

these modalities could be used to represent the dynamic behaviour of a system. The

current automated display systems are yet to stimulate all of the human senses.

The second level (Level 2) of Endsley’s (1995a) situation awareness is to understand the

current situation. To achieve this, pilots need to filter out confusing, disordered or

unstructured information perceived from the environment. For example, at this level,

pilots can recognise or identify an Intruder on a collision course. With the information

acquired from the environment, pilots should be able to understand and assign meaning

to the perceived information, integrate the perceived information to form patterns with

other elements within the environment and create the overall picture of the

environment, including understanding the importance of the perceived information of

the current events.

At the highest level (Level 3) of Endsley’s situation awareness is the ability to project

the current events into the future. At this level, for example, time estimation is crucial

for collision avoidance. Pilots have the ability to project the future location of an

Intruder within the environment, at least in the short term. The pilot’s accurate mental

models will decide when to manoeuvre or to diagnose the problem. Completing this

task successfully will depend on the state of the pilot’s mental models on the future

state of the Intruders position. Abstract data can be visualised with a computer-

supported display to help achieve situational awareness. Preferred formats are diagrams,

images, and movable objects.

of 303

Endsley (2001) has suggested principles associated with situation awareness oriented

design that transforms data into information in a timely manner. One of the

recommended design principles is to incorporate a direct presentation of understanding

and projection of the system’s future states, instead of presenting only low-level data

structures requiring operators, including pilots, to cognitively integrate and interpret

data. Currently, automated display systems can display vast volumes of data, more than

the pilot can handle. Endsley proposes that to present these data in a way that is usable

to pilots, there is a need to integrate data into a single display. However, others have

argued that it is not a matter of integrating large volumes of data into a single display to

provide better situation awareness (Rupert, 2000). For example, aircraft accidents still

occur even when equipped with the simplest device, such as the aircraft’s artificial

horizon which contains less available data, as compared with the primary flight display.

Endsley (2000) also suggested that integrating all information into one manageable

display minimises cockpit scanning, thereby providing, or creating, better situation

awareness. For example, an aircraft’s integrated display, such as the Electronic Flight

Instrument System (EFIS), has multiple pages that pilots must scan through to recover

relevant information. In emergencies, locating relevant procedures or information may

take considerable time. Pilots who navigate through multiple pages seeking different

information might find themselves using a wrong page (Bainbridge, 1983). The wrong

page might be as a result of the way the information is structured or presented to pilots.

This increases pilot workload and the decision-making process.

However, others have argued the design of the system may be part of the problem, and

not part of the solution, to provide situation awareness (Staggers and Norcio, 1993;

Rupert, 2000). Rupert (2000) states that designing complex displays that integrate all

relevant data would not solve pilot errors or improve situation awareness but, rather,

will increase mental workload and reduce situation awareness. Accordingly, Rupert has

suggested that a system must be developed which can presents information in a

different manner. The use of a tactile modality may therefore support pilots to improve

situation awareness.

of 303

2.6.1 System Awareness

The monitoring of automated display systems is necessary in a wide range of domains.

Monitoring aircraft flight mode, as used here, is a process of acquiring systems

awareness. Wickens (2002a) states that, in a domain such as aviation, aircraft automated

display systems are complex and dynamic. The rationale behind the use of automated

systems is straight forward; to reduce pilot’s workload. Designers often present

solutions to reduce pilot’s mental workload by allocating tasks to automated systems

such as the used of an autopilot (i.e., maintaining aircraft altitude and heading).

However, due to the complexity of these systems, and poor design, maintaining systems

awareness has proven difficult (Staggers and Norcio, 1993; Sarter and Woods, 1995).

In addition, human operators remember activities that are self-initiated and not those

initiated by an automated display system (Wickens, 2002a). This situation challenges

pilots to maintain system awareness in the event that activities are generated by the

automated systems. However, to sustain awareness and maintain control of the aircraft

through a variety of flight conditions, while also being aware of the action taken by the

automated system, is a challenge for pilots (Sarter, Woods, and Billings, 1997). A

pilot’s mental workload is likely to increase when attempting to maintain automated

display systems awareness, as well as vigilance (Wickens, 2002a).

The automated system may have many interacting components, and each one of those

components has its own set of mode operations. These sets of modes may behave

differently under certain flight conditions thus leading to automation surprises, such as

an autopilot failing to capture an altitude from a commanded change by the pilot

(Sarter, Woods, and Billings, 1997). Automation surprises occur because the automated

system takes actions that pilots have not expected, or failed to perform specific tasks as

instructed (Endsley and Kiris, 1995), and as a result this lowers a pilot's performance.

Inappropriate or inadequate information from automation systems can also result in

mode confusion (Norman, 1990). For example, a current mode of operation could

mislead pilots to operate below the required thrust level. This mode confusion is

reported as the primary cause of the Air France Airbus A-320, which crashed in

Habersheim-Mullhouse Airport, France (Commission of Inquiry, 1990).

of 303

According to Sheridan and Parasuraman (1995), maintaining mode awareness can be

achieved by allocating attentional resources proportionally to two or more automated

displays which provide the same information. A breakdown in interaction with one of

the systems can lead to loss of mode awareness, and thus mode errors (Sarter, and

Woods, 1995; Sarter, Mumaw and Wickens, 2007), as a result of inconsistences in the

way the information was presented to pilots. These studies suggest that a loss of mode

awareness occurs when a user has an input that is contrary to the system’s status, which

leads to unexpected results of a user’s actions. A pilot’s ability to interact with

automated display systems can lead to difficulties in predicting aircraft behaviour

(Rudisill, 1995). For example, a user may press a key of an unresponsive system,

expecting the system to carry out instructions as commanded. According to Rudisill,

pilots have reported that their workload increases with these systems when trying to

understand and anticipate the aircraft behaviour.

In the supervisory control of a process, a pilot interacts with a computer to obtain

information and issue commands. The artificial intelligence, through artificial sensors,

perceives their environment and actuators, and executes these commands to control the

process (Sheridan, 1983). Past research has shown that aircraft flown with advanced

automated features pose supervisory control problems. Supervisory control problems

are attributed to a loss of mode awareness (Sarter and Woods, 1995; Rushby, 2002),

resulting in many aircraft incidents and accidents (Sarter and Woods, 1995). Losses of

mode awareness are attributed, in the literature, to inadequate mental models of the pilot

or an incompatibility between the mental model and the automated display system.

Thus, it could be argued that pilots have failed to keep up with the pace of the mode

progression of the automated system. To keep up with a system status, pilots need to

analyse the information presented, understand it, and carry out changes according to the

aircraft’s current status (Singer and Dekker, 2000). Pilots become confused about the

automation system when the status of two modes are incompatible and wrongly interact

with the system (Sarter, and Woods, 1995). Mode confusion in an automated system

imposes greater demands on the pilots, increasing the risk of mode confusion.

of 303

A number of factors that have a significant impact on the monitoring strategy of

operators under a certain set of conditions have been identified by Moray (1986). For

example, examining flight instrument information to detect a system’s critical status is

important. As the flight progresses, the pilot is hopefully fully familiar with the

system’s mode, and the familiarity may place the pilot in a passive, supervisory role

(Wickens and Kessel, 1980). As a result, detecting system failures, faults, or mode

changes at an inappropriate time may challenge pilots’ cognitive abilities and

limitations. Lack of physical or mental activities in performing tasks, such as radio

communications, coordination, decision making, and manipulation of manual controls

have also placed pilots in this supervisory role (Parasuraman, Singh and Molloy, 1993).

In addition, the problem with the passive monitoring is that an automated display

system does not state how or when communication with the operator should take place.

For example, adaptive control research that monitors the pilot’s performance for a

period before communication as to whether to engage or disengage cruise control, has

not been investigated. A pilot’s infrequent monitoring of Flight Mode Annunciations

(FMA) results in failure to cross check the selected mode (e.g. auto throttle or autopilot

in an active mode) or the status of the automated display system (e.g. auto throttle or

autopilot disconnected) (Sarter, 2008). These factors have a considerable impact on

aircraft performance when the information displayed is visually not salient enough to

draw a pilot’s attention. Wickens and Kessel (1980) suggest that, compared to a visual

signal, an auditory signal that is underutilised in such conditions could be used to

communicate mode-related information to the pilot.

2.6.2 Spatial Awareness: Mental Models

A pilot acquires spatial awareness of the physical environment by flying the aircraft

through the environment (Richardson, Montello and Hegarty, 1999). As aircraft move

through space, unknown or unpredictable events, such as the Intruder occupying the

Ownship flight path may challenge pilots’ spatial awareness. The concept of spatial

awareness is an essential quality in representing the moving aircraft through a three

dimensional space with obstacles on its flight path (Wickens, 2002a). Richardson,

Montello and Hegarty (1999) suggested that the process of moving through space can

lead to gaining navigational knowledge.

of 303

The knowledge can either be of a route (i.e., this procedural knowledge is an egocentric

(“inside out”) view of the airspace) or map knowledge (i.e., an exocentric (“outside in”)

view of the airspace) to support different pilot behaviours. The route knowledge is

characterised and supported by a sequence of spatial locations.

To promote pilots’ spatial awareness, understanding the use of two-dimensional (2D)

display formats to present attitude information in respect to pilots is important

(Wickens, 1995; Pevic and Ercoline, 1999). A pilots mental workload is increased when

combining both lateral and vertical manoeuvres on a 2D display (Alexander and

Wickens, 2002). Alexander and Wickens suggested that lateral manoeuvres in three-

dimensional (3D) displays degrade certain information, such as line-of-sight of objects.

The authors also noted that the 2D coplanar display is best suited to support traffic

awareness, as compared to a 3D display. However, the 3D display format supports

flight path control (Haskell and Wickens, 1993). For the 3D display, the mental

workload of pilots increases with mental rotation of vectors between the Ownship and

Intruder, to gain more precise resolution (Alexander and Wickens, 2002). Alexander

and Wickens concluded that the choice between the two types of design will depend on

whether air safety is of utmost importance. For example, they suggested that a pilot’s

choice to use a coplanar display with side-view is related to cost variations (i.e., fuel

saving, or when the descending manoeuvre is considered, airspeed naturally increases),

while the rear-view display is usually associated with time efficiency (i.e., airspeed

naturally does not increase or decrease during conflict manoeuvres).

The choice between these formats may also depend on both the type of operational tasks

and the type of users. More importantly, how information is being displayed to pilots,

that is as moving-horizon “inside-out” versus moving-aircraft “outside in” orientation

(the “frame-of-reference”), should be considered (Wickens, 2002a). The development in

these areas has encouraged the design of airborne obstacles with situation awareness to

include Horizontal-Situation Display and Cockpit Display Traffic Information (CDTI).

Both of these displays more dynamically support pilots with the navigational

information required to fly the anticipated flight path support than the conventional

basic flying instruments.

of 303

Spatial awareness may depend primarily on whether navigational information should be

displayed in egocentric-referenced terms (“the aircraft is at 2:00 position”) versus

exocentric-referenced terms (“the aircraft is 270 degrees (due west) from the Ownship’s

current position”). An egocentric display is characterised by rotating objects in space

with the Ownship in a fixed position and the heading in the north up orientation

(Wickens, 1995). As such, the egocentric format displays positions of the surrounding

aircraft as moving figures (i.e., map rotation), relative to Ownship. Problems may arise

when pilots need to avoid obstacles to get from their current state to a desired state.

A display that supports pilots with one-to-one matching of traffic information, such as

the position of the Intruder relative to the Ownship, is more desirable than decoding

information about the relative locations when making a judgement of relative positions

and the line-of-sight of distant objects (Thorndyke and Hayes-Roth, 1982).

Furthermore, a display that supports both guidance and spatial awareness should be

closely related to conventional navigational displays (Alexander, Wickens and Merwin,

2005). To capture pilot's spatial awareness of avoiding obstacles in the real world, the

(2D) display may be configured as an egocentric-referenced collision avoidance system.

2.6.3 Loss of Situation Awareness

The processing of a large amount of data often requires multi-tasking which causes a

load on attentional resources. In general, human operators, including pilots, are not well

suited to multi-tasking, and situation awareness may become compromised (Snook,

2000). Loss of situation awareness is considered to be a major contributor to aircraft

accidents in civil and military mishaps (Endsley, 1999; Stanton, Chambers and Piggo,

2001). Flight management systems allow pilots to program and perform complex

navigational tasks (e.g. change flight path, flight levels, and airspeeds). The system

silently carries out these tasks as programmed, but sometimes with “automation induced

surprises”. These “automation induced surprises” may be as a result of the pilot’s poor

mental models (i.e., lack of detailed information on how the system actually works to

achieve a pilot’s goals), poorly integrated displays systems (i.e., information is spread

across the cockpit), system complexity (i.e., provide a number of options to achieve

those goals), or due to the pilot being relegated to a supervisory role (i.e., keep the pilot

of 303

out of the control loop). These issues often lead to loss of situation awareness (Wickens,

1995). Loss of situation awareness may lead to an inappropriate decision, in spite of the

fact that the information required to support an appropriate choice of action is available

in the environment (Endsley, 2006). Endsley further argues that human error is not due

to a failure in decision making, but it is due to poor situation awareness. Indeed,

situation awareness is an important factor for effective decision making, especially in a

dynamic environment (Jones and Endsley, 1996).

Therefore, understanding pilot errors that could occur in this dynamic environment is

crucial for safety. Endsley (2006) stated that 88% of the aircraft accidents in

commercial aviation that are due to human error were attributed to loss of situation

awareness. Three levels of pilot errors due to loss of situation awareness have been

identified (Endsley, 1995b). Level 1 SA errors (76.3%) occur when relevant data is not

available, or is difficult to distinguish or to detect. This is due to a lack of monitoring or

observation of information presented, or misperceived because of memory loss. Level 2

SA errors (20.3%) involve inaccurate mental models, use of incorrect mental models,

over-reliance on default values, and many other factors and Level 3 SA errors (3.4%)

are associated with failure to predict future events.

In summary, a number of definitions of situation awareness have been proposed.

Bedney and Meister (1999) describe the nature of the individual situation awareness as

reflective, with the support of mental models that have the operational features to

understand the physical system (i.e., problem space). Smith and Hancock (1998),

however, state situation awareness primarily in the context of the interaction between

the individual and the physical world. By contrast, Adams, Tenney, and Pew (1995)

argue that situation awareness is a process or a product. The proposed free-flight

environment, in which pilots are responsible for maintaining safe separation, when

implemented, will impose a heavy demand on situation awareness. This is due to a more

complex “envelope” of safe separation around each aircraft, in terms of minimum time

to contact, rather than the fixed distances of separation that exist currently under the

ATM (Wickens, 1995). Because of the limited airspace at a given flight level, time to

contact becomes more complex to visualise these envelopes around Intruders in a close

proximity.

of 303

This challenges both the pilot’s and the controller’s ability to maintain situation

awareness. This section also described how spatial awareness involves an understanding

of a location in space, with specific information within the environment. That is, spatial

awareness is an environmental awareness for navigation.

This idea of using egocentric format displays (i.e., map rotation) will be implemented to

improve situation awareness with minimal costs that may require pilots to engage,

including time and effort-consuming mental rotation, which is elaborated on in Chapter

3 of this thesis. System awareness refers to the understanding of an object’s status in

specific terms, for example, the engine indicator for oil level.

Crucially, studies on mental models and situation awareness will continue to play a key

role in our understanding of the process of trying to acquire or develop a quality or skill.

The mental models’ relative importance accommodates a psychological interpretation

of conceptual models, as well as a means of gaining access to the relationship between

situation awareness and mental models.

To conclude a number of specific implications can be draw in relation to mental

models. First, situation awareness is obtained by using the sensory systems to monitor

or scan the environment and compare the results with mental models. Thus, the extent

of the relationship between the situation awareness and conceptual models will

determine the extent of acquiring or developing a quality or skill from the pilot’s

monitoring of a physical system. Second, complex system representations would

significantly influence perceptions of situations and the time frame required to control

the system. Third, the affordance of situation awareness and mental models from

memory recall (training, knowledge, experiences) will determine the extent to which

pilots monitoring or searching influences perceptions of aircraft conflict on a collision

course or movement. Thus, mental models that are more feasible will have a substantial

influence on the perception of conflict scenarios. Fourth, conceptual models take time to

develop (i.e., mapping) in the mind of the pilot. The longer a pilot spends on the

monitoring of conflict displays, or of the particular contents of displays, the stronger the

mapping effects will be on memory. However, the mental models approach, including

situation awareness and conceptual models, will become increasingly important with

our understanding of situations in relation to decision making.

of 303

The next section draws together discussions regarding mental models, mental workload

(i.e., cognitive resources), and situation awareness to support pilot decision-making.

Decision-making models are presented as an active decision-making process guided, in

part, by the pilot's mental models. As a result, decision-making may be influenced by

the availability of cognitive resources. The cognitive resources allocated to tasks should

support the development of improved situation awareness, thus improving the decision-

making process. In some cases, displays may impose their own constraints and

limitations (e.g. navigating through many pages) and, in fact, co-determine the nature of

the task.

2.7. Decision-making

Decision-making is of fundamental importance to pilot activities (Jenkins et al., 2008).

To fly an aircraft, pilots must process a vast amount of information (Endsley, 2000).

The information they process may be simple or complex, clear or distorted, complete or

incomplete. Decision-making facilitates the process of choosing the “right” available

plan that would accomplish, or is related to the accomplishment of the task (Hastie,

2001). According to Hastie, decisions are situations involving behaviour combinations,

which are based on three mental concepts. These concepts are related to choosing

courses of actions, or outcomes of an event, especially as related to individual pilots,

and events that are subject to change as the flight progresses.

Pilots make decisions as a result of rational behaviour, provided that such behaviour is

well-suited to their goals (Simon, 1993). Studies, such as those of Edwards (1954) have

assumed that decision-makers behave rationally, maximising utility when choosing

among alternatives. According to Abraham and Sheeran (2003), past decisions, as well

as levels of satisfaction or an unfulfilled goal, support future decision-making. These

decisions may be based on the reversibility factor (i.e., individuals prefer making

decisions that are reversible) (Gilbert and Ebert, 2002). Gilbert and Ebert concluded that

individuals prefer to have the option to change their minds however, the ability to

change their minds actually inhibits their ability to be satisfied with their choice.

of 303

For example, pilots performing collision avoidance might engage in a two-step process;

first they decide to initiate manoeuvres for a heading change once the problem situation

is analysed and then they decide if their corrective courses of actions are furthering their

goals once the aircraft heading is changed.

A variety of decision approaches have been proposed. One such approach is

aeronautical decision-making (ADM), a systematic approach to the mental process used

by pilots to consistently determine the best course of action in response to a given set of

circumstances in complex and dynamic environments (Jensen, Adrion and Lawton,

1987). According to Jensen et al., the possibility of pilots making a good decision while

under cognitive stress is dependent on the relevant information available that may

influence her or his decision-making process. ADM executed in dynamic and complex

environments is frequently characterised, for example, by open-ended real world

problems, time constraints, and shifting unstructured or competing goals as key factors

for decisions under real-world conditions (Zsambok and Klein (1997)

Studies examining the impact of constraints of time and information on decision-

making have suggested an operator’s performance is in accordance with Naturalistic

decision-making (NDM) because the task demands under real-world conditions exceed

their cognitive capabilities. For example, Hutchins (1997) found that the command-

level naval decision-maker in threat detection scenarios had difficulty maintaining

situation awareness and had insufficient cognitive capability to accommodate secondary

tasks. NDM is another approach to prescriptive and normative decision-making, as

suggested by Klein, Orasanu, Calderwood and Zsambok (1993). The NDM replaced

those approaches associated with the well-structured environment under which

decisions are made. Decision-makers in this environment are fully informed and

rational, with perfect information. According to Klein et al., the practical application of

these approaches is not ideal for solving real world problems. Moreover, these

approaches required the used of display systems to support decision-makers making

better decisions. To address these limitations, NDM studied operators in field settings,

such as airline pilots and fire ground commanders, to examine how these experts were

able to make decisions under time pressure, uncertainty, high stakes, ill-defined goals

and dynamic conditions (Klein, 2008).

of 303

Klein’s findings suggest that fire ground commanders evaluated a course of action by

using mental simulation within the context of the current situation. The commanders

continue to evaluate current situations until an appropriate solution is found to allow the

action to be initiated.

The NDM approach has been used to design a system to support decision-making

(Noble, 1998). Noble (1998) refers to this system as a distributed situation assessment

tool. The tool supports experts to assess overall situations using the integrated

contribution of individual experts. Similarly, Orasanus and Fischer (1996) proposed

another model for decision-making, which involves two components; situation

awareness and a course of action. According to this model, situation awareness involves

identifying the problem, and the levels of risk associated with it, as well as the time

required for solving it. Building on a framework by Rasmussen (1986), a choice is made

by selecting an “appropriate action” from the options available, once the problem is

identified. A distinctive use of NDM was highlighted in a study by Wiggins and O’Hare

(2003). The study discussed the use of a cue-based training (CBT) for the recognition of

deteriorating weather conditions during visual flight rules. A similar study was

conducted by Morrison, Wiggins, Bond and Tyler (2009) to illustrate the effect of cue-

recognition on decision-making by both experts and novices. The study concluded that

cue-recognition experts were superior to novices in recognising decision-making cues.

Another, long-standing, approach is the cognitive continuum theory (Hammond,

Hamm, Grassia and Pearson, 1987). This approach states that decisions differ in the

degree to which decision makers depend on intuitive processes (e.g. implicit,

associative or automatic) and analytical processes (e.g. explicit, ruled-based, or

controlled). However, the differences between these processes are based on situation

awareness (Hastie, 2001). As noted by Hastie, there are some fundamental cognitive

functions that are basically considered to be implicit. These fundamentals include the

cognitive operation of accessing information in the long-term memory, familiarity and

similarity judgment, estimation of experienced frequencies, and causal judgment. These

can be used in higher-order explicit, analytic, goal-directed strategies.

of 303

Klein, Calderwood and Clinton-Cirocco (1986) proposed a particular form of NDM

called a recognition-primed decision (RPD). The model differs from naturalistic

decision-making in that people make decisions based on pattern recognition. The

pattern recognition, according to the study, facilitates quick decisions by matching

previous experiences with the current situation. Furthermore, the model predicts how

successfully decision-makers can make very rapid decisions from intuitive knowledge

based on past experiences. They concluded that experienced decision-makers, in this

case fire ground commanders, can choose a course of action by using mental simulation

to predict an outcome within the context of the current situation. Although the model is

designed to take experienced decision-makers into account, it can be argued that using

mental simulation to choose a course of action, without alternatives and under time

pressure, can lead to errors, high mental workload, and stress.

More recently, Krantz and Kunreuther (2007) presented a “construct-choice model”,

and suggested that decision-making is the selection of a plan. With this model, pilots are

encouraged to focus on goals or usefulness. According to Krantz and Kunreuther, plans

are designed to meet one or more of these goals. That is, pilots make plans to

unconsciously or consciously meet their specific goals, and some plans may satisfy

several of these goals. For example, pilots performing collision avoidance may be

satisfying several goals such as adjusting aircraft heading accordingly to avoid

obstacles, mental stimulation from a conflict scenario (i.e., that may lead to

consequences or reward), and potentially gaining useful conflict knowledge from

avoiding the conflict scenario. According to this goal/plan based model, goals are

context dependent and plans are based on their ability to meet the goals. Fundamentally,

the context provides the framework for the decision that needs to be made. The

decision-making rules are implemented and influence the plan that is ultimately chosen.

Krantz and Kunreuther apply this theory to the insurance business, but suggest the

theory may be appropriately applied to a variety of contexts.

According to Rasmussen et al. (1994), pilots perceive information from the

environment to determine the conflict situation. In the event that the conflict situation is

familiar, pilots will simply apply an acceptable set of procedures to avoid collision.

of 303

However, even though the information presented to the pilots is unclear or ambiguous,

further clarification is needed. The tasks or procedures are then changed. For example,

regarding rule based behaviour (RBB), if tasks or procedures for how to avoid collisions

are not clear, pilots will have to examine the state of the environment for more

information, or alternative routes for safe passage to destinations, based on a set of

rules. In a situation where no skills or rules knowledge exists, pilots will have to

perform mental simulation (i.e.,knowledge-based behaviour-KBB) of the current

situation to examine what other possible options are available, with respect to their

goals or objectives. This process is repeated until the set objectives or goals are

achieved (i.e., safe passage to a destination).

In high-risk environments, such as airspace, pilots need to diagnose conflict situations

according to standardised procedures and account for aircraft performance limitations.

Therefore, procedural decision-making is enacted. The procedural process includes go

and no go decisions. A pilot’s primary safety concern is which route to fly to avoid

obstacles, that is, a pilot must select the safest flight path. “Bounded rationality” is the

fundamental process responsible for this action, according to Reason (1990). This

action is based on a simple rule of reality. However, these simple rules should lead to

appropriate actions, and must be adapted to a specific environment to be ecologically

valid (Forster, 1999). Pilots’ appropriate actions in this context are to avoid

environmental constraints. For example, the simple rule of turning left or right to

change the aircraft heading is usually used to avoid collision. Therefore, the intuitive

approach adopted by pilots may not be based on their understanding of the task at hand;

however, it may be similar to other conflict scenarios for which the same appropriate

behaviour is familiar.

Furthermore, if the task is complex, an intuitive approach may be the only option

available to the pilot. For example, switching off a faulty system will be necessary to

shut down an engine that has just malfunctioned. These rules are based on the

evaluation of mental models, rather than formal logical conclusions (Johnson-Laird,

1983). According to Todd and Gigerenzer (2003), ecologically rational decision-making

is “making good decisions with mental mechanisms whose internal structure can exploit

the external information structures available in the environment”.

of 303

Pilots perform tactical manoeuvres when the current environment requires them to

change the flight path immediately, or delay response due to some unforeseen danger.

Thus, it is a short-term decision-making process. For example, aircraft performance

constraints may prevent pilots from maintaining the current flight path due to ATM

regulations. Factors that may influence pilots’ decisions in this situation include lateral

or vertical minimum separation standards. Strategic manoeuvring, on the other hand,

refers to planning and implementing pilots’ decisions to meet the overall objective of

conflict avoidance. However, a pilot may not be satisfied with the current aircraft

performance or air traffic conditions. Therefore, he or she may need to reconsider that

first decision (i.e., reject the present course of action) by changing aircraft heading or

speed. In a similar manner to the adaptation of the flight route selection heuristic, the

reconsideration of the flight route choice may result from the pilot’s experience carried

over from previous flights (Rasmussen, 1983).

The following section will discuss the different approaches used to design a system.

Many approaches have been developed and applied to automated display systems.

These include the User, Control, Technology and Ecological-centred approaches to

examine pilot cognitive activities. A review of these approaches will be conducted to

identify appropriate designs for the novel display to be used in a free-flight

environment.

2.8. Automated Display System Design Approaches

Pilots develop a personal perspective such as feelings, beliefs, wishes, or what

important role they will play in the environment (Sinreich et al., 2005). Technological

advances have resulted in a Flight Management System (FMS) that can handle and

process a large volume of data within a short time; however, human sensory abilities

and cognitive processing of these data, for example, in working memory, remain

unaltered (Miller, 1956). As pilots manoeuvre in space, they perceive and perform

flying tasks by considering parameters such as conflict angle, relative speed and

altitude, rate of change of altitude (Thomas and Wickens, 2006; Xu and Rantanen,

2007), and rate of turn (Andrews, 1978). Pilots manipulate these parameters to resolve

detected conflicts in space (Rantanen et al., 2004; Thomas and Wickens, 2006).

of 303

A number of studies (Andrews, 1978; Van Dam, Mulder, and van Paassen, 2008; Bach,

Farrell and Erzberger, 2009) have highlighted some of the flight constraints for

managing safe separation. These studies support the designer’s objective of creating a

system to support pilots’ task performance for detecting and resolving conflict. One of

the basic principles for designing such a system is to simplify the pilot's task by

reducing the pilot’s tasks demand when interacting with the system. This idea

influences the design of automated systems. An understanding of automated system

limitations and capabilities is required for humans to achieve control over its functions

(Parasuraman, Sheridan, and Wickens, 2000). However, it is practically impossible for

humans to know all limitations and capabilities of automated display systems under

different sets of flight conditions. Currently, the literature has proposed four approaches

to design a system to improve human and automated system interactions. These

approaches are 1) the Control-Centred Design, 2) the User-Centred Design , 3) the

Technology-Centred Design, and 4)the Ecological Centred Design. The following

section will discuss the main limitations of the first three approaches and will describe

how the fourth approach, ecological-centred design, can be adopted to support pilots’

problem-solving for collision avoidance.

2.8.1 The Control-Centred Design Approach

This approach addresses the question of how much information pilots are capable of

perceiving and remembering, and what kind of decisions or problems they are likely to

face to achieve their goal. The ability to perform and successfully complete a task

defines the pilots’ capability and limitations. The design of an automated display system

under this approach revolves around computer capabilities and limitations

(Parasuraman, Sheridan and Wickens, 2000). The implications of these limitations

guide the designer not to exceed users’ capability to process relevant information. In

other words, the information being processed should be compatible with the users

mental model, provided that human expectations are to be met (Gersh, McKneely and

Remington, 2005; Flach et al., 1998). However, this approach has some limitations.

Pilots are not always kept in the control loop, and this approach is based on an

assumption that pilots can perform their primary tasks without providing any feedback.

of 303

Another problem with this approach is that translating specific data into the design may

be difficult. According to Lee and Yoon (2004), designers must to work within these

constraints.

2.8.2 The User-Centred Design Approach

The User-Centred Design approach focuses on an interaction between different

properties of a system and humans (Flach et al., 1998). According to Flach et al., the

user-centred design approach combines humans and technology as a system. The

allocation of a human role to function as a “manual” controller, or a supervisory

controller is an important consideration. Thus, human operators are placed in a

supervisory role (Sheridan 1999) with unexpected results, such as inadequate feedback,

a change in feedback, or a loss of feedback; each contributing to out-of-the control loop

performance problems (Kaber and Endsley 1997). Norman (1986, p.1) states ‘… user-

centred design emphasises that the purpose of the system is to serve the user, not to use

a specific technology, not to be an elegant piece of programming. The needs of the

pilots should dominate the design of the interface, and the needs of the interface should

dominate the design of the rest of the system’.

Another important concern regarding this design is observability. The approach allows

the user to view all information displayed in an interface to minimise time delays.

Norman further stated that the stability of a system depends on how the operator can

cope with the system’s dynamics. User-centred design is mostly to provide operators

with a “predictive display” to stay ahead of the system as long as the assigned

parameters are consistent with the system’s expectations. In general, an automated

display system, such as autopilot, can increase an aircraft’s stability and reduce the time

required to perform a task. This type of automated display system operates within the

outer loop, which is based on global assumptions. In the case that these assumptions are

violated, the result will be that controlling the system will prove difficult. Thus, the

system can become unstable without any intervention from a supervisor. However, the

supervisor will have difficulty in keeping up with the inner loop demands. The

fundamental question is therefore how to design a system that satisfies both the inner

loop and outer loop constraints to maintain the system’s stability.

of 303

2.8.3 The Technology-Centred Design Approach

Unlike the user-centred design approach, the technology-centred approach is based

primarily on system performance limitations and capabilities. The available evidence

suggests the design concept behind this approach is to support a number of factors

including shrinking costs by reducing the number of flight crew (i.e., from three (B727)

to two flight members (B787), managing fuel cost efficiently, flying higher and faster

and reducing or even eliminating certain types of pilot error.

The system capabilities can be quantified as how fast the new system can perform or

how “smart” the basis of its technological design is. The system is usually identified

with vast numbers of input data. For example, a single-indicator approach is used to

display each of these data (Flach et al., 1998; Mitchell and Miller, 1986). The indicator

is to display the internal process of several sensors that are not “integrated” into a

system. Monitoring all these indicators necessitates allocating attention resources to an

appropriate display to create and maintain situation awareness (Sheridan and

Parasuraman, 1995). With displays spread across the cockpit, humans must process

these data to develop suitable control actions (Mitchell and Miller, 1986). Any

additional new sensor will require a new display (Flach et al., 1998). This approach

poses potential problems, especially in aviation, where many state variables are

associated with confusion (Flach et al., 1998) and pilots may be forced to execute an

inflexible task. For example, a computer password might be computed in many forms

such as "31st January 2010", "31/01/2010", "31/1/10", "31 Jan 10". However, only

specifically formatted data are valid. This implies the automated-centred design

approach does not have the capability to solve problems outside the “black box”, and

requires pilots to be aware of the problem. Another problem with this approach is that a

system component may meet a situation it is not designed to handle. This limitation

may not be visually communicated to pilots, resulting in mode confusions and potential

errors.

of 303

As discussed in the three previous sections, each of these approaches have been applied

to human and machine interactions, without addressing environmental issues. However,

data presented by these systems, or the way pilots interact with these systems is not

visible to both pilots and machines. In certain situations, pilots are prevented from

interacting with the machine. A pilot’s state of mind (mental models) is not considered

during the interaction with these systems. For example, with the current FMS, in the

event that a pilot receives a new routing from ATC or needs to avoid hazards, he or she

may not be able to easily re-compute new data into the FMS at short notice. An ATC

instruction to deal with environmental conditions, such as air traffic, requires pilots to

adjust aircraft heading by re-programming the Flight Management System (FMS) to

circumnavigate obstacles.

This solution has obvious limitations. First, although the FMS could perform fast,

accurate and complex calculations, it could keep pilots out of the control loop (Endsley

and Kiris, 1995). Second, the system is not designed to consider pilots changing goals

or environmental conditions (Johnson, Battiste and Bochow, 2003). Third, pilot

workload will change when confronted with unexpected events, such as the need to

change aircraft heading by inputting complex information into the system within a short

time. Fourth, conceptual models are not compatible with users’ mental models. Finally,

applying flight computer rules is usually based on assumptions, or “heuristics”, which

may require a pilot to deal with new information or unforeseen circumstances.

2.8.4 The Ecological -Centred Design Approach

Ecological Interface Design (EID) differs from the other approaches in that it is based

on the constraints of the environmental and user cognitive capabilities as a core part of

an “ecological system”. Gibson (1979) laid the foundation for the ecological design

approach. According to Gibson, both perception and action are connected through the

physical world (i.e., environment). Rasmussen and Vicente (1992) explored the

application of Gibson’s theory of direct perception and action to design an industry-

scale application based on the EID approach. The importance of the EID approach is in

simplifying information processing by transformation into perceptual tasks, thus making

environmental constraints visible to the pilot.

of 303

Rasmussen and Vicente note that the EID can be used to make constraints visible,

which support direct perception of affordances and are controlled by the three

fundamental cognitive levels of human behaviour [i.e., Skills-, Rules-, Knowledge

(SRK) model]. This approach is designed to provide users with a “tool” to support their

interaction. The “tool” decides how efficient the interactions are between the user and

the environment. The SRK model through the EID essentially describes the possible

ways conflict avoidance information should be displayed on the interface and

understood to make good use of users’ perception and action abilities (Vicente, 1999).

Table 2-1 shows the most common different modes of SRK for human-machine

interactions approaches. Importantly, the system’s representations should convey

“meaningful” information to the pilots, rather than information as a function of an

interface. Thus, how does perception inform a pilot that he or she cannot avoid a

collision? A theory that addresses this question must consider both indirect and direct

perception (Jones, 2003). First, for indirect perception, pilots need to perform mental

simulation when interacting with the environment, and storing the meaning of events in

the long-term memory. Direct perception, on the other hand, does not require that pilots

perform mental simulation when interacting with the environment. Thus, events have an

inherent meaning. Jones further argued that the meaning of avoiding obstacles is not

internally constructed and stored however, it is part of a system where there is

interaction between humans and the environment. This suggests that humans use their

basic instinctive behaviour to acquire information directly from their natural

environment and may not need mental simulations. Therefore, an individual’s

perception affords access to certain forms of action they choose to perform.

Affordances are perceivable possibilities for direct perception and action by the

individuals.

of 303

Table 2-1 Overview comparison of (SRK) model between the Classical and Ecological Interface Design Approaches

Three Levels Cognitive Control

“Classical” Interface Design Approach

Ecological Interface Design Approach (Principles)

Knowledge-Based

(Conscious level)

Users assign meaning to what has been interpreted and manipulated with reference to a mental model; requires considerable feedback

Correct, complete and accurate mental model by making a visible presentation of information and constraints conforming to the work domain of the system

Rules-Based

(Sub-conscious level) Perceptual cues are

optimised, or stored procedures are matched with previous experience, but can lead to procedural traps in novel situation

Taught problem solving/planning.

Provide unique and consistent mapping between the cues and symbols that the interface displays

Support users with salient perceptual cues to evaluate and select solutions to the most suitable for the problem, are proposed.

Skill–Based

(Automated- little conscious)

Familiar environment and highly routine activities require little feedback

Facilitating direct manipulation of familiar objects

Invariants are perceived directly at all levels of abstraction to reduce the cognitive workload

Affordances are actions possibly formed by the relationship between an observer and

properties of its environment (Gibson, 1979). Rasmussen and Vicente (1992) define the

interrelationships between affordances and the five levels of abstraction hierarchy (AH).

These levels of AH include physical form, physical functions, general functions,

abstract function and functional purposes of the system referred to as work domain

analysis (WDA). They are based on generic questions, such as “What?” which relates to

declarative knowledge in the form of a physical system such as an aircraft; "How?"

which relates to procedural knowledge of flying an aircraft; and “Why?" which relates

to meta-knowledge about the purpose of flying the aircraft.

of 303

However, a pilot can "slide" up or down the levels of abstraction hierarchy to cope with

complexities and unexpected events in the natural environment. Two systematic

approaches were considered based on analysing information requirements from physical

systems, Task-analysis (”Classical” Human Factors Technique) commonly used in the

previous Section 2.8.1-3 approaches (Diaper, 2004) and Work Domain Analysis

(Rasmussen, Pejtersen and Goodstein, 1994; Vicente, 1999). In the Work Domain

Analysis, designers first analyse the work environment (i.e., constraints analysis) before

analysing what users are doing (i.e., what is actually done-tasks) and map important

properties of the abstraction hierarchy onto visual displays. The comparisons between

the task analysis and work domain analysis is presented in Table 2.2

of 303

Table 2-2 Summary comparison of two analysing information requirements techniques

Task Analysis

(”Traditional” approach)

Work Domain Analysis

(ecological approach)

Exert influence over what users do in relations to the various tasks

Exert influence over how users should comprehend in relation to functions and constraints of their environment

Effort directed toward analysing users characteristics and identify what users want

Effort directed toward analysing the environment and making constraints visible to users

Instructions on how to navigate to destination in steps are analysed.

Results are limited to the tasks-related.

More than two solutions to reach goal are provided

A picture of the environment that includes navigational tasks is analysed.

At least one solution to reach the goal is provided.

Cope with familiar and unexpected situations.

Support indirect perception of information

Support direct perception of information

Users behaviour is influenced by the devices they use to perform tasks

Users behaviour is not influenced from previous interface designs but from the environment.

User cognitive resources restrictions are imposed on the processing of information from the physical systems

Goal-directed behaviour is externalised on the work domain as a mental model of the physical system

Perception and action is performed through users cognitive system

Perception and action is performed through the environment

Events have to be interpreted Events have an inherent meaning

of 303

Several studies have used the EID approach for the purpose of designing systems, such

as for the redesign of TCAS (Cleveland, Fleming and Lee, 2011), driving of an

automobile (Stoner, Wiese and Lee, 2000), terrain awareness (Borst, Suijkerbuijk,

Mulder and Van Paassen, 2006), and military command and control (Bennett, Posey

and Shattuck, 2008). The EID as presented by Van Dam, Mulder and van Paassen

(2008) to support pilots’ collision avoidance manoeuvres is based on relative motion

analysis. Relative motion requires the understanding of the movement of aircraft

relative to each other therefore the dynamic system can be simplified to consist of five

variables (Andrew, 1980). These variables include range rate, time to closest approach,

tau – range and/or range rate, miss distance, and crossing angle. The effect of control

actions utilise range rate, relative bearing, and speed. These three variables are useful in

understanding horizontal collision avoidance. Though the display used in the study by

Van Dam, Mulder and van Paassen (2008) aids pilots to perform horizontal collision

avoidance, it failed to support pilots with the relative velocity vector visible to pilots.

With the Intruder and Ownship in relative motion, the successive movement of the

Intruder relative to the Ownship is not shown in this study. The continuous display of

the Intruder at the measured distance (i.e., line-of-sight) and relative bearings measured

positive clockwise from the velocity of each aircraft is also missing.

Another limitation of this display is that it did not support the “actual” progressive

impact velocities of two colliding aircraft (i.e., protective zone), in terms of displaying

time-to-collision. This prevents pilots from having crucial visual information, such as

how fast the Intruder is approaching the Ownship. Furthermore, as indicated by pilots,

the rate at which the Forbidden Beam Zone (refer to Figure 2-2a) expands in the study

makes it difficult to predict its future state. The interface in this study, as shown in

Figure 2-2a, did not show a direct relationship (i.e., line-of-sight) between the Ownship

and Intruder in conflict. In terms of conflict resolution manoeuvres, it would clearly be

difficult to visualise or determine when to start or complete the angle of rotation before

collision.

In Figure 2-2b, however, the apex of the protective beam zone (the shaded area) may

indicate the Ownship’s relative position (i.e., centre) in addition to the Ownship

positioned at the centre, as a point of reference to the Intruder in conflict.

of 303

(a) Horizontal separation assistance display (b) Constraint on Ownship velocity

A graphical display solution is required to make the rate and direction of the actual

movement of the Intruder relative to the Ownship visible to pilots. The visibility of

these constraints should reduce pilot cognitive workload. Thus, the solution will

improve pilots’ performance and ascertain how situation awareness and mental

workload fits into the attainment of the pilots’ goals, such as maintaining safe-

separation in a free-flight environment. One solution based on a relative velocity vector

provided by Ellerbroek et al. (2009) is similar to that provided Van Dam, Mulder and

van Paassen (2008). The screenshot of the display is shown in Figure 2-2a.

A similar study conducted by Dole et al. (2005), to examine pilots’ operations in en-

route free manoeuvring, focused on the use of an airborne-based conflict resolution

display. The proposed prototype of an autonomous operations planner interface was

designed around conflict prevention information in the form of spatial locations of “no

fly” restriction zones. The interface provides conflict alerting and resolution advisories

using the state and intent information of the Ownship and traffic aircraft, as shown in

Figure 2-3. The figure shows a screen shot of a manoeuvre restrictions band on the

heading and vertical displays. Dole et al. reported that no consistent trends were found

across traffic scenarios for workload or safety. The findings also suggest that pilots are

supportive of the autonomous flight rules although acceptability decreases with slight

Figure 2-2 Self- separation assistance display (adapted from Ellerbroek et al., 2009)

of 303

increases in traffic. However, there are some issues with the way Dole et al. present the

conflict resolution information. For example, the resolution information is spread across

the interface, that is, the manoeuvre restrictions band is displayed on the heading

display, the Intruder location is off the Ownship’s right wing, and the conflict zone is

displayed along the Ownship’s flight path. In addition, the representation of the

manoeuvre restrictions band in relation to which way to turn (i.e., behind or in front of

the Intruder) is not clear. These issues were also replicated in a study conducted by Van

Dam et al. (2008a; 2008b) which showed that pilots’ attentional resources are spread

across the interface (i.e., spatial-based attention).

Accordingly, with systems such as these, the information provided indicates that

individual pilots may have to integrate their plan and interpret information obtained

from the interface to support their situation awareness and update mental models. This

integration, and the interpretation of this information, may vary from pilot to pilot.

Though this may not endanger flight progress, there is a need to integrate or group this

information to minimise scanning. This problem is an issue that increases in importance

with the new cockpit systems currently under development.

Figure 2-3 Autonomous operations planner interface (adapted from Dole et al., 2005)

of 303

2.9. Summary

Studies of mid–air collision problems have been documented since the Grand Canyon

mid-air collision in 1956. Statistics revealed that more than two-thirds of these

collisions involve two aircraft travelling at convergence angles. As in many of these

cases, one of the pilots, or possibly both the pilots involved in the mid-air collision were

unable to use the “see and avoid” techniques due to vision restrictions at the critical

mile of closure to avoid the collision.

The current air traffic management system is designed to constrain pilots to fly within

specific flight corridors to maintain separation. However, because of the increase in air

traffic, an alternative approach is required. The future of air traffic management

systems, as proposed in the recent literature, suggests that the number of aircraft will

double in the next three decades. As air traffic increases, new challenges and

unexpected problems in a free-flight environment will emerge. The solution to the

looming problem is to develop a system to manage more air traffic movements safely,

with less interaction between pilots and controllers than the current system. Allocating

or delegating responsibilities of monitoring and maintaining minimum separations to

pilots is a novel task. This is a step toward a “Free Flight” environment. Therefore,

“Free Flight” can be defined as a “pilot-preferred flight path” that does not compromise

the current minimum separation standards of five nautical miles. The proposed FCAS

display makes the use of ADS-B data to display traffic information that has been

accepted for a free-flight environment.

The basis of a free-flight environment is for pilots to use an airborne display, such as

Cockpit Display of Traffic Information (CDTI) to maintain minimum safe separation, as

compared with the Flight Management System (FMS). The FMS requires

reprogramming to change flight paths and this increases pilot workload and can change

the structure of the workload over time, particularly if reprogramming occurs during

critical phases of flight, such as the final approach. It also carries problems such as

“automation surprises” or “mode of confusion”. This implies that there is a mismatch

between the pilots’ mental models of the behaviour of the system and the realistic

behaviour of the system.

of 303

To perform self-separation manoeuvres in a free-flight environment, pilots need a

supportive tool that clearly shows conflict geometries and provides alternatives to

overcome a loss of situation awareness. Other problems that come with supportive

systems, include their complexities. These complexities can challenge the three levels

of situation awareness (perception, comprehension and projection). Pilot difficulties in

maintaining situation awareness arise from a lack of ability to plan, or arrange displayed

information in a specified form that will aid decision-making.

This chapter has also outlined some theoretical and practical aspects of the importance

of research into mental models and conceptual models. A mental model has a similar

quality to personal software about how a physical system operates in the real

environment. It is a mental representation of the surroundings of a physical system, the

relationships between the system’s parts, the pilot’s natural perception of their actions

and the results of these actions. Mental models may also help to introduce a new

approach to solving cognitive problems and performing flying tasks. By understanding

pilot capabilities and limitations, it will be possible to design a system that is

compatible with their mental models. The implications of incomplete mental models of

complex tasks have been discussed. A pilot’s mental model is formed based on

conceptual models of the system. The conceptual model enables a pilot to construct a

mental model from their working environments or an image of how to use the model

(i.e., physical system) effectively. The importance of different approaches to design

automated systems as conceptual models has been highlighted.

In this context, the Ecological Interface Design (EID) examines both the interaction of

pilots with their workplace setting and technological artefacts and the latter is very

important as modern technology becomes increasingly sophisticated. The technological

evolution has made displays more powerful however, human cognitive processing

capability remains the same. The design of FCAS using EID was founded on an

abstraction hierarchy of a system for collision avoidance system and was influenced by

pilots’ levels of cognitive control [i.e., the Skills, Rules, Knowledge (SRK) framework].

of 303

As a means of reducing pilot mental workload and supporting knowledge-based

behaviour, EID aims to support pilot performance and system infallibility for both

excepted and unexpected problems in complex socio-technical systems. This is

important when pilots make critical decisions in a free-flight environment.

Predominantly, naturalistic decision-making (NDM) describes decision-making by

experts in real world situations (e.g. aviation, nuclear power, fire ground command,

command and control). These situations typically involve time constraints, dynamic

environments, goal conflicts, and uncertainties. Decision-makers may need to have

decision support tools to assist their decision-making processes. With this supportive

tool, individuals rely on intuitive risk judgments to evaluate hazards. The discussion

presented in this chapter shows that further analysis is required to develop and evaluate

a novel display for conflict resolution. This problem is an issue that increases in

importance with new cockpit systems currently under development. Chapter 3 will

discuss the theoretical framework for the development of the FCAS with the aim to

limit the induced complexity by automated systems.

of 303

Chapter 3. Theoretical Framework for FCAS

“It is not how much information there is, but rather how effectively it is arranged” Tufte, 1990

As discussed in Chapter 2, the present study has highlighted issues related to airspace,

and pilots’ cognitive ability to interact with automated display systems. This chapter

presents the theoretical framework for the development of FCAS. As outlined in

Section 2.8.4, pilots need a supportive tool that clearly shows conflict geometries (i.e.,

flight course the pilot should not take so as to avoid a collision as pilot difficulties in

maintaining situation awareness arise from their inability to plan or arrange displayed

information in a specified form to support safe-separation,. This problem is an issue that

increases in importance with the new cockpit systems currently under development.

New cockpit systems in and of themselves don't make planes more collision-prone.

After all, there is (roughly) the same number of planes sharing the same airspace, so

statistically the probability of a collision is unchanged. Presumably the additional

volume of information from more advanced systems exacerbates the pilots' inability to

cope with all the information. The chapter begins with a discussion on how attention

theories are related to the design of the Flight Collision Avoidance System (FCAS),

followed by the theoretical Cognitive Work Analysis framework, one of the Ecological

Interface Design based approaches. Both of these theories support the development of

the FCAS. This chapter also describes several attentional concepts, such as filtering,

search and orienting theories. The contents of attention can reside within or outside the

pilot’s vision field of view. Information presented at a system's user interface should be

structured to capture the pilot's attention, by using, for example, perceptual boundaries,

such as the colour of protective zones. The perceived information should be able to

improve situation awareness and reduce mental workload. The chapter ends with a

design solution of the FCAS.

of 303


of 303

3.1. Introduction

As stated in Section 2.8.4, in complex sociotechnical systems, Cognitive Work Analysis

(Vicente, 1999) is a framework developed by (Rasmussen, Pejtersen and Goodstein,

1994) and is based on earlier work done by Gibson. The main objective of this approach

is to make environmental constraints visible or audible to pilots (Rasmussen et al.,

1986; 1994). The visibility of these constraints should reduce the pilot’s cognitive

workload activities, which will in turn allow pilots to focus on both anticipated and

unanticipated events in a complex system, thus improving the pilots' performance (i.e.,

maintaining minimum separation) and situation awareness.

CWA has the ability to analyse real-life phenomena in areas including computer

science, nuclear power plants, and aircraft systems, while at the same time maintaining

the complexity that is inherent in such systems. Furthermore, CWA has been used to

evaluate the usability of displays at the system level (Hunter and Randall, 2013), and

make more visible constraints of the system that define actions, or facilitate the process

of making choices among possible alternatives (Xiao and Sanderson, 2012). CWA first

evaluates systems already in existence, and makes recommendations for design. The

evaluation is based on the analysis of information behaviour in context. According to

Sanderson, Naikar, Lintern and Goss (1999), the applications of CWA have focused on

interface design. Notably, some of the early applications of CWA have focused also, for

example, on analysis and error investigation (Rogers et al., 2004), development of

decision-making support systems for cardiac care nursing coordinators (Burns,

Momtahan and Enomoto, 2006), military team building and training (Naikar, 2006), air

traffic control (Ahlstrom, 2005), and manufacturing (Higgins, 1998).

Studies have provided comprehensive applications of CWA to address human and

machine interface designs to analyse constraints imposed on activities within a

particular type of system (e.g. Naikar, Pearce, Drumm and Sanderson, 2003; Salmon et

al., 2007). There have been at least four general concepts by which CWA is used to

design a system. Firstly, the output analysis of the CWA can be mapped to a design

(Amelink et al., 2003).

of 303

Secondly, CWA can provide detailed guidance to pilots about the ideal way(s) to

perform tasks; normative techniques can optimise pilots’ workload and minimize error

(Naikar, Moylan and Pearce, 2006). Thirdly, the CWA can capture detailed design

requirements, in an iterative manner, in the form of prototypes of support systems

(Chalmers, Webb and Keeble, 2001). Finally, the CWA can be used in a large-scale

system design process, such as nuclear plants (Lau et al, 2008). However, while the

suitability of CWA for understanding complex system performance in a variety of fields

is widely acknowledged, the process of doing so, and specifically how CWA provides

information for design and technology specifications, remains unclear.

The CWA consists of five phases, which can be classified into two groups. The first

group comprises of Work Domain Analysis (which addresses the questions of how and

why the system is designed) and Control Task Analysis (i.e., decision-making). The

second group comprises Strategies Analysis (i.e., the processes by which actions can be

carried out by the users), Social Organisation and Cooperation Analysis (i.e., deals with

the relationships between pilots, either in groups, teams, or with automated systems, and

under what authority they operate) and Worker Competencies Analysis (i.e., users'

capabilities and their abilities to conduct tasks) (Rasmussen et al., 1994; Vicente, 1999).

These phases provide pilots or the system designers / analysts with a way to deal with a

situation that is difficult to understand. The evidence for the CWA framework as a

direct design methodology is inadequate, as noted by Salmon et al. (2010). Lind (1999)

argued that the abstraction hierarchy suffers both methodological and conceptual

problems. For example, there are no acceptable procedures to support designers in the

acquisition of the background knowledge necessary for modelling tasks. Further to this,

there is also a lack of consistency in the usage of the framework’s levels of the

abstraction hierarchy. For example, the first level of the physical system representation

of the abstraction hierarchy as an airborne ecological interface display has been

excluded in the design process (Abeloos, Van Paassen and Mulder, 2003). This thesis

has recognised that the framework can be used to understand complex system

performance; the details on how to translate framework outputs into precise design

requirements that support the operator in decision-making were still being worked out

in the literature.

of 303

There is a need for CWA to go beyond analysis to support the design of the flight

collision avoidance system, so the current study uses attention theories to complement

CWA with precise design requirements of the FCAS. In the subsequent sections, the

fundamentals of the Ecological Interface Design (EID) approach were discussed.

3.2. Ecological Interface Design Applications

This section will elaborate on the discussion on the application of the Ecological

Interface Design (EID) approach part of Cognition Workload Analysis (CWA) to

support pilots in resolving air traffic conflict. The objective of the EID framework is to

guide the design of a new system for use in a known environment. The EID is a two-

stage analysis and design process (Vicente and Rasmussen, 1992). The first stage is the

Abstraction Hierarchy (AH) and comprises both physical and functional information at

five (5) levels; these consist of physical form, physical functions, general functions,

abstract function and functional purpose. This activity is frequently referred to as the

Work Domain Analysis (WDA). Accordingly, Burns and Hajdukiewicz (2004)

suggested that WDA is an essential component of the EID. The WDA is a formative

work analysis that should facilitates the understanding and identifying of cognitively

relevant and significant constraints at each level. The WDA decide what critical

information should be structured, arranged and displayed on the interface to minimise

users' mental workload. The WDA promotes an accurate and correct mental model by

providing visualisation of information and constraints of the physical system.

The second stage of EID involves mapping all three (3) levels of perceptual processing

(i.e., Skill, Rules, and Knowledge levels of human information processing), constraints

and invariants, in the context of the WDA. This approach to design presents the user

with the most appropriate support, using direct manipulation interfaces at the lowest

possible perceptual processing level (i.e., Skills and Rules) to minimise dependence on

analytical knowledge-based behaviour.. A pilot's ability to focus on desired information

for a significant time depends upon the ability of his/her cognitive processes to process

the information from the environment that they perceive as relevant (i.e., problem

space).The application of the EID approach to make flight constraints visible to support

pilots in resolving air traffic conflict, can now be discussed.

of 303

3.2.1 Work Domain Analysis (WDA)

The first component of CWA is the WDA. The WDA is a theoretical framework used

for analysing and modelling systems across many domains. For example, WDA has

been applied to weather information for air traffic controllers (Ahlstrom, 2005), training

equipment for military applications (Lintern and Naikar, 2000), medical applications

(Miller, 2004; Watson and Sanderson, 2007), the aviation domain (Ho and Burns,

2003), and the nuclear domain, robotics, and oil sectors (Jamieson and Vicente, 2001).

With WDA as a modelling approach, environment constraints can be modelled (Naikar,

Hopcroft and Moylan, 2005). These studies have affirmed the usefulness of the WDA

approach, concluding it is useful for the following reasons: firstly, it incorporates the

functional relationship between the levels of the abstraction hierarchy in a system;

secondly, it provides a template that helps comprehend pilots’ goals, and how they

interact with the systems in question; thirdly, WDA is structured for analysing a specific

situation under study, and not for analysis of theories, or models under consideration.

These properties make the framework a powerful model for evaluation and designing of

information systems for a specific situation. Rasmussen (1986) presented and described

the “how-what-why” of a system’s operation. Rasmussen also pointed out, because

abstraction hierarchy levels also known as WDA are interrelated through “means-ends”

relationships. When pilots move along the “means-ends” link, questions as to why the

system is so designed should be answerable. Finally, the abstraction hierarchy is a

useful tool for designers using an EID approach. The approach is used to analyse a

pilot’s behaviour when interacting with a system within an environment. For example,

pilots’ interactions between levels of abstraction hierarchy should support them in

anticipating changes in a system by suddenly noticing another aircraft encroaching on

his airspace, which are likely to occur in the future. A system that supports or engages

pilots actively to carry out these changes might allow them to stay out of any conflict

with other aircraft.

Table 3-1 illustrates the five levels of abstraction hierarchy of conflict separation in a

free-flight environment. These levels, discussed in detail in the following subsections,

provide answers to questions of conflict resolution.

of 303

For example, pilots would like to know how close their aircraft is to the Intruder’s

safety zone. How much faster or slower will they have to fly in order to avoid collision?

What bank angle is required for turning manoeuvres without exceeding aircraft

performance limitations? What is the aircraft's maximum and minimum heading

required to avoid conflicts? What is the purpose for manoeuvring: cost or safety? The

information presented in Table 3-1 indicates that considerations could be structured as a

means-ends hierarchy, and thereby function as a mechanism for pilots to cope with

complexity and dynamic environments to achieve the functional purpose of the system.

Table 3-1: Airborne separation display in a free-flight environment

Function Purposes Safety goal

Circumnavigation Current and future safety

Values and Priority Measures (Abstract Functions)

2-D representation Line of Sight (LOS) Relative velocity Relative position Relative speed

Purpose-Related Functions (Generalised Functions)

Visual observation Time to collision/ avoidance Closure rate (loss of separation) Expand rate of protective cone

Object Related Process (Physical Functions)

Visibility Aircraft heading, Angle of rotation Protective zones

Physical Objects (Physical Form)

Object related to Seeing (geometry) Aircraft constraints Relative velocity vector Flight path trajectory Time

of 303

3.2.1.1 Functional Purpose

This level states the objective behind the design of the system. For example, the goal of

pilots in a free-flight environment is to maintain minimum separation standards and

avoid air traffic or obstacles within the aircraft flight envelope. Safe flight can be

achieved through four factors that can influence and control the pilots’ goal (Wickens et

al., 2000). These factors are 1) Significance - of the goal, i.e., purpose and importance

2) Expectancy - the selection of a specific goal, determined by the desirability of the

outcome, i.e., shorter versus longer distance to the destination 3) Prediction - of aircraft

future states (i.e., a change in flight path) enables pilots to instantly predict the possible

future state of the system in many conflict situations 4) Effort - The use of a pilot’s

physical or cognitive processes required to manoeuvre the aircraft. However, the effort

hypothesis predicts a linear increase of apparent heading change with range.

The functional purpose of a free-flight environment is to allow pilots to choose the

desired flight path so as to provide safe passage to a destination. The possibility of what

the pilot does should have a value, e.g. whether to fly in front of or behind the intruding

aircraft is an important consideration for collision avoidance. A conflict avoidance

resolution can be influenced by important factors, such as quality, value, and

availability of relevant air traffic information. The extent to which pilots accomplish

their objectives depends on how effectively this information is presented. This

information must be clear, decisive, and enable pilots to handle unexpected air traffic

events (Rasmussen, 1985; Dinadis and Vicente, 1999), and pilots must be able to make

sense of the information presented (Flach et al., 1996).

of 303

3.2.1.2 Abstract Function

The Abstract function defines the principles or laws that govern how the system should

work to achieve the functional purposes. The level defines the independent variables

that pilots need to control to achieve the functional purposes.. For example, a pilot’s

ability to violate minimum separation standards in a free-flight environment depends on

traffic density as a function of dynamic constraints, aircraft performance limitations,

conflict geometry, and cognitive workload. Understanding aircraft performance

limitations (particularly speed) is critical to all aspects of flying aircraft, especially for

obstacle clearance.

Accidents such as mid-air collisions may have occurred due to lack of information on

the relative velocity and position of obstacles that were not adequately displayed to the

pilots. For example, turning a flight to avoid obstacles can be constrained by aircraft

heading as shown in Figure 3-1. These constraints address the pilot's ability to instantly

change the flight path and return to the original flight route in a timely manner.

However, the cost constraints of flight path displacement depend on the displaced

distance, and originally planned route to the destination. Estimated time of arrival is a

function of acceleration (~a ). Because acceleration is defined as the instantaneous

change in velocity as a function of time, the equation is often rewritten as:

dt

Vda ~

~ (3.10)

The velocity vector is expressed as:

~~~ kk vjiuV

(3.11)

The expression (3.10) for V, airborne aircraft acceleration can be written as:

~~~ kk jviua

(3.12)

where kiu~ is the tangential and kjv

~ is the radial acceleration.

of 303

In Figure 3-1 L is lift, T is thrust, W is the weight, D is drag, 𝞿 is the heading change, a is the acceleration of the aircraft. r is the radius of turn and ω is the angular velocity.

The tangential acceleration (~a ) of the airborne aircraft can be written as the derivative

of two functions:

dt

vda ~

~ (3.13)

dt

rdtr

dt

da t

t~

~~

~

~

(3.14)

Where ~

~ rdt

d t

and dt

rd~

~ are the tangential and radial acceleration, respectively,

~̂u is

the unit normal vector for the aircraft trajectory, and ~r is its instantaneous radius of

curvature. Substituting equation (3.11) into (3.12), the following expression is obtained:

Ownship

r

V

D

T

W

L V

Figure 3-1 Ownship in a level turn

of 303

dt

rdu

R

va t

~

~~

~

2

~ˆ (3.15)

Figure 3-1 is used to derive the aircraft’s dynamic equation of motion, is represented

by:

~amF (3.16)

The equation (3.16) governs the law of aircraft dynamics. The equation allows us to

sum up all the aerodynamic forces acting on the aircraft when turning during a flight.

This can be rewritten as:

~~~~~amDWTL

(3.17)

Where ~L

is lift vector, ~T

is thrust vector, ~

W

is the weight vector, ~D

is drag vector and

m is the mass of the aircraft.

T A

β

x

Y

xh

D

A’

yh

Flight path

Figure 3-2: Nomenclature for turning flight: plan view (adapted from Hull, 2007).

of 303

Figure 3-3: Nomenclature for turning flight: front view (adopted from Hull, 2007)

In Figure 3-2 and Figure 3-3, is the bank and is heading angle. As shown in Figure

3-3, the resultant vector sum acting on the aircraft is obtained (Hull, 2007). The figure

shows the coordinate systems used in the derivation of the equations of motion that is a

restricted horizontal plane. The ground axes system Exyz is shown, but it is not in the

plane of the turn. The x-axis is in the original direction of motion. The subscript h refers

to the local horizon system, and the subscript w the wind axes system.

hjLTDF ]sin*sin)sin(cos)[(cos

hiLTD ]cos*sin)sin(cos)[(cos

hjLTW ]cos)sin([( (3.18)

of 303

3.2.1.3 General Functions

The purpose-related function describes the functions, responsibilities and processes that

influence the constraints at the abstract function level. In this case, at this level, the

pilot’s responsibility is to maintain separation standards at all times. However, under

this environment, pilots might be constrained by aircraft system dynamics from

achieving this objective (i.e., the purpose of an aircraft is to fly passengers to a

destination safely). To achieve this objective, it is often necessary for pilots to alter the

aircraft’s heading and speed as the flight progresses. To constrain the scope of this

study, collision avoidance options are limited to change of heading and speed; change

of altitude, though in reality an option available to the pilot, has been explicitly

excluded; that is, one aircraft was not ascending to descending into the flight path of

another. Heading and speed were restricted to lateral manoeuvre.

A heading change is a basic aircraft manoeuvre for avoiding conflict (Van Gent,

Hoekstra & Ruigrok, 1997). Turning manoeuvres are directed at changing the flight

paths (i.e., a heading change) as effectively as possible for lateral separation,

particularly when entering and exiting controlled airspace. At this level, airspace

dynamics for conflict avoidance can be described in terms of either a continuous or

discrete environment. In a discrete environment, changes to the environment, and pilots’

decision-making processes, occur at a discrete time. For example, a pilot operating in

this environment acts on coded messages such as “climb and maintain FL150 or reduce

speed to 200 knots” at a specific time during flights. These discrete decisions are

usually issued by the ATCO to enforce safe-separation standards. On the other hand,

continuous environments can be considered as free flight. The decisions and responses

of pilots will be in concordance with the continually changing environment. Pilots

receive no specific instructions from ATC under this flight environment. Actions such

as changing speed or heading, and/or application of certain operational procedures to

avoid obstacles can be applied at any time of free flight.

of 303

3.2.1.4 Physical Functions

The physical function specifies the individual parts from which a composite system is

made (including in this study weather conditions, obstacles, and/or air traffic, aircraft

and density) describes possibilities, characteristics, and properties of the system. This

level provides pilots with the information required to perform flying activities,

including lateral manoeuvres to avoid obstacles and collisions. Pilots are required to

manoeuvre their aircraft outside the Intruder’s protective boundary so as not to violate

minimum separation standards. Fundamental physical qualities of aircraft are relative

positions, velocities and protective zones, which are the key elements of conflict

geometry. The ability of pilots to change an aircraft's direction is constrained by its

manoeuvrability. An aircraft’s manoeuvring capability for lateral control is also

constrained by the minimum separation standard and wind speed. However,

understanding these constraints provides pilots with the mechanism to control and

maintain the desired flight path.

3.2.1.5 Physical Form

This level specifies the physical location of the speed and heading vector. In a free-

flight environment, ADB-S7 will display the relative positions of the aircraft and the

ground speed vectors of both the Ownship and the Intruder on the solution space8. The

physical presence and appearance of the Ownship, obstacles and the Intruder are

included at this level, which is where, consideration of basic parameters, such as the

bank angle, relative speed, and heading, is performed. A pilot’s action to avoid

obstacles is performed at a lowest level of the abstraction hierarchy. Table 3-2 and

Table 3-3 show the abstraction hierarchy for the two separate systems associated with

means-ends relationships to link the different representations of the flight collision

avoidance system.

7 Automatic Dependent Broadcasting -System 8 Solution Space (SS): the problem space modified to provide the solution state for the

flight collision avoidance system.

of 303

Table 3-2: Abstraction Hierarchy of Lateral flight

Function Purposes





Table 3-3: Abstraction Hierarchy of Flight Collision Avoidance System

Function Purposes





Manoeuvres Cruising

Elevator Ailerons Rudder

Automatic Dependent Surveillance – Broadcast (ADS–B)

Detect Loss of Separation Circumnavigation

Flight guidance

Dynamic equation of motion: Lift, drag, weight, thrust = ma

Yawing Speed Pitching Rolling

Protective Cone Physical Laws of Collision Avoidance

Relative Velocity Vector Intruder Relative Position

Ownship Relative Position

Ownship Heading

Ownship Airspeed

Intruder Heading

Intruder Airspeed

Distance to Ownship

of 303

The next section examines the modelling of the flight collision avoidance system. The

modelling is to address the system’s requirements for the uppermost level of abstraction

hierarchy. These requirements should relate to the system’s description as closely as

possible. The basic conflict tasks are required to address the system’s functional

requirements. The modelling uses a graphic representation to display conflict geometry

(i.e., conflict resolution), thus indicating aircraft relative velocity and protective cone

functions.

3.3. Flight Collision Avoidance System Modelling

An FCAS can be modelled, based on the EID approach, using a two-step process

(Burns, Bryant and Chalmers, 2005). The first step is based on Abstraction Hierarchy

(AH). The second addresses the physical interface and structure of the system. Firstly,

the five levels of Abstraction Hierarchy analysis show the physical constraints needed

to achieve related goals, such as conflict resolution. Secondly, pilots must perceive

information in such a way that it does not conflict with their mental models. The EID

allows pilots to develop accurate mental models of how the system works, thereby

guiding them in solving mid-air collision problems efficiently. The accuracy of conflict

information depends on the interpretation of digital versus clockface signals from on

board Radar (i.e., display). It has been shown that the clockface display was just as

useful a guide to altitude or speed as the digital display McCann and Foyle (1996).

However, digital signal representation is often accurate, but needs pilots to have a

mental “picture” of information being presented, as opposed to a clockface

displaysignal (Wickens, 2002b;). The combination of these two kinds of signals is

essential in acquiring situation awareness. A significant challenge is to develop a

display that enables pilots to correlate pieces of dynamic information at a glance. The

proposed display addresses this issue by using two-dimensional graphic representations

to make flight constraints visible on display, and support pilots’ performance in

avoiding collisions.

The modelling of a flight collision avoidance system requires judicious setting of the

system’s boundary (Burns, Bryant and Chalmers, 2005; Naikar et al., 2005). Naikar et

al. (2005) suggested the boundary of the analysis must be first defined, based on the

of 303

system’s objective, such as deciding which problems are to be solved. For example,

FCAS’s boundary for avoiding collision is determined by the physical system

representation within the environment in which a conflict situation is said to exist.

There are some key decision issues the analyst must take into account before analysing

a system’s boundary (Burns, Bryant and Chalmers, 2005). For example, these issues

that need to be addressed are the purposes of the system for which the system exists

within the environment, and the pilot’s role in the environment. In deciding parameters

for a collision avoiding system, boundary conditions have to be identified for both

pilots’ tactical and strategic conflict resolution (Tang et al., 2008), as shown in Table 3-

4, and discussed below.

Table 3-4: A system boundary for conflict resolution

Tactical Conflict Resolution State information

Strategy Conflict Resolution Intent information

1 Conflict awareness Situation – What is the current situation? A pairwise conflict? Evaluate the current situation.

2 Decisions to avoid conflict situations Path – a possible flight path to achieve goals/objectives (i.e., conflict resolutions).

3 Detailed manoeuvres to achieve objectives set by strategy

Plan – What properties are necessary and sufficient to achieve conflict resolution? Flight envelope (bank angle, speed, altitude etc.)

Pilots need to perform tactical manoeuvres during critical situations because of the

limited time available (Erzberger and Paielli, 2002). Tactical manoeuvres address

conflicts (by applying procedural manoeuvres such as the use of TCAS) for a period of

about three minutes into the future, while a strategic manoeuvre addresses conflicts

expected between three and twenty minutes into the future (Tang et al., 2008).

of 303

These procedural manoeuvres include changing aircraft heading, altitude or speed

(Paielli, 2011). This flight information is compared to information viewed on the

automation display system (Sarter, Mumaw and Wickens, 2007; Van Gent, Hoekstra

and Ruigrok, 1997), and these two sources of information should be consistent with the

pilot’s mental models of flying, and this consistency enables pilots to resolve conflict

tasks (Moray, 1998; Niessen and Eyferth, 2001). To have a collision avoidance display

system, such as Conflict Detection Traffic Information (CDTI), would allow pilots to

carry out “visual flight rules” manoeuvres to avoid protected zones or expected

obstacles (Thomas and Rantanen, 2006; Wickens, 2009) in addition to the “see and

avoid” technique (Zingale and Willenm, 2009). Thus, to avoid collision, two aircraft

behavioural functions based on geometries information are considered:

A protective cone function, and

relative motion function

3.3.1 Relative Motion Function

Consider a typical conflict scenario of two aircraft at the same altitude converging in the

vector diagram, as shown in Figure 3-4. The Ownship and Intruder’s track vectors are

represented in relative motion, as shown in the Figure, under no wind conditions. This

situation reduces to a problem of relative motion function, as shown in Equation 3.2.

The velocity of the Ownship ( ownv ) relative to the velocity of the Intruder ( intv ) is given

by:

intint/ vvv ownown

(3.1)

intint/ vvv ownown

(3.2)

of 303

Ownship

Intruder

Protective zone

N

ownvintv

int/ownv

intv

The aircraft will remain on a collision course, for as long as the pilots maintain a

constant speed and heading without implementing any conflict resolution. The

perception of the conflict situation, defined as a geometric vector, can be considered as

a constraint. Pilots perceive airspace geometry in four dimensions: (1) conflict angle,

(2) relative speed and altitude, (3) rate of change of relative altitude (Thomas and

Wickens, 2006), and (4) rate of turn (Andrews, 1978) and rate of change of speed.

Mogford (1997) discovered that aircraft heading and altitudes are important information

for conflict awareness, while information such as speed, position and aircraft call signs

are not used as much as expected for novice air traffic controllers to maintain situation

awareness. Mogford suggested that an analysis of conflict detection tests is further

needed to improve ATCOs' performance of high priority tasks. There are four key

pieces of information that should be considered if separation conflicts are to be

identified (i.e., the pilot needs this information to avoid collision):

In level flight; at the same altitude?

Conflict angle; on a converging angle?

Relative speeds; showing impending conflict?

Relative positions; showing impending conflict?

Figure 3-4: A typical 2D conflict scenario for lateral separation

of 303

Studies conducted by Morris (2005) and Rantanen et al. (2004) revealed that regardless

of the flight phase under consideration, pilots find it difficult to detect aircraft on a

converging collision course. As the grain of texture gets smaller, the aircraft gradually

diminishes into distance (i.e., - that the further away a potential intruder is, the smaller it

appears to the pilot). Pilots can perceive this as surfaces moving further away from a

previous position. When the conflict angle between the Intruder and Ownship remains

constant, it could mean that the Intruder is tracking towards, or away from the Ownship.

Because the Intruder appears small and motionless and is blended within the

environment, the Intruder may remain invisible or unnoticed for some time before a

collision (Morris, 2005). It is difficult to assess an Intruder’s manoeuvres by using the

“see and avoid” technique alone when it is a heavy aircraft at slow speed. For pilots to

perform self-separation manoeuvres in a free-flight environment, they need a supportive

“tool” that clearly shows conflict geometry to assist them (Rantanen et al., 2004).

The next sections will examine how the relative motion function should be incorporated

with protective cone function, so that the information presented is not communicated by

the influence of higher cognitive processes.

3.3.2 Protective Cone Function

To prevent the aircraft from drifting off course, the heading flown takes into account the

wind vector. Consequently, Air Traffic Controllers use vector information to guide

pilots onto a specified flight path (e.g. turn right to heading 070, or turn left to heading

220). Computation of an airborne-aircraft's heading to change a flight path to avoid

obstacles can be quite complex due to computing aircraft angles and the distances

between the aircraft and obstacles.

Resolving conflict or intercepting with the original waypoints at the required radius of

turn can be challenging for pilots for two reasons. Firstly, choosing the correct rate of

turn for a heading change, while travelling at high speed, could challenge the pilots

setting up and operating the flight management system (Wickens, 1995). Secondly,

pilots’ decisions to proceed with, or to delay the initiation of, the selected roll

manoeuvre could lead to violating or avoiding protective zones.

of 303

These two variables pose a challenge to unaided pilots seeking to precisely coordinate

the avoidance of obstacles. In visual flight rules, avoiding a flight over rocky terrain,

avoiding a mid-air collision, or the increasing size of background images in the sky

(such as cloud patterns) may obscure obstacles along the flight path and so cause a

sudden change in heading due to a sudden change in environmental features. Pilots in

this circumstance may perform inappropriate control actions without accurate

information of the relative positions of those hazards. This effect may well have played

a part in most mid-air collision incidents or accidents (Morris, 2005). Rushton and Duke

(2009) indicated that having time-to-contact improves observers' accuracy at judging

relative distances in space with the use of relative information. Thus, the future position

of a moving object can be estimated, rather than using the absolute speed information.

This suggests that observers are not better at making distance judgments in space with

the support of absolute speed information.

In geometric measurements, the protected cone function is defined as an angle ( )

subtended by two rays tangential to the Intruder’s airspace protective zone (refer to

Figure 3-5). The protective cone function used in the FCAS is a directional projection

of the pilots’ forward field of view in space. Visual information outside this view is not

important to pilots. For example, to artificially produce a light beam, a lamp and a

parabolic reflector are used in many lighting devices, such as car headlights Or aircraft

landing lights. The beam is spread at a specified angle. Car drivers driving at night are

only interested in the directional projection of a light beam to identify obstacles for

them to avoid, and, more generally, to show them the roadway as they travel to their

destination. Similarly, an Instrument Landing System (ILS) is used to provide a glide

path for an aircraft approaching a landing strip with minimum visibility (Owen, 1993).

A protective cone function, introduced by Chakravarthy and Ghose (1998), has been

adopted in several studies of instruments for conflict resolution (Paielli, 2003; Van

Dam, Mulder and van Pusan, 2008; Bach, Farrell and Erzberger, 2009; Ellerbroek et al.,

2009). However, these studies use the protective cone as a geometry conflict resolution

model, and have not incorporated it as part of a collision avoidance display (i.e., a

physical system), which is how the protective cone presented in this study operates to

guide pilots to avoid obstacles.

of 303

Intruder

Ownship

N

ownvintv

int/ownv

intv

Protective Cone

Protective zone

For pilots to effectively perform their collision avoidance activity, they need to know

how far the Intruder is from the Ownship, which way to turn to avoid collision, and

what the approaching angle is. To guide pilots to perform self-separation, FCAS makes

use of a protective cone function and relative motion function, as shown in Figure 3-5,

because when geometric properties (i.e., relative velocity vectors and protective cone

functions) are grouped into more meaningful collections of information, the grouped

information is assumed to be easily understood, quickly acted upon and proactively

presented (Treisman, 1982).

Paielli (2003) proposed a simple numerical algorithm to determine the speed or heading

changes for conflict resolution. According to Paielli, conflict involving more than three

aircraft is rare, and algorithms addressing it have no practical value. Paielli pointed out

that a separate procedure is required to cope with multi-aircraft conflicts, if pairwise

models are incapable of providing acceptable resolutions. Andrews (1978) presented a

mathematical model for determining the importance of a turn to resolve conflict in a

horizontal plane. The model suggested that the preferred manoeuvre for lateral

separation would be for the faster aircraft )( OwnV

to avoid conflict. According to

Andrews, modelling horizontal resolution assumes that the Ownship flies faster than the

Intruder, and thus, it manoeuvres to avoid the risk of conflict.

Figure 3-5: A typical 2D conflict scenario with the protective cone

of 303

Farrell and Erzberger (2009) presented an algorithm to compute conflict avoidance in

the horizontal plane. According to Farrell and Erzberger, to avoid conflict, a conflict

resolution will require the Ownship )( OwnV

to obtain a new relative velocity )( resR

V

by

changing its heading; a heading change is a basic aircraft manoeuvre to avoid conflict

(Van Gent, Hoekstra and Ruigrok, 1997). By rotating the relative velocity )( resR

V

vector

in a clockwise or anticlockwise direction ( i.e., correspond to a change in roll), it is

possible to at least align the relative velocity )( resR

V

tangential to the Intruder protective

zone (Van Dam, Mulder & van Pusan, 2008; Bach, Farrell & Erzberger, 2009) while

maintaining the same speed ( ownv ). Therefore, the new heading is expressed as:

*int

1** sinsin RRownship (3.3)

Where, int*v is the new Ownship heading, ownshipo VV

/int , *

R is the new relative

velocity, and int is the Intruder’s heading. However, a larger magnitude of change of

the aircraft’s heading could lead to exceed the required heading change by more than –

2µ (β-α) for the turn-back point (Bach, Farrell and Erzberger, 2009). This constraint

needs to be visualised to avoid re-entering the conflict zone. Speed changes are

modelled, as both the Intruder and the Ownship’s heading remained constant (Paielli,

2003).

o (3.4)

Where o is the initial angle of relative velocity, osS /sin min1 and if os is less

than minS , conflict cannot be resolved. Equation 3.5 gives us two speed solutions for

conflict resolution, one slow and high-speed (Paielli, 2003).

Ownship

OwnshipVV

sinsin intint

(3.5)

Where OwnshipV

is the Ownship speed, is the ownship is the Ownship heading, and int

is the Intruder’s heading.

of 303

Figure 3-6 shows a heading change resolution performed by the Ownship )( OwnV

,

supported by the protective cone.

The flight collision avoidance system proposed functions were changed to interface

representations by semantic mapping processes (Vernon, Reising and Sanderson, 2002).

The process mapped the information to be displayed in visual form in order to support

the pilots to perform safe-separation tasks and take effective action at the physical level.

According to Montello (1997), perception and cognition of relative distance is relevant

to a wide range of systematic arrangements of pilots’ spatial behaviour. Pilots’

behaviour is shaped by a thorough examination of information presented by the cockpit

display, checklists and placards (Hutchins, 1995). Pilots ideally need to perceive how

far, or close, they are to the Intruder, in order to gain sufficient situation awareness for

resolving the conflict.

Pilots are trained to acquire and interpret the incoming information as long as it is

consistent with their expectations. With respect to geometry of conflict, pilots should be

able to see aircraft approaching when they expect to. However, the currently available

systems do not allow pilots to confirm their perceptions and interpretations of the

incoming information when using a “see and avoid” technique for mid-air conflicts.

{

dM

N

Ownship

Intruder

Protective Cone*

*ownship

int/*

ownV

ownv

int/ownv

Figure 3-6: A typical 2D conflict resolution scenario for lateral separation

of 303

Thus, pilots will reject any information that contradicts with their mental models of not

being able to see the aircraft approaching (Morris, 2005). According to Morris, this

would affect pilots’ physical and behavioural constraints. Pilots’ behaviour in relation to

how they perceive and use both relative motion and protective cone function

information supports situation awareness and decision-making.

Perception of the stimulus within the environment may not have to depend on prior

knowledge or past experience; however, it depends on the concept of affordance (i.e.,

the relationship between the environment and the users that supports users to naturally

perform an action). According to Gibson (1979), ecology is regarded as a relationship

between an object and an environment that affords the opportunity to perform an action,

for example, to circumnavigate obstacles. Gibson clearly and distinctly presented a

theory of direct perception in which he argued that perception is a direct "What you see

is what you get" phenomenon. According to this theory, the environments provided

sufficient information that is relevant to an understanding of our visual sensory systems

to directly perceive the information present in the stimulus (e.g. size, shape, distance,

etc.).

Visual information obtained by pilots is relative to the environment, and dynamically

changing with ambient optic arrays (i.e., provides explicit information about the layout

of objects in space). Gibson’s theory of perception yields three components of optic

arrays: optic flow patterns, invariant features, and affordances. Gibson widely used the

phrase “optic flow patterns” to give a representation of visual awareness. For example,

aircraft approaching in the pilot's field of view may appear to be stationary, with the rest

of the visual environment moving away from that aircraft. According to Gibson, such

optic flow patterns can support pilots with clearly defined information about their

heading, speed and altitude, in relation to movement in space. Each representation is an

arrangement of light, which is determined by the environment, and thus affords

information about the objects, relative to an observer’s movement (Gibson, 1979).

of 303

A parameter of great significance for representing a conflict is the predicted time-to-

contact (Bach, Farrell and Erzberger, 2009). However, if the moving object has a time-

to-contact, this information can be used to project the future distance (Rushton and

Duke, 2009). Though, by definition, that future distance will be 0 at the point of contact.

A system that displays information, such as absolute distance or speed, may not be

enough to perform these conflict avoidance tasks.

However, an instrument that displays constraints, such as being too close to or too far

from an obstacle, can provide pilots with situation awareness and aid in decision-

making processes (Stanard, et al, 1996). For example, displaying a circle, which

indicates a protective zone about the Intruder, can prompt the retrieval of a series of

declarative associations from pilots’ mental models (Lang, Davis, Ohman and Arne,

2009). The circle motion (e.g. displacement) cues can also activate time-to-collision.

For example, the intersection of the Intruder’s protective zone with the Ownship’s

protective zone can clearly activate time-to-collision cues to support collision avoidance

tasks.

According to McLeod and Boss (1983), there are two approaches pilots can use to

visually obtain time-to-contact information from the environment. The first approach is

in line with the ecological optic array (i.e., directly from the changing optic array). With

this approach, time-to-collision is specified by the relative rate of expansion of the

retinal image over time. Pilots only need to watch an Intruder approaching on a

collision course with little or no interpretation. The second is a cognitive approach,

which is derived from low-order information, as expressed in Equation 3.7.

Lee (1976) presented a theory of visual control of braking for car drivers. The theory is

based on visual information about time-to-collision. According to the theory, drivers

subconsciously derive time-to-contact information from distance, speed before

collision. Furthermore, drivers use time-to-contact information to register when they are

on a collision course, or when they must start or delay braking to avoid a collision.

Similarly, reliance on visual information in a free-flight environment may impose

demands on situation awareness, built around the flight constraints of each aircraft.

Safe-separation in this environment is based around time-to-collision, instead of fixed

metric separation provided by protected cones (Wickens, 1995).

of 303

The protected cone function will expand, as the distance to the Intruder becomes less

than the minimum separation standards. As a result, pilots can perceive rapid expansion

of the protected cone as conflict becomes imminent. The perceived cues activate the

time-to-collision . Thus, represents the observation time as the Intruder is

approaching the Ownship, as expressed in Equation 3.6.

intVV

d

Own

(3.6)

Where is the predicted time to reached loss of separation ( Rt

Rt VV

21

) OwnshipV

and

in tV

is the Ownship and Intruder groundspeed, respectively, d ( AOwnship ) is the

distance to first loss of separation along the relative velocity vector (refer to Figure 3-7).

The rate of expansion of the protective cone, which relies on distance change

information, is derived from the following equation:

osS /sin min1 (3.7)

IntOwn

min1

VV*sin

S (3.8)

os

Ownship

Protective Zone

Intruder

1

RtV

2

RtV 3

RtV

RtV

Protective Cone

N

ownv intv

intv

Figure 3-7: Relative motion of aircraft on a collision course

of 303

In Figure 3-7, is the rate of expansion of the protective cone and minS is the

minimum separation standard, while os is the line of sight (Ownship - Intruder).

The next section examines the systematic confirmation of the model that requires the

use of criteria governed by scenario mapping (Burns, Bryant and Bruce, 2001).

3.4. Flight Collision Avoidance System Mapping

This section discusses the rationale behind the use of the attentional theories, and the

way these theories are related to the development of the FCAS (i.e., solution space). It

goes on to discuss the decision ladder to support pilots’ behaviour in decision-making.

As Amelink et al. (2003) suggested, if constraints are mapped into an interface that is

based on abstraction hierarchy modelling, pilots' mental models could be captured and

externalised to improve situation awareness. With well-mapped constraints, a pilot

might be able to instantly predict the possible future state of the system in conflict

situations. Flight information is obtained from different instruments across the cockpit

(Hutchins, 1995), to enable pilots to develop situation awareness (Endsley, 1995, 2000).

This requires pilots’ visual attention to “effectively” scan and extract information from

these instruments across the cockpit.

Thus, it is possible to miss an important piece of information (Nikolic, Orr and Sarter,

2004). If a piece of information that is relevant to resolve conflicts such as miss distance

is not presented saliently enough, pilots are likely to drift further away from (or too

close to) the intended target, as compared with pilots equipped with a display that

provides accurate information.

To support the presentation of conflict information on the display, the following

discussion on attentional theories is presented as a means to satisfy one of the conflict

avoidance requirements needed to make flight constrains “visible” to pilots. Therefore,

EID might support pilots to resolve a collision related problem by providing a visual

representation of the problem, highlighting the relationships between the perceptual

cues and constraints. Accordingly, WDA represents the constraints of both functional

properties and physical systems that enable pilots to perform a task within these

constraints (Vicente, 2002), as shown in Table 3-5.

of 303

In Table 3-5 (a; d), different basic features of visual information on the display, such as

colour, size, line-ends, closure and orientation, are all extracted automatically (i.e.,

quickly and effortlessly) on the pilot’s entire visual field of view. For example, the air

traffic controller’s display uses flash coding to draw attention to situations requiring

immediate attention (Yuditsky et al., 2002). Remington et al. (2000) also discovered

that air traffic controllers could rapidly identify traffic conflicts when aircraft altitude is

colour coded. Controllers’ performance is enhanced by the use of simple features to

make selective attention more efficient. A sudden movement of an object can also

capture attention in a visual search task (Franconeri and Simons, 2003). For example, if

critical target information, such as the two movements of protective zones approaching

(i.e., Ownship and Intruder), is identified, it is likely to be detected quickly and

efficiently, as indicated in Table 3-5 (a). Because the Ownship and Intruder share the

same boundary as a single object, it allows a pilot’s attention to select its representation.

The advantage afforded to the pilots in detecting the Intruder position (e.g. location)

relative to the Ownship is enhanced when it is presented as a single object, as it is

assumed they share the same conflict boundary (Marino and Scholl, 2005). It is

suggested here that a collision avoidance system displaying two separate objects should

have a representation as a unified single object, or an entity unto itself. Table 3-5 shows

for example, a pilot’s attentional system being directed to a location within the

boundary. Objects within the boundary, such as the position of the line of sight also

acquire an attentional advantage (via enhanced sensory processing) (Chen and Cave,

2006) as expressed in Figure 3-8.

of 303

(a)

.

(b)

Figure 3-8: The problem (a) space is modified to provide the solution (b) space

Ownship

Intruder

Protective Cone Ownship

Intruder

Protective zone

N

ownvintv

int/ownv

intv

Line of sight

Boundary

Protective zone

N

ownvintv

int/ownv

intv

Line of sight

of 303

The feature integration theory, developed by Treisman and Gelade (1980), has made a

significant contribution, especially for the visual modality of selective attention. In

Table 3-5 (b; g), according to this theory, a single feature, such as triangulation, can

define critical target information and searching within the triangulation is efficient.

Table 3-5 (e) also presents how selective attention theories have suggested that pilots

have a tendency to serially process or concentrate on a piece of information, while

ignoring other available information.

Broadbent's (1982) selective attention theory explains how people use attentional

resources to select certain information within the environment, on the basis of physical

characteristics only, and select input for attention. More specifically, Broadbent's model

suggested that people use selective attention to notice relevant (i.e., salient) information

about their environment from the vast amount of information available. For example,

pilots can rotate the relative velocity vector out of the protective cone to avoid a

collision. This concept has been applied in aircraft displays. Seagull and Gopher (1994)

examined the used of Helmet Mounted flight Displays (HMD) with the “window field-

of-view” to improve the pilot’s performance. They reported that pilots supported by

HMD had fewer aircraft crashes, but increased their overall head movements.

To support development of FCAS, selected domain invariants and flight parameters are

mapped onto the final ecological interface design as suggested by attentional theories

previously discussed and as shown in Table 3-5. Lau and Jamieson (2006) conducted a

similar study by mapping domain features of invariance based on work domain

analysis. Lau and Jamieson state that the WDA has highlighted the key domain

characteristics of the condenser subsystems, some of which can be expressed in generic

design concepts that may transfer to other work domains.

of 303

Table 3-5: Some selected features of domain invariants based on work domain analysis

Process Description

Geometric Invariants Levels of Abstraction Hierarchy

Geometric Forms From Figure 3-10

Protected cone to provide collision avoidance.

The protected zone should be the same.

Physical Form d

Heading and bank angle indicators are related to one another.

Abstract functions k

The relative velocity tip must lie inside the protective cone at a given conflict.

Physical Form e

The relative velocity tip must be inside the protective cone at a given conflict.

Generalised/ Physical Form

e

Ownship and Intruder vectors should denote a triangulation operation.

Generalised/ Physical Form

b

g

The circle (Yellow) represents 5 nm of protected zone limits centred around Ownship and Intruder.

Abstract functions/ Physical Form

d

Ownship and Intruder’s speed vectors set the conditions of the resolution process.

Abstract functions b

g

of 303

The second component of CWA, Control Task Analysis (ConTA), determines what

activity needs to be performed, within a system, by an individual agent or computer

systems. The components of ConTA consist of both activity analysis in the work

domain and decision-making (Rasmussen et al, 1994). The activity analysis in the work

domain has been discussed extensively previously and has highlighted the

environmental constraints and information necessary to map a pilot’s decision-making,

but it does not illustrate how the features of the work domain are used over a period of

time to support control tasks in response to situations. For this, activity analysis in

decision-making determines the skills, rules and knowledge-based behaviour of pilots’

activity or control tasks that produce changes towards a goal. The decision ladder

developed by Rasmussen, Pejtersen and Goodstein (1994) is a “map” that can represent

a pilot's path to making a decision. As suggested by Rasmussen et al., a pilot’s decision-

making behaviour can be mapped onto the decision ladder’s template. The decision

ladder captures pilots’ states of knowledge, and information-processing activities

necessary to reach a decision.

The decision ladder in Figure 3-9, demonstrates what knowledge and information pilots

need during the decision-making process. The process of mapping cognitive strategies

onto the decision ladder provides a structure that shows what knowledge and

information pilots use during this decision-making. The process describes the pilot’s

behaviour in acquiring and analysing information before implementing and executing a

choice of actions. However, pilots may not be able to acquire, analyse, evaluate and

implement all relevant courses of action "in the heat of the moment" as their cognitive

capabilities are limited. Thus, the decision-making process must be simplified. In

reality, simplifying the decision-making process may be more effective and efficient

when a specific model is used than it would be with a generic model. For example,

according to studies conducted by Rasmussen (1974) and Klein (1989), expert pilots

tend to shunt across the decision ladder to reduce cognitive activity while novices adopt

a sequential approach. According to Xiao and Seagull (1999), as much as pilots prefer

to use an automatic mode of behaviour (such as SBB, discussed below); they often

perform different tasks at three levels of behaviour.

of 303

The following subsections will discuss pilots’ cognitive models, in relation to the three

levels of pilot’s behaviour in (Rasmussen, 1983), namely Skills, Rules, and Knowledge-

based behaviour.

3.4.1 Supporting Skill-Based Behaviour (SBB)

Pilots’ skill-based behaviour involves mainly physical actions with no conscious

control, or without any monitoring of conscious attention. At this level, the display

should be designed to capture pilots’ interaction between the perceived information and

the action performed. Thus, pilots should be able to manipulate the controls of a display

directly, requiring less physical and/or cognitive effort to accomplish or understand the

task. SBB should support some, if not all of the information needed at the level of

Knowledge Based Behaviour (KBB, discussed below). Pilots’ familiarity with the

environment minimises the cognitive resources to interact with it (Wickens and

Hollands, 2000).Skills can also be acquired in several ways. For example, a pilot may

consciously pay attention to the individual parts of the tasks being performed until

his/her behaviour is smooth and “pre-programmed”. Once the skill is acquired, pilots

may find it difficult to pass this skill on to team members, or articulate it. Pilots' skill-

based behaviour may also need some feedback from the environment in much the same

way that driving a car still needs experienced drivers (operating out of skills-based

behaviour) to obey road signs. However, under high cognitive load, pilot situation

awareness may change, and warning cues (i.e., visually or auditory) may go unchecked.

Thus, pilots operating out of SBB may fail to apply an appropriate step to correct a

particularly abnormal operation of the system.

Another example of skill-based behaviour is when pilots perform repetitive tasks, such

as performing a checklist in the cockpit. If the checklist is repeated many times, it is

possible for pilots to perceive the present task as already completed, when in fact, it was

previously performed and completed on the same type of aircraft with the same

configuration, but at a different time. This activity can affect the pilots’ decision-

making by planting “false memories”. For example when the checklist gets changed

(i.e. a new version is issued); pilots operating out of habit may inadvertently skip the

additional steps / not do the modified ones properly.

of 303

3.4.2 Supporting Rule-based behaviour (RBB)

Pilots’ rule-based behaviour (i.e., procedural behaviour) describes performance related

to routines with some conscious and standardised behaviour patterns. (RBB contrasts

with SBB in that the latter is mostly subconscious, whereas the former has a conscious

component to it). This type of pilot behaviour involves practicing, sometimes-automatic

behaviour with a well-established set of rules or procedures to check a system’s

progress. For example, a checklist placed in the cockpit encourages rule-based

behaviour. If expert pilots look at a familiar checklist, they can assume what the

checklist entails, without actually performing the tasks on it.

Phrase differently, pilots’ rule-based behaviour uses conditions such as “When, If……

Then” for selecting an appropriate action, based on sets of signals, rules or procedures.

These sets of rules and procedures can also be formed from pilots’ past experience

when operating a specific type of aircraft system that requires application of a specific

set of rules or procedures in a particular situation. These sets of rules or procedures are

system constraints. RBB is also suitable for automated system applications, as long as

all the possible sets of rules or procedures can be codified. However, from a pilot’s

perception, it is practically impossible to examine a priori all the possible solutions to

every emergency situation. In emergency situations, pilots may be prevented from

interacting with the system due to a lack of knowledge of the system’s current state,

thus keeping the pilots out of the control loop (Endsley and Kiris, 1995a; Kaber and

Endsley, 1997).

3.4.3 Supporting Knowledge-based Behaviour (KBB)

Pilots’ knowledge-based behaviour (KBB) involves complex problem-solving. This

behaviour allows pilots to identify, interpret, and ascertain system status in normal or

abnormal situations. The activity of pilots at this level involves planning, adjusting,

creating and implementing new solutions to an unexpected problem. It is the highest

cognitive level of the pilots, and is applied to novel situations in a wide range flight

conditions. A pilot’s interaction within the environment is consciously controlled. With

knowledge-based behaviour (KBB), the pilot’s mental models represent deeper internal

structures of the environment.

of 303

Mental models are built, based on the acquisition and analysis of information from the

environment, to formulate goals and plans to handle the current events. Mental models

would also come with experience.

Knowledge-based behaviour in work domains is an outward representation that reflects

mental models, and is independent of the abstract cognitive process performed by pilots.

To evaluate the future trajectory of aircraft on a collision course, pilots need to operate

at the level of knowledge-based behaviour (Rasmussen, 1983). The acquired knowledge

at this level enables pilots in this situation to predict the future trajectory of aircraft and

so their ability to avoid conflicts is determined by the aircraft's manoeuvrability.

In contrast to Rule-Based Behaviour, pilot behaviour often changes, based on the

current situation. Stored sets of procedures and rules no longer necessarily apply, but

pilots can still access these sets of information when needed. If a novel situation is

presented, a new plan must be developed to solve the problem. In most cases, solutions

to these new plans are based on “trial and error” (Rasmussen, 1983). These new plans

are prone to human error, yet pilots will tend to avoid knowledge-based behaviour, due

to the high cognitive workload in trying to analyse novel events. For example, when

coping with complex problems such as unexpected engine failure, pilots tend to shut

down the failed engine without proper diagnostics, thus, applying RBB (Wright, Pocock

and Fields, 1998). Figure 3-9 shows the decision ladder for a pilot's decision to avoid a

potential mid-air collision. In general, the figure illustrates the process of mapping

pilots’ mental strategies onto the decision ladder. Pilots using KBB use an analytical

approach to base their decision on how the problem is structured. The analytical

approach weighs the relationship between what options are available and the results.

For example of a combined RBB plus KBB solution to an issue, a pilot may

experience an issue while at cruising altitude descended to below 10,000 feet (so that

everyone onboard of the aircraft could breathe without oxygen) so as to buy time to

figure out what to do next.

of 303

Diagnose state

Alert

Conflict detection

Infor- mation

Choice of task

Target state

Execution

Proce-dure

Plan of action

Evaluate Options

Predict Consequences

Goal chosen

Options

Goals

State state

Task

1

3

1

2

4

5

6

1

7

8

SBB

RBB

RBB

KBB

Is there a line of sight?

Conflict detected by crew

Where is the intruder location? What is intruder’s heading? What are boundary conditions? What are the rules of avoidance collision?

How should the pilot manoeuvre to avoid collision? What additional sources of data are available?

What steps are needed to engage the target?

Aircraft constraints

Possible choice of actions

Observation

Information

processes

activities

State of

knowledge

between

activities

activities

Figure 3-9: Decision ladder for the flight collision avoidance display

of 303

Wickens and Gempler et al. (2000) pointed out that with the nature of pilots’ behaviour,

it is impossible to design a decision system to support their decision-making in all air

traffic scenarios. However, according to Rasmussen et al. (1985), pilots use common

sense rules (or simple rules) intended to increase the likelihood of solving the problem,

for example, avoiding aircraft on a collision course.

The application of a simple rule in this situation may reduce the pilot cognitive

workload by rejecting the information they consider as not important. There is a risk

that in doing this what they discard is important. And that's a consideration in support of

an automated FCAS that can always take these things into account. This information

may include, for example, weather cells, fuel, desired flight path, restricted airspace and

separation. For example, maybe in the event of toxic fumes in the cabin, the action

would be to conduct an emergency landing at the nearest suitable airfield, so the pilot

will disregard fuel level and desired flight path. Rasmussen and Goodstein pointed out

that to maintain situation awareness, the current alerting systems are designed to

indicate two or more alert levels by using different colours or auditory commands.

Contrary to this approach, the proposed new collision avoidance display system is able

to continue to present the evolution of conflicts over a period of time as execution is

performed. Thus, the pilot is instantly informed as to whether his or her actions are

effective in solving the problem.

To support development of FCAS, the anticipated benefits of visualisation of the skills,

rules, and knowledge-based behaviour, captured in the forms for graphical expression

are mapped onto the protective cone and relative motion functions as shown in Table 3-

6 and Table 3-7 while Figure 3-10 presents the FCAS.

of 303

Table 3-6: Function and conceptual advantage of the graphic forms in the ecological interface

Graphic Form

From Figure 3-10

Description and Function Conceptual Benefits

c

The white line depicts the Intruder’s vector.

(i.e., velocity and heading)

Provides an interpretation, and leads to only one conclusion:

Illustrates future changes in vector (SBB), rather than numerical computation and mental simulation (KBB).

Illustrates the extent of a vector with a line (SBB), rather than a mental simulation of numerical values (KBB).

l A miniature image of the aircraft is centred on the navigational compass.

Provides an unambiguous aircraft's heading indicator (SBB).

b The dark red vertical line depicts the Ownship’s vector. A change of the line length indicates speed change and wind vector.

Provides an interpretation, and leads to only one conclusion: increase or decrease in length (SBB), rather than numerical computation (KBB).

a Two blue lines, tangential to the Intruder’s protective zone limits, form the cone-shaped zone.

Illustrates an interpretation, and leads to only one conclusion (SBB).

The angle of connecting the two blue lines expands perceptually to illustrate the rate of change for the LOSs.

Illustrates the expansion of the protective cone (SBB).

of 303

Table 3-7: Function and conceptual advantage of the graphic forms in the ecological interface

Graphic Form

From Figure 3-10

Description and Function Conceptual Benefits

f The white line connecting the Ownship and the Intruder is depicting the Loss of Separation (LOS).

Illustrates robustness of the system in maintaining separation (SBB).

Illustrates deviation of the actual flight path spatially (SBB), rather than numerical computation (KBB).

d The circle represents 5nm of protected zone limits centred on Ownship and Intruder.

Provides appropriate constraints for heading and bank angle indicators (SBB).

d The yellow diamond depicts air conflict.

Provides an indication of a conflict between the two aircraft in conflict (SBB).

j The curved green line represents the bank angle. This should be the same under the normal bank angle range. Yellow is caution, and red is the limit

Illustrates normal or abnormal operations of the bank angle.

e The velocity vector of Ownship relative to Intruder is given by the red vector. The objective is to move the tip of the relative velocity vector out of the protective cone to obtain a new relative velocity, required to avoid the risk of collision.

Provides two possible solutions to avoid conflict, by at least aligning the relative velocity vector tangential to the Intruder protective zone, by rotating the relative velocity clockwise or anticlockwise (RBB) rather than using mental simulation (KBB).

of 303

3.5. Summary

Studies analyse ways to model and evaluate human-system interactions with a view to

design and improve interfaces for complex systems. Gibson's ecological principle

highlights the significant importance of the correlation between the individual and the

environment in a bottom-up approach. The Flight Collision Avoidance System (FCAS)

handles both top-down and bottom-up approaches that involve perception based on the

principles of the Ecological Interface Design (EID) approach. The bottom-up approach

promotes affordance that allows users to perform an action. The top-down approach

represents deeper information of the system at a higher level of the abstraction

hierarchy. The AH and the Skills, Rules, Knowledge-based behaviour are the core

components of the EID framework. The AH constitutes five levels of human cognitive

activities, that are interrelated through “means-ends” mapping and present and describe

the “how–what-why”. Pilots’ mental models are captured and externalised on AH to

improve situation awareness, and cope with complexity and dynamic environments.

Figure 3-10: Flight Collision Avoidance System (FCAS) for horizontal separation

of 303

In situations where information is not reliable, where events are difficult to predict, and

there are multiple simultaneous objectives, pilots’ performance may be time

constrained.

The Cognitive Work Analysis (CWA) and attention theories have been studied

independently. The CWA theoretical framework and attentional theories address the

problems of the development of a novel FCAS. In view of this, the chapter presented

two aircraft behavioural functions (i.e., protective cone and relative motion) to support

the modelling and development of the FCAS so as to demonstrate the usefulness of

integrating and visualising the dynamics of multiple elements, such as relative motion

function, protected cone function and protected zones. These elements can be grouped

into more meaningful collections of information to support object-based attention. The

grouping allows pilots to perform effortless, quick and pre-attentive tasks. These

particular properties of attention in processing information will greatly improve the

intuitiveness of interface representations, producing a faster and more natural way of

acquiring information from the environment, thereby speeding up pilots' active learning

of the system and enhancing their experience of it.

The current Ecological Interface Design displays for conflict avoidance do not

adequately map the relationship between these constraints to show the geometry of

conflict and operational constraints. The FCAS displays graphically, and as a result it

displays the state of relative velocity vector and protective cone which is intended to

provide pilots with both low and high level information with direct-action capabilities,

an understanding of the conflict and situation awareness. The next chapter will

investigate the reliability and usability of the FCAS.

of 303

Chapter 4. Research Design and Method

Chapter 3 presented the FCAS based on the Ecological Interface Design approach and

Chapter 4 is concerned with methodological issues involved in evaluating the

effectiveness of a graphic representation of conflict geometry in the FCAS. Both

qualitative and quantitative approaches were used to evaluate the expected effect of the

FCAS on pilots’ decision-making. A quantitative approach measured dependent

variables such as pilots’ mental workload, task performance (i.e., deviating to avoid

collision) and situation awareness. The Qualitative approach was used to elicit, and

capture, pilots’ mental models by assessing the usability, and understanding perceptions

of, the FCAS, through interviews. The methods for data collection used in the current

study were questionnaires, simulators and interview methods and a between-subjects

design was used. Descriptive statistics summarised the performance of the experimental

and control group, and inferential statistics were used to determine any statistically

significant differences between these groups. This chapter will outline the methods of

data collection, experimental methodology, ethical considerations, independent and

dependent variables, and a framework methodology for data analysis.

of 303


of 303

4.1. Introduction

The goal of free-flight concepts is to ensure that pilots have the freedom and

responsibility to maintain safe separations. The purpose of this thesis is to develop and

test a Flight Collision Avoidance System (FCAS) designed to maintain safe separations.

The FCAS has been developed with the aim to improve situation awareness, reduce

mental workload and support decision-making in a free-flight environment (see Chapter

3).

To address the research objective as stated in Chapter 1, this chapter outlines the

research design and method used in the current study. In order to maximise the validity

of responses in answering any given research question, Bryman (2006) suggests using

both qualitative and quantitative approaches, to gain both objective and subjective

measures. Therefore, the current research design was based on this mixed approach.

The qualitative method is particularly useful in answering open-ended questions such as

“Why did the participant fly the aircraft behind or in front of the intruder?” Answers to

these interview questions were recorded using an electronic tape to elicit participant's

mental models (Van Someren, Barnard and Sandberg, 1994, p. 26). Adopting only a

qualitative approach and excluding a quantitative approach risks leaving out vital

information from the overall analyses. According to Hunt and Lavoie (2011), neither of

the two approaches can, by themselves, capture, describe and explain, in detail, the state

of things as they actually exist. For example, qualitative studies do not interfere with the

natural behaviour of participants being examined. Thus, narrative reported data, with

contextual descriptions, and direct quotations, are obtained from participants.

Alternatively, quantitative studies and data, are reported through the statistical

significance of any findings, such as, pilots’ performance ratings.

An essential part of quantitative research is a collection method of data, such as from

experimental t-test, causal-comparative, correlation, or multivariate analysis of variance

(MANOVA).

of 303

This study examined the influence of the FCAS on the task performance in two groups

of pilots, experimental and control, and the differences between these groups were

examined. A between-subjects design was used for the present study, a way to prevent

carry-over effects. All twenty-one (21) pilots enrolled for this study were males and

were asked to complete the pre-and post-questionnaires. Participants ranged from 18 to

90 years of age. Participants were recruited on a voluntary basis from the Swinburne

and Aviation Community via advertisements on campuses. Three instruments, semi-

structured interview, Likert Scale Questionnaires (see Appendix E and F) and flight

scenarios, were employed. The three independent variables used in the current study

were air traffic density, conflict avoidance display, and conflict geometry.

The flight simulator consisted of a standard computer monitor with steering wheel and

rudder pedals. As Prinzel et al., (2002) state, flight simulator results can be compared

with the real aircraft experimental results. This indicates that it is possible to have a

well-designed flight simulator experiment that is comparable to real flight situations.

Three conflict scenario approach manipulations were targeted in the current study

including head-on, starboard and portboard. Each participant performed practice trials

and final exercises. A number of dependent measures were used in the study, such as

heading change manoeuvre and miss distance. Six-dimensions of subjective mental

workload (see Appendix F) ratings were used to check the overall mental workload

ratings of tasks (Hart and Staveland, 1988).

of 303

4.2. Data Collection Methods

This section provides the justification for the use of both qualitative and quantitative

approaches to answer the research question and to test the research hypotheses.

Participants from each group provided both qualitative and quantitative data, which was

used to compare the task performance of collision avoidance between the experimental

and control groups. For example, the quantitative data included a set of structured

questionnaires with rating scales followed by an unlimited comment field (see

Appendix E). No follow-up questions were used in an attempt to reduce confusing

responses. That is, the response was entirely left to the participants to follow-up their

answers with a write-up in the fields provided for these purposes.

The quantitative research of the current study began with a survey questionnaire (see

Appendices E and F) that included both open-ended and closed-ended, questions. A

closed-ended question is aimed to restrict participants to a limited list of answers. The

goal of using closed-ended questions was to avoid bias problems (Schuman and Presser,

1979). Questionnaires such as these are routinely used to obtain quantitative data such

as opinions, levels of knowledge, skills, or characteristics of a specific demographic

(Leung, 2001). The post-questionnaire (see Appendix D) was divided into 5 sections.

Participants rated their opinion on the system by placing an “X” at the desired point, on

a 5-point utility scale. The scale reliability assessment is particularly important to show

that there is consistency in the way data were collected, analysed and interpreted

(Bryman and Cramer, 2005). This assessment draws the researcher’s attention to what

extent a measure actually captures the idea that it claims to measure.

Qualitative techniques, related to open-ended question interviewing, provide rich and

in-depth individual information about group differences (Van Someren, Barnard and

Sandberg, 1994: 26). Answers to questions were recorded using audio-tape, pen and

notebook. The contents were then transcribed and analysed using codes, themes and

relationships, as described by Van Someren, Barnard and Sandberg (1994). For

example, italics always denote coded data and pilots’ comments from the interviews.

of 303

Previous research has elicited participants’ thoughts through interviews that capture

their knowledge (Zhang et al. 2008). As Zhang et al. (2008) note, choosing a specific

method to measure and capture participant’s knowledge will depend on what kind of

task is under investigation. For example, causal and qualitative mapping are used to

support mental model elicitation (Desthieux, Joerin and Lebreton, 2010), self-

explanations (Chi et al., 1994), pre- and post-intervention (Doyle, Radzicki and Trees,

2008), and flight deck observations (Andre, 1995).

Furthermore, interviews have previously been used to collect flight deck automation to

examine pilots in flight scenarios, based on autopilot functions and operations (Sarter,

Mumaw and Wickens, 2007). Furthermore, Andre (1995) examined current problems

experienced by pilots during the taxi operations using questions such as “What

problems did you face when taxiing your aircraft in low-visibility conditions?” which

related to the introduction of electronic taxi map displays in the cockpit. To increase

face validity, Bryman and Cramer (2005), through face used pilot interviews, tests, and

questionnaires to identify a pilot’s typical reaction to the use of the new system just

tried. Therefore, in the current research, a mixed approach was used to analyse pilots’

situation awareness, task performance, and mental workload.

4.2.1 Measures of Mental Workload and Task Performance

Mental Workload is varied and influenced by a number of factors, such as the problem-

solving skills of the pilot, operating standard procedures, environmental constraints,

allocation of task and responsibilities, task demands and difficulty, and problem space-

display (Cuevas, 2003). In order to address these factors, the following variables were

considered, heading angle, time to contact and miss-distance (Bach, Farrell and

Erzberger, 2009). In addition, the problem-solving skills of the pilot, the different

approach position of Intruders, environmental constraints, standard operating

procedures and task demands and difficulty were also included.

of 303

Performance measures are based on a pilot's “mental capability” to perform a task

(Wickens, 2000). A pilot’s mental workload might be likely to increase, or decrease, as

a result of a change in a pilot’s task performance. To measure flight performance during

the experimental sessions, flight simulator software was used. These data included bank

angle, deviation, time-to-contact and the Ownship distance to the Intruders. As Prinzel

et al. (2002) clearly state, flight simulator results can be compared with the real

aircraft’s experimental results. As a result, the measurement of the pilot workload has

been widely used for the evaluation of pilot task performance during the flight operation

(Dahlstrom and Nahlinder, 2006).

Because mental workload is related to pilot task performance, workload issues are

considered in relation to aviation safety, cockpit automation design, and/or relating to

manoeuvre effectiveness. The NASA Task Load Index (NASA-TLX) is a subjective,

multidimensional measurement technique that rates a pilot’s perceived overall workload

score for a particular task and/or system. The NASA-TLX’s paper and pencil version of

workload rating is used in the current study. Hart and Staveland (1988) developed the

NASA-TLX, which includes scales of 21 points, with increments of high, medium and

low estimates. The application of TLX has been widely used beyond its aviation origin

for the past twenty years (Hart, 2006). For example, the recent use of eye-tracking

technology to assess the mental workload of surgeons in the operating room (Zheng et

al., 2012).

Subjective workload assessment techniques have been accepted by many researchers as

sensitive measures, as assessed by factor validity (Hill et al., 1992). However, according

to Luximon and Goonetilleke (2001), one of the main problems of using subjective

workload assessment techniques is a lack of sensitivity to low mental workload.

of 303

The NASA-TLX ratings require pilots to rate mental workload components by placing a

mark (‘X’) anywhere on the rating scale. The TLX subjectively assumes that the pilot’s

mental workload components are influenced by mental demand, performance, physical

and temporal demand, frustration, and efforts. The mean of NASA-TLX’s mental

workload score is determined by combining these components (see Appendix F) and

then standardised to 100%. A pair of these components was obtained to form six factors

and the six tallies were added to obtain the overall score. The overall score is divided by

weights (15), to obtain workload weighted scores, as expressed in the following

equation, where MD = Mental Demand, PD = Physical Demand, TD = Temporal

Demand, E = Effort, P = Performance and FL = Frustration Level, TI= Tally

Importance and R = Ratings.

10015

(%)

FLPETDPDMDMW (4.1)

RTIMD (4.2)

of 303

4.2.2 Measures of Situation Awareness

Pilots use mental models to seek information on computer screens, which depend on the

pilot’s experience and mode of flight (Verma, Corker, and Jadhav, 2004). However, the

failure to update mental models would prevent pilots from employing an effective and

efficient attentional allocation strategy. Thus, the result is either a random, occasional

scanning, or alternately, a rigid “open-loop scan” that is not sensitive to mode

progressions. Clearly, there is a need to communicate partial information to enable the

pilot to introduce a plan or to change it, thereby preventing accidents (Klein, Pliske,

Crandall and Woods, 2005).

Ideas, images, and representations characterised by perfect conformity with mental

models are necessary as a condition that must be fulfilled before achieving situation

awareness (Jones and Endsley, 1997; Sarter and Woods, 1991; Mogford, 1997).

Endsley (2000) presents a model that states how mental models are important in

achieving situation awareness. Furthermore, the mental models provide a process for

channelling attentional resources of the current situation, as well as integrating

perceived information, and forming an understanding of the situation. Mental models

also allow the pilot to predict a future system’s states that is based on its current state

and understanding of its dynamics. Mental models have highlights that situation

awareness is important in acquiring information from the environment. For example, a

pilot's decision-making depends on interpretations of the current situation. Therefore,

decision-making is formed through developing and maintaining situation awareness and

is modified by previous decisions (Endsley, 2000). However, improved situation

awareness is obtained through perception of the environment, even though perception is

not always provided by an automated display system. Nevertheless, providing

automated feedback can be done in many different ways. For example, situation

awareness can be created through decision aids to promote better situation awareness.

Despite the importance of situational awareness, some studies have highlighted the

distinction between situation awareness and performance. According to Wickens et al.

(2004), while participants can still be aware of elements of the situation, surprisingly,

performance measures cannot be perceived accurately. Furthermore, Endsley (1995)

disapproved of the use of explicit situation measures.

of 303

According to Endsley, even though it may seem that situation awareness can be

measured directly, the measure does not provide necessary information about how

participants will perform. Hence, there is no clear-cut agreement on the relationship

between situation awareness and task performance.

As discussed in Chapter 2, the context of a participant’s traffic situation awareness

(TSA) measurement is presented based on Endsley’ model (1998), as shown in Table 4-

1. Similar studies have used a self-rating technique (questionnaire) to assess information

related to situation awareness (Van Dam, Mulder, and van Paassen, 2008; Matthews

and Beal, 2002). The self-rating technique, in this case, involved interviewing

participants, along with the use of post-trial questionnaires, as presented in Table 4-1. A

five (5) point rating scale was used to measure the subjective situation awareness

assessment. The scale was developed to provide a means to predict the amount of

interference that will occur between different levels of situation awareness in

multitasking during mid-air collision situations.

Subsequently, the mid-air collisions usually occurred because two or more aircraft came

into contact during flight. The mid-air collision processes focused on the use of two

questions (SA1 and SA2) of the collision avoidance manoeuvres, in order to develop

situation awareness of the Free-Flight Environment (i.e., problem space), and the

Intruder and threat. Questions SA1 and SA2 comprised two initial phases, the Free-

Flight Environment Evaluation (FFEE) and the threat evaluation, followed by a threat

integration scenario. These questions also involved defining the flight environment,

describing the environment effects, evaluating the Intruder, and describing the Intruder

course of action. The FFEE development involved an assessment of the problem space

on both the Ownship and Intruder operations. The purpose of the problem space (i.e.,

environment) was to analyse the space, so that the conflict approach and manoeuvre

preferences could be identified. The threat evaluation stage was used to identify the

Intruders, characterised by patterns of behaviour considered acceptable. Once the FFEE

and threat evaluation stages were complete, threat integration was used to identify how

the Intruders were likely to operate during the collision avoidance. Inferential statistics

analyses were used from the questionnaire and the interview responses, comprising the

information elements used by the two groups during task performance.

of 303

Table 4-1: Question Contents and Levels of Situation Awareness

Question

(Level of Situation

Awareness)

Question Contents

Q1 The response of the yoke and throttle was too sensitive for me to track the path I wanted to follow.

Q2(SA1) How useful was the system for understanding the location of the Intruder relative to the Ownship? (Pilot’s ability to understand an awareness of other aircraft and separation-perception/visual observation).

Q3(SA1) How useful was the system for understanding the direction of travel of the Intruder? (Pilot’s ability to understand an awareness of other aircrafts’ heading).

Q4(SA2) How useful was the system for avoiding conflict?

(Pilot’s ability to understand the problem).

Q5(SA2) How useful was the system for avoiding stalling when manoeuvring the aircraft?

Q6(SA3) How useful was the system for providing sufficient information for following a desired flight path? (Pilot’s understanding of implications of information provided).

Q7 Please rate your overall opinion of the system.

of 303

4.3. Experimental Methodology

4.3.1 Study Participants This study was voluntary. Participants were recruited from Swinburne University and

the aviation community in Melbourne via advertisements on campuses. The participants

were students and professional pilots with at least basic flying experience. The

participants had no previous flying experience with protective cone and geometry

relative vector display.

The study used a between-subjects design with the control and experimental groups.

Table 4-2 shows the number of participants aged between 18 and 80 years old. This

sample size was deemed appropriate as similar studies that have used fewer

participants, for example Kim et al. (2011) used only 16 participants. According to

Eberts (1993), six participants per group is the minimum sample size required for

Human Computer Interaction (HCI) research. The first thirteen participants who were

recruited were allocated to the experimental group, while the second eight (8)

participants was assigned to the control group which remained balanced throughout the

experiment (see Table 4-2).

Table 4-2: The mean age of participants classified by group (N=21)

Group n Mean age

Experimental 13 39.0000

Control 8 24.6923

of 303

4.3.2 Ethical Considerations

Ethical approval was granted for all experimental procedures, including interviews and

surveys (see Appendix A), used in the study. Moreover, informed consent was obtained

from each participant prior to the start of the experimental procedures. The consent

form for the research project is provided in Appendix C. The participants’ workload

was expected to be minimal and reversible, different from flying in real case scenarios,

because of the controlled laboratory environment. The workload included close

activities that involved the visual tasks of viewing a standard colour monitor, and

possible hand soreness from using the stimulated Saitek Pro Flight Yoke System was

minimal.

4.3.3 Experimental Environment

The experiment was conducted with a standard desktop computer equipped with a

QWERTY keyboard and mouse, as shown in Figure 4-1. The keyboard and mouse were

used for data entry, such as a participant’s identification number. The task simulation

was run on an ACER G235h 23”, full high-definition (HD) widescreen colour LCD

monitor-crystal bright. The computer screen had a resolution of 1920 x 1080 at 60Hz

(with a 1GB graphic card) and a refresh rate of 75Hz. The CPU E8400 was a duo core

processor at 3.00 GHz with 3GB of RAM. A Saitek Pro Flight Yoke System was

connected to the PC. The yoke system, without rudders pedals and throttle interface,

was implemented in C++ programming language libraries. The simulator code was

written using MATLAB. The program enabled pilots to avoid obstacles by tracking,

navigating, maintaining or deviating from the intended flight path and the software

included an algorithm that represented basic performance constraints in flying a twin-

engine aircraft within conflict environments. The algorithm of the flight collision

avoidance display resolution model was based on a study by Bach, Farrell and

Erzberger (2009). The refresh rate of the common monitor ranged from 60 to 100Hz,

which was sufficient for smooth movements of scenery and the displayed information

was updated at an interval of 0.05s. Participant data were recorded electronically using

a Morae 3.2 software recorder.

of 303

The recorder captured audio, video and on-screen activity, and stored data digitally.

Questionnaires, numerical simulations and audio-recorded information were used for

data acquisition.

Figure 4-1: An Experimental Setup for Simulation

The first use of a flight simulator can be traced back to the 1930s and, since this time,

simulators have been applied in various fields of study designed to improve personal

skills or identify future directions, for example, in the military. However, different types

of simulators have been developed for various programs, such as medical programs

(Wilson et al., 2005), aviation, psychology and the automobile industry. Flight

simulation for both commercial and military aircraft is widely used in the aviation

industry to train pilots and other flight crewmembers. The concept behind the use of the

cockpit is to house in-flight instruments to allow pilots to practise their flying skills by

using the flight instruments in a safe environment. Environment settings for simulating

aircraft cockpit displays ranging from low-fidelity (“level of detail”) with one or more

aircraft system’s functional capabilities to a high-fidelity (i.e., simulator) (Liu et al.,

2009).

of 303

According to Liu et al., (2009), there are two acceptable fidelity measurements; an

objective method where the mathematical measurement variable calculated is

equivalent to one shared with the real world and a subjective method which is used to

obtain, or retrieve, evaluation of strategies from human performance data and then

compare it with real-world performance. As Prinzel et al., (2002) stated, flight simulator

results can be compared with the real aircraft experimental results. This indicates that it

is possible to have a well-designed flight simulator experiment that is comparable to

real flight situations.

High-fidelity simulators present comprehensive aerodynamic and systems modelling

that is relevant to the real aircraft and its environment. This simulator encompasses

scene-linked Head-up Display (HUD) symbology for taxi and surface operations (Foyle

et al., 1996), and the real collision of automotive crashes (Lee et al., 2002). The high-

fidelity simulator has proved to be more reliable candidate than the low-fidelity

simulator when it comes to conducting experimental studies, however, the financial cost

is often high with these high technological devices (Liu et al., 2009). The low-fidelity

simulator has been applied in various fields. For example, Thornton et al., (1992) used a

low–fidelity simulator to investigate automation effects in the cockpit.

4.3.4 Experimental Procedures

4.3.4.1 Briefing

The briefing lasted around fifteen minutes. Participants were first issued a copy of the

Consent Form and Information Sheet on the set-up of the experiment, and the purpose

of the research to the participants (refer to Appendix D). The participants in both groups

then filled-out questionnaires about their age and experiences with collision avoidance

displays. The participants then received instructions on how to undertake navigational

tasks that involved conflict resolution tasks.

In the current study, participants were asked to fly simulated Instrumental Flight Rules

tasks. The pilots were provided with pictures of the display to familiarise themselves

with the different display symbology, and the types of conflicts, that they may

encounter during the experiment runs.

of 303

Each test run consisted of ten (10) minutes of tracking tasks. Participants were

instructed to fly the prescribed flight route, as shown in the upper right corner of the

display, with minimum flight path deviation. Participants were allowed to have practice

runs during the training sessions, as requested. However, on average, they performed

approximately three practice runs. Each of these practice runs consist of a three (3)

block of conflict resolutions. Participants were required to resolve each of these

conflicts sequentially within three (3) minutes. After completing the navigational tasks,

a three (3) block conflict resolution was then followed. In the event where the

participants could not resolve conflicts within the setup constraints, a red button was

provided on the yoke system to “notify” Air Traffic Services (ATS) via a data link

when pressed. Participants’ final task performances were recorded to consist of all the

three conflict scenarios for analyses. At the end of the experiment, participants were

given a self-administered post-questionnaire (refer to Appendix E) in relation to the

tasks and an interview was conducted after the exercises were completed. The overall

exercise lasted less than two hours.

4.3.4.2 Debriefing

At the completion of the exercises, participants were shown a poster presentation of the

display to re-familiarise themselves with the types of conflicts and to recall the features

encountered on the screen. The objective of re-presenting the poster to the participants

was to elicit their mental models and situation awareness. Participants were also

interviewed to evoke cognitive control level models (Rasmussen, 1993) and mental

models. In addition, the interview was necessary to record each participant's opinion on

their interaction with the system. Participants completed a post experimental

questionnaire (refer to Appendix E) to assess situation awareness and cognitive

workload (refer to Appendix F). For example, to assess situation awareness, participants

rated their opinion on the system by placing an “X” at the desired point on a 1 to 5 point

continuous scale. Question 1 through to Question 7 employed a two-dimensional

scaling strategy that passed through a midpoint.

of 303

In the case of Question 1, the scale values ranged from “fully disagree (1)” to” fully

agree (5)”. From Question 2 through to Question 7, the scales ranged from “extremely

un-useful (1)” to “extremely useful (5)”. Participants then completed a questionnaire

(see Appendix D) about their background and flying experience, as well as a post

experimental questionnaire (see Appendix D) to assess the level of their acceptance of

the system and for the mental workload.

4.3.5 The Scenarios and Collision Avoidance Resolutions

A computer-based simulator was configured as a multi-engine aircraft with a cruising

speed of 356 knots. The conflict was set to occur at a Flight Level of 130 due to ground-

based navigation and communication equipment failure. Because of the unavailability

of radar coverage, a procedural self-separation was employed to provide a long-term

solution to the air traffic problem. The study selected two variables to analyse

participants’ inputs to avoid collision. The input variables were the roll angle (i.e., bank

angle) and speed. The roll angle was measured in degrees of the yoke position about the

longitudinal axis, the throttle level positions were used to measure the engine power

output for speed and the engine output was not coupled with the pitch angle. It was

assumed that a change in height was negligible for horizontal flight.

A standard rate of turn of three degrees per second (3º per second) was used for a

heading change. A change in aircraft heading is a function of the bank angle, and there

was a direct indication of the bank angle displayed as an arc on the FCAS, with pilots

having a direct representation of ground speed on the FCAS. The FCAS display

provided participants with basic information about the relative positions of other

aircraft, including velocity vectors, altitude and airspeed. A vector represented the

Intruder’s direction and a text presentation of the intruder’s relative speed and altitude

was also provided. The FCAS has a diamond icon representation for presentation of the

intruder’s position with a top-down view and heading up orientation. The Ownship

position is centred as a frame of reference. The protective cone appeared once the

Intruder position was about twenty one nautical miles away, relative to the Ownship

position.

of 303

The scenarios were based on an algorithm for level conflict resolution in a model

proposed by Bach, Farrell and Erzberger (2009), as illustrated in Table 4-3. These

scenarios were modelled based on the assumption that the two aircraft were on the same

flight level maintaining constant altitude, speed and heading. However, collisions exist

between the two aircraft with crossing trajectories at a constant conflict angle. The three

scenarios were configured for lateral conflict, Starboard, Portboard and Head-On

approaches. These scenarios are presented in Table 4-3 for both groups. There were no

weather situations is no considered any of the conflict scenarios. A number of weather

issues could result in a much more difficult self-separation task due to such

considerations as flight plan changes, and display clutter issues (Cashion et al. 1997).

The conflict test began after completion of navigational tasks. This was followed by a

pre-programmed Intruder, which would appear randomly, from any of the three

locations (Starboard, Portboard and Head On approaches) which could pose a threat to

the Ownship protected zone. A TCAS-like audio alert was generated to inform the

participants of a threat. For each trial, participants observed the development of conflict

scenarios for seventy seconds. The Ownship was allowed to manoeuvre to avoid

conflict, while the Intruder maintained constant heading and speed. The pilot needed to

determine an appropriate manoeuvre in order to avoid the traffic. The possible solution

to avoid conflict was to make sure that the relative velocity vector was outside the

protected cone without making contact with the yellow circle (i.e., protective zone)

around the Intruder. This was done by changing directions, either by rotating the yoke

system clockwise or anticlockwise and/or using the throttle lever to reduce/ increase the

airspeed, thus allowing the Ownship to pass in front of, or behind the Intruder, as shown

in Figure 4.2. The pilots returned the simulator to straight and level flight from a set

initial condition of bank angles once the conflicts were resolved. The pilots maintained

straight and level flight for approximately three (3) minutes, while waiting for the next

conflict. Conflict times and aircraft headings were a slightly varied sequence across the

three scenarios to minimise the repetition of the conflicts. The protective zone colour

turned red if the Intruder was within seven nautical miles of the Ownship, and if pilots

did not initiate a manoeuvre to mitigate the threat.

of 303

Table 4-3: A summary of the conflict resolutions and data required to avoid conflict for both groups

Scenario Approach Conflict Resolution Angle

Scenario 1 Starboard

A level left or right turn at the current airspeed with rotation clockwise > 0 or rotation counter clockwise < 0 was required to escape a collision. Reduce or increase speed.

Manoeuvre to the Rear (behind the intruder ) |β| = 22.5º

Manoeuvre in Front (in front of the Intruder) |β|=32.7º

Scenario 2 Port

A level left or right turn at the current airspeed with rotation clockwise > 0 or rotation counter clockwise < 0 was required to escape a collision. Reduce or increase speed.

Manoeuvre in Front (in front of the Intruder) |β|=32.7º

Manoeuvre to the Rear (behind the intruder) |β| = 22.5º

Scenario 3 Head-On A level left or right turn at the current airspeed with rotation clockwise > 0 or rotation counter clockwise < 0 was required to escape a collision.

Manoeuvre to the Right |β|=18º

Manoeuvre to the Left |β|=18º

Under the free-flight environment, pilots have the freedom to choose their flight path to

avoid collision. However, this freedom is constrained by environmental factors

including the Intruder and aircraft performance constraints. Pilots are allowed to change

aircraft flight path to accommodate flight constraints (Wiener et al. 1991). With no

established preferences, pilots needed to determine appropriate manoeuvres to avoid air

traffic. For example, one way for the experimental group to avoid conflict was to ensure

that the relative velocity vector was outside the protected cone. This was achieved either

by rotating the yoke system clockwise or anticlockwise and/or using the throttle lever to

change speed. Figures 4-2 and 4-3 show the conflict resolution scenarios in the display

for both the experimental and control groups. Figures 4-4, 4-5 and 4-6 show screenshots

from both groups. Figure 4-2 and 4-3 shows conflict resolution scenarios both

of 303

experimental and control groups display. Figure 4-4, 4-5 and 4-6 shows screenshot both

groups.

Figure 4-2: Overview of Conflict Resolution Manoeuvres by the Ownship for the control group

Figure 4-3: Overview of Conflict Resolution Manoeuvres by the Ownship for the experimental group

Manoeuvres

In Front

Manoeuvres

Behind

Starboard Approach

{

dM

N

Ownship

Intruder

Protective Cone*

*ownship

int/*

ownV

ownv

int/ownv

Intruder

Swin 06

FL 130 250

Md

Ownship

Starboard Approach

of 303

(a) Screen shot of FCAS-EID (b) Screen shot of non-EID

(a) Screenshot of FCAS-EID (b) Screenshot of non-EID

(a) Screen shot of FCAS-EID (b) Screen shot of non-EID

Figure 4-5: Screen of Display Formats (Starboard Approach)

Figure 4-4: Screen of Display Formats (Portboard Approach)

Figure 4-6: Screen of Display Formats (Head-On Approach)

of 303

4.4. Independent and Dependent Variables

4.4.1 Measures of Independent Variables The experimental task was to test the functionality of the FCAS. The three independent

variables were air traffic density, conflict avoidance display, and conflict geometry.

Air Traffic Density

The proposed future of Air Traffic Management Systems suggested that numbers of

aircraft would double in the next three decades. Therefore, the scenarios used in this

research included high traffic density, three times the current traffic situation, to

simulate how pilots will cope in a free-flight environment.

Conflict Avoidance Display

The study tested two types of 2D conflict avoidance displays: the FCAS (EID) and

conventional (non-EID). The FCAS display presented solutions showing the protective

cone to include the following variables, line-of-Sight (i.e., twenty-two nautical miles of

separation between the Intruder and Ownship), intruder position and speed relative to

the Ownship and a circle of seven nautical miles representing a minimum protective

zone for both the Intruder and Ownship.

Conflict Geometry

The conflict angles adopted in this study are based on the FAA course definitions for

ATC standard separation, which related to the current horizontal geometry conflicts

(FAA, 2000). The conflict angles in this study ranged from greater than 270 (acute) to

less than 090 degrees. Three different conflict geometries were tested on the FCAS and

the control display. These geometries assist pilots to manoeuvre Head-on, Starboard and

Port conflict scenarios.

4.4.2 Measures of Dependent Variables

The performance variables selected for analysis were situation awareness, manoeuvre

preferences, heading, mental workload, and loss of separation. These variables are valid

measurements of the pilot’s performance for turning manoeuvres.

of 303

4.5. Methodology for Data Analysis

The methodological approaches adopted in this study concurrently collected data for

qualitative and quantitative approaches and both these approaches were used for data

analysis. Specifically, in the current study, quantitative data refer to numeric variables

for statistical analyses, while qualitative data provided narrative descriptions of

participants’ comments about their experiences with the FCAS. The evaluation of the

FCAS includes the assessments of interviews questionnaires (quantitative approach),

and numerical data, from flight simulations. The purpose of the interview was to

explore participants’ general perspective on the use of the FCAS in a free-flight

environment. The data collected from the participants were comments about their

experiences with FCAS, the transcripts from individual interviews, and responses to

open ended questions were used for analyses

In addition, questionnaire and numerical data from flight simulations were analysed.

The data were analysed by using boxplots, simulator’s data plots, t-test, Univariate

Analysis of Variance (UNIANOVA) and correlation analyses, to determine the

differences between the participants. The data consisted of counts or frequencies,

percentages, and performance tests. A statistical software package, Statistical Package

for the Social Sciences (SPSS), was used for the analyses. To validate the results, the

following four assumptions were considered, 1) independence of observation; this is a

design issue, 2) no outliers in either of the two groups, 3) normality; data is normally

disturbed in each of the groups, and 4) homogeneity of variances.

To address the differences between the control and experimental groups, an independent

sample t-test analysis was performed to test significant difference between these two

groups (Cohen, 1992), the result was significant is p-value < 0.05. Specifically, thesis

examined the differences is a number of dependent variables including preference,

workload, heading and distance. According to Hair (1998), the use of a single criterion

to indicate the level of a statistically significant difference in the system’s usability

scores between the experimental and control groups is too superficial. Therefore, to

maximise the difference between the participants, correlation analysis assessment was

also used to perform Multivariate Analysis of Variance (MANOVA) (Meyer, Gampst

of 303

and Guarino, 2006). Finally, to understand the behaviour of pilots performing conflict

resolution, there is a need to understand the relationship between dependent variables.

One way to measure the relationship between these variables was through Pearson's

correlation coefficient (Pearson's r). The study examined the correlation between

dependent variables to improve the understanding of the pilot’s interaction with

collision avoidance displays for self-separation in a free-flight environment.

Human behaviour in general is considered complex. The complex nature of this

behaviour has multiple causes and effects. Behaviour defines a pilot’s personality,

underlying values, and beliefs. Under different sets of environmental conditions, the

traits differ from pilot to pilot, and can elicit different actions or behaviour from each

pilot. Studies in these areas require multivariate procedures with respect to the complex

world of behavioural research (Sherry and Henson, 2005). However, due to its difficulty

in analysis, and the complexity of interpreting the results, limited behavioural studies

use this approach (Campbell and Taylor, 1996).

In contrast to Analysis of Variance (ANOVA), Multivariate Analysis of Variance

(MANOVA) differs in that it provides a way of analysing between group effects for

multiple related dependent variables. It calculates a linear combination of the dependent

variables which can then be used to compare groups. That is, a multivariate analysis is a

technique concerned with determining the relationships between dependent variables in

a data set, and to state which of the dependent variables plays an important role in the

data sets. In contrast, discriminant analysis (DA) is used to analyse the predictive

power of dependent variables in predicting group membership (i.e., discriminate

between the two occurring groups), and there are two steps involved in this type of

analysis, tests of significance for discriminant function and canonical discriminant

functions. According to Sherry and Henson (2005), because a test can be conducted

only once to investigate the relationship between independent and dependent variables,

the risk of committing a Type I error is minimised. The use of this technique provides

additional information about how pilots interact with the FCAS. The information is

useful in identifying the specific type of a composite variable that influences a pilot’s

behaviour during the interaction with the FCAS.

of 303

4.6. Summary

This chapter described the research design and methodology used in the current study.

The design approach is to maximise valid answers to our research question and

hypotheses. The evaluation methods used to assessing the participants’ task

performance included a qualitative, exploratory-descriptive approach that is contextual.

The evaluation includes the assessments of interviews, questionnaires, and numerical

data from flight simulations. A desktop flight simulator, interviews and questionnaires

were used as measurement instruments. A between-subjects design was used for the

present study. Participants (21) were recruited on a voluntary basis. The study also

ensured that the participants were morally and ethically protected. The three

independent variables were traffic density, conflict avoidance display, and conflict

geometry. The pilots’ performance variables selected for examination are subjective

pilot preferences, aircraft heading, mental workload, and minimum deviation from the

intended flight paths.

Subjective mental workload ratings are based on elicited pilots’ opinion on the FCAS.

The most commonly used methods to elicit mental workload ratings is the NASA-TLX.

A sample t-test was used to assess these data to determine the differences between the

participants (p < 0.05) and further assessment was required to maximise the differences

between the participants (i.e., experimental and control group). Correlation analyses

were performed as part of multivariate techniques concerned with determining the

relationships between dependent variables in a data set. Discriminant analysis (DA) was

then used to analyse the identical task as that of multiple linear regression (MLR)

technique. The purpose of using this technique was to maximise the differences

between the participants. This technique uses a two (2) step process in which dependent

variables play an important role in the data sets. The first step is group classification and

the second is revealing participants’ capabilities by discriminant functions. The next

chapter will evaluate the flight avoidance display and present findings.

of 303


of 303

Chapter 5. Findings and Analysis

This chapter presents the findings and analyses, and discusses the results from the

questionnaire, simulation and interview data to determine the merits of a flight collision

avoidance display. Effective FCAS design is fundamental for pilots to maintain

separation. The main objective of this research was to examine the influence of the

protective cone on collision avoidance in a free-flight environment by developing and

evaluating a flight collision avoidance and control system. This chapter compares

pilots’ mental workload and system usability and examines the comparative measure of

pilots’ performances in both the experimental and control groups. The experimental

group consist of thirteen (13) participants supported by the FCAS, a control comparison

group made of eight (8) participants with a conventional display. The group differences

are evaluated between a between-subjects design.

of 303


of 303

5.1. Introduction

The proposed FCAS is designed to improve some of the current system limitations,

such as the use of the “see and avoid” technique, as discussed in Chapter 2. The aim of

this study was to assess which factors associated with the environment, aircraft

constraints, and perception of collision, significantly influence a pilot’s decision making

when manoeuvring their aircraft, such that the protected zone is not violated. This

chapter presents the findings and analysis in relation to collision avoidance by pilots’

participation in the flight simulations, as described in Chapter 4 of the research design.

Chapter 5 was consisted of five main sections. The first section, the research method is

outlined including on data collection, scope of analyses, overview of research

methodology and hypotheses. The second section describes the results, including both

the quantitative and qualitative findings results. The quantitative section includes t-test

and correlation analyses as well as a discussion of the data from the questionnaire (both

closed and open-ended questions), the background information of the participants

(flight experience), and numerical simulations. This subsection also presents

Multivariate Analysis of Variance and discriminant analyses. The qualitative section

includes the results from pilot interviews with the aim to gain an in-depth understanding

of pilots’ cognitive behaviour in a free-flight environment and the reasons that govern

such behaviour. The third section presents modifications of two models, Wicken’s

(1984) and Parker (1973) and the fourth section presents implications of mental models

theory. Finally, the results of the current study will be summarised.

of 303

5.2. Methods

Data obtained from participants included their responses to simulator exercises,

interviews (i.e., mental models), and questionnaires that were completed following the

experimental session. Statistical analyses determined if differences existed between the

experimental group and the control group. Resolving conflict with minimum deviation

from the intended flight path is associated with three possible difficulties. First,

choosing the correct rate turns for a specific roll manoeuvre at high speeds. Second, the

pilots’ decision to perform earlier, or delay the initiation of the selected roll manoeuvre

could lead to violating or avoiding the specified minimum separation. Finally, the pilot

must choose the current bank angle that provides minimum deviation from the original

flight path to avoid collisions. This study examined pilot mental workload, performance

and situation awareness in a conflict situation. That is, how safely a pilot can perceive

and correctly respond to a conflict situation. Therefore, the objective is to examine pilot

performance using the FCAS compared to a conventional display. Specifically, we

hypothesise that the FCAS will:

Have an improved situation awareness compared with the situation awareness


Violate separation constraints less frequently than the horizontal minimum


Have a lower mental workload compared with the mental workload experienced


5.2.1 Scope of the analyses

The purpose of the FCAS is to perform collision detection and avoidance analysis. In

the current experiment, the system notifies and identifies aircraft at the same flight

level. Whenever there was a new collision, an alert notification for the pilots was

displayed. The system also detects the conflict situation to provide a more organised

display to the pilot. For this analysis, a collision detection support system was

developed. Only the positioning of the aircraft, which is relevant to determine a

collision situation, is salient.

of 303

The aircraft position for collision detection appeared from three locations, Head-On,

Starboard and Portboard approaches. The conflict scenarios were all constructed using a

similar geometry, with a co-altitude intruder at an 80-degree convergence angle,

approaching either from the Portboard or Starboard, or at a 0-degree angle approaching

Head-On. The conflicts in the high-density versions of each scenario type were

identical, with the only differences being that the aircraft had different call signs and

different levels of non-conflicting traffic were represented. One of the three scenario

types included a lateral conflict in which the Ownship had the right-of-way (i.e.,

Ownship was on the right).

To assist in collision avoidance, the resolution allowed three levels of safety, from level

3, with no alert of collision; level 2 (protected zone is coded yellow) with alert of

collision and level 1 (red, where collision is imminent and cannot be avoided, provided

that no manoeuvres were made), with different levels of near misses between level 2

and level 1. Separation standards for defining these levels may change with time.

Navigation plans and reroutes were not considered, and there were no weather

situations in any of the scenarios. The conflicts involved a pair wise proximate aircraft,

and a co-operative manoeuvre is not assumed. The comparison was only based on the

experimental and control groups. Individual participants’ data were not analysed, and a

discussion about think-aloud protocol to capture participants’ thoughts was not

conducted in this study.

5.2.2 Overview of research methodology

The overview of the research methodology is illustrated in Figure 5.1. Typically, a

mixed approach consists of both quantitative and qualitative approaches. The mixed

approach was used here to inquire into the research question and hypotheses. With the

quantitative approach, data were collected from a survey questionnaire, NASA-TLX

subjective mental workload ratings, performance, and participants' opinions on the

utilisation of the system being tested. These data were analysed using an independent t-

test to reveal any significant differences between the control and experimental groups.

A reliability analysis tested the consistencies of the descriptive questionnaire of

participants’ opinions used for situation awareness ratings.

of 303

Moreover, canonical correlation analysis was used to maximise the differences between

the participants as part of Multivariate Analysis of Variance (MANOVA) and, finally,

discriminant function analysis determined which dependent variables discriminated

between the two groups. The use of the qualitative approach was to gain an in-depth

understanding of pilots’ behaviour in a free-flight environment. Qualitative data were

collected from interviews (using open-ended questions) conducted with the participants.

The contents were transcribed and analysed based on thematic analysis to elicit

participants’ mental models on the use of the protected cone display. The overall

quantitative and qualitative findings were presented to answer the research question.

SPSS software (version 22) was used for these analyses.

Response Pattern

Discriminant Analysis

Mental Workload NASA_TLX

Performance Separation Heading

Situation Awareness

(Level 1,2 &3)

Correlation Analysis

Qualitative method

Interview

Mental models

Mixed methods

Findings

Quantitation Method

MANOVA

T-test

Figure 5-1 Overview of research methodology

of 303

5.3. Results of Interaction with FCAS

As mentioned previously, this research combines quantitative and qualitative

approaches, a strategy aimed at serving the purpose of a mixed approach, as explained

in Chapter 4. The following issues were discussed: pilots’ experiences; performance;

manoeuvre preference; pilots’ mental workload; situation awareness; and pilots’

appraisal of FCAS.

5.3.1 Flying Hours

This section provides information on the distribution of participants according to their

flying experience. Table 5-1 presents the frequency and percentage of flying hours for

each group, collected via the questionnaire. The participants in the control group (8)

have more flying hours than the experimental group. One of the participants in the

control group had experience with radar systems and another participant in this group

having experience with TCAS systems. From Table 5-1, the results in the table reveal

that, on average, pilots in the control group had significantly more hours of flying

experience (173.63) than the experimental group (106.62).

Table 5-1 Presents the frequency and percentage of participants classified by the number of flight hours (N=21)

Group n Total Hours Mean(SD)

Experimental 13 1386 106.62(70.83)

Control 8 1405 173.63(73.01)

Total 21 2791

Flying qualifications may have a significant influence on task performance. A t-test

analyses on flying experiences was conducted. The results of the two-sample t-test for

unequal variances indicates that the mean flying hours for experimental group (M =

106.62, SD = 70.83, n = 13) is significantly different (lower) than the mean flying hours

for control group (M = 173.63, SD = 73.01, n = 8), [t (19) = 2.44, p < 0.05]. Hence, we

reject the null hypothesis and conclude that, on average, pilots allocated to the control

group have significantly more hours of flying experience than pilots allocated to the

of 303

experimental group. It may be the case that the mental workload of the pilots, the

environmental demands, and the ability to deal with those demands, are better dealt

with by pilots with many years of flying experience, than those with less years of flying

experience. As the control group has significantly more flying experience than the

control group, this may have a significant influence on the mental workload of each

group.

5.3.2 Performance measures

A clear understanding of the meaning and perception of separation and heading is

essential to resolve any conflict. Pilots’ two control variables for the analyses are

separation and change in an aircraft’s heading. These performance variables were

analysed from primary data recorded by the flight simulator during the experimental

exercises. The following definitions of the two performance variables are provided

below:

1. Separation assurance refers to the distance between the Intruder and Ownship on a

collision course. This is a measure of the number of significant miss-distance deviation

in a trial. The line-of-sight (LoS) vector so is between aircraft. Larger magnitudes of this

distance correspond to higher rates of roll change, with the pilot judging the amount of

movement of the Intruder’s protective zone and its observed rate of change.

2. Aircraft heading (i.e., change in aircraft’s direction to avoid collision) and roll yoke

angle were measured in degrees of the system position on the roll axis and used to

measure the amount of movement to command a change in aircraft bank angle, and thus

a change in the aircraft’s direction. The rate of change of an aircraft’s direction is

proportional to the tangent of an aircraft’s bank angle. The pilots observed a rate of

change of the relative vector and the Intruder’s protective zone.

of 303

5.3.2.1 Separation Assurance: loss of separation

The flight path data were analysed for deviations around the minimum separation

standards (e.g. 7 nm) while flying within instrument flight rules. Table 5-2 presents

descriptive data analysis showing the results of multiple mean comparisons of minimum

separation standards across both groups. To test the hypothesis that pilots in the

experimental group will violate separation constraints less frequently (H-2), an

independent t-test was conducted. The experimental and control group distributions

were sufficiently normal for the purpose of conducting a test (i.e., skew < |3.0| and

kurtosis < |9.0|), (Schmider, Ziegher, Danary, Bayer and Buhner, 2010). Furthermore,

the assumption of the homogeneity of variance was tested, but not satisfied via

Levene’s F test, F (8.07), p =.010, and rejects the null hypothesis (i.e., no difference

between the two group’s variances) for the assumption of homogeneity of variance, to

conclude that the variability in the two groups is significantly different. Separate

variances and the Welch- Satterthwaite correction were used.

Table 5-2 presents the results of the t-test analyses on perceived separation performance

measures for both the Head-On and Starboard approaches. With Head-on approach, an

independent sample t-test assuming unequal variance was conducted to see if there are

statistically significant differences in the mean separation performance scores of

experimental group and control group at a 0.05 level of significance. The results of the

two-sample t-test for unequal variances indicates that the mean separation performance

score for experimental group (M=14.61, SD= .12, N= 13) is significantly different

(greater) than the mean separation performance score for control group (M=14.36,

SD=.31, N= 8), [t (19) = .44, p < 0.05]. These results suggest that experimental group

have a relatively more favourable manoeuvre towards separation performance

compared to control group; hence, we reject the null hypothesis. The research

hypothesis was supported.

With a Starboard approach, the Intruder is positioned on the right hand side of the

Ownship and is assumed to have the right-of-way (i.e., the Intruder is not required to

manoeuvre). An independent sample t-test assuming unequal variance was also

conducted to check statistically significant differences in the mean separation

of 303

performance scores of experimental group and control group at significance level of

0.05. The results of the two-sample t-test for unequal variances indicates that the mean

separation performance score for experimental group (M=14.43, SD= .44, n = 13) is

significantly different (greater) as compare to the mean separation performance score

for control group (M=14.28, SD=.44, n = 8), [t (19) = -.64, p > 0.05]. Therefore, there is

no statistically significant difference in the overall performance between the

experimental and control groups for the Starboard approach. This indicates that the

FCAS has made little difference on performance. The possible explanation is that the

difference can be reasonably attributed to random fluctuations of pilots’ manoeuvring

decisions, related to interpretation of the VFR right-of-way procedures. Therefore, there

is no significant influence of the starboard approach on separation performance; hence,

reject the null hypothesis (H-2).

Table 5-2 : Descriptive statistics associated with pilots’ performance on separation measures

Separation Measures

Means (SD)

t

df

p

95% CI

Control Group

Experimental Group

Lower limit

Upper limit

Cohen’s d

Starboard Approach

14.28 (.44)

14.41 (.43)

-.64 19 .53 -.53 .28 0.20

Head-On Approach

14.36 (.12)

14.61 (.31)

-.44 19 .00 -37 -.13 0.71

Port Approach

13.92 (.63).

14.66 (.26)

-3.78 19 .010 .21 1.27 0.61

With Port approach, the experimental group was associated with a higher mean score of

loss of separation (M = 14.66, SD =.26, n=13) compared to the control group (M

=13.92, SD =.63, n = 8). The independent sample t-test revealed a statistically

significant difference between the groups [t (8.45) = 3.17, p = .012 (two tailed)] and

therefore it can be concluded that the experimental and control groups differed

significantly on their separation test performance. The effect size, as estimated by

Cohen’s d = 0.61, corresponds to the percentile standing of 73% of the mean score of

of 303

participants in the experimental group, in comparison to an average participant in the

control group. However, in relation to the control group, the Cohen’s d is a moderate

effect size and suggests 73% of the experimental group performed above the average

score of the control group. As expected, the protective cone has improved pilots’

situation awareness on separation in the experimental group.

The overall results suggest that participants in the experimental group have exhibited

larger lateral deviations during the Portboard and Head-on approaches than the control

group, and that FCAS has improved and supported pilots’ situation awareness to

maintain minimum separation standards. The possible explanation is that the FCAS has

provided pilots with confidence in performing manoeuvres to avoid separation conflicts,

and as a result generated larger minimum separation standards.

Figures 5-2 and 5-3 indicate a series of pilot manoeuvres, illustrating a response during

flight conflict for the experimental group. To judge the relative effectiveness of FCAS

lateral guidance, resolutions are displayed by highlighting the Ownship aircraft and

indicating the region of conflict resolution along the active flight path with a “green”

coloured line, as shown in Figure 5-2(d) and 5-3(d). The highest peak induced conflict

resolutions occurred when pilots followed FCAS guidance. As shown in the figures, the

highest peak induced is denoted by “deviation” in graph ‘e’, as a measure of the ability

of pilots to resolve a conflict and remain out of conflict. This highlights the advantage

of the FCAS when resolving conflicts involving pair wise proximate aircraft.

of 303

0 1 2 3 4 5 6 7 8 9 100

200

400

Time(min)

Hea

ding

( )

ATC Command vs Pilot3 Response

0 5 10 15 20 25 30 35 40 4550

100

150 Ownship Flight Path

Hea

ding

( )

Time(s)

0 5 10 15 20 25 30 35 40 450

10

20Separation

Dis

tanc

e(nm

)

Time(s)

0 5 10 15 20 25 30 35 40 45200

400

600Relative velocity

Kno

ts

Time(s)

0 5 10 15 20 25 30 35 40 45-20

-10

0Rolling

Ang

le(

)

Time(s)

ATCPilot

Ownship's position Intruder's positionDeviation

Figure 5-2 Pilot (E3) Spacing performance in the experimental group

(a)

(b)

(c)

(d)

(e)

of 303

0 2 4 6 8 10 120

200

400

Time(min)

Hea

ding

( )

ATC Command vs Pilot 12 Response

0 5 10 15 20 25 30 350


Hea

ding

( )

Time(s)

0 5 10 15 20 25 30 350

10

20Separation

Dis

tanc

e(nm

)

Time(s)

0 5 10 15 20 25 30 35200

300


Kno

ts

Time(s)

0 5 10 15 20 25 30 35-50

0

50Rolling

Ang

le(

)

Time(s)

ATC Pilot

Ownship's position Deviation Intruder's position

Figure 5-3: Pilot (E12) Spacing performance in the experimental group.

Figures 5-4 and 5-5 below indicate the equivalent information for the control group. To

assess the relative effectiveness of the conventional display for lateral guidance,

resolutions are displayed by also highlighting the Ownship aircraft and indicating the

region of conflict resolution along the active flight path with a “green” coloured line, as

presented in Figure 5-4(d) and 5-5(d).

(a)

(b)

(c)

(d)

(e)

of 303

As shown in Figures 5-4 and 5-5, the highest peak induced is denoted by “deviation” in

graph ‘e’ as a measure of the ability of pilots to resolve a conflict, and remain out of

conflict. “Deviation” is the position of the Ownship when it is just separated from the

Intruder. This highlights the advantage of the conventional display when resolving

conflicts involving pair wise proximate aircraft.

0 1 2 3 4 5 6 7 80

200

400

Time(min)

Headin

g(

)


0 2 4 6 8 10 12 14 16 18 20100

150200

Ownship Flight Path

Headin

g(

)

Time(s)

0 2 4 6 8 10 12 14 16 18 200

1020

Separation

Dis

tance(n

m)

Time(s)

0 2 4 6 8 10 12 14 16 18 20500

600700

Relative velocity

Knots

Time(s)

0 2 4 6 8 10 12 14 16 18 200

1020

Rolling

Angle

( )

Time(s)

Pilot ATC

Intruder's position ResolutionOwnship's Position

Figure 5-4: Pilot (C6) Spacing performance in the control group.

(a)

(b)

(c)

(d)

(e)

of 303

0 1 2 3 4 5 6 7 80

200

400

Time(min)

Hea

ding

( )


0 5 10 15 20 25100

150


Hea

ding

( )

Time(s)

0 5 10 15 20 250

10

20Separation

Dis

tanc

e(nm

)

Time(s)

0 5 10 15 20 250

200


Kno

ts

Time(s)

0 5 10 15 20 250

20

40Rolling

Ang

le(

)

Time(s)

ATCPilot

Deviation Ownship's positionIntruder's position

Figure 5-5: Pilot (C7) Spacing performance in the control group

The overall pilot performance from the figures above (Figure 5-2 to Figure 5-5) of both

groups’ avoidance manoeuvres indicate that the experimental group conflict resolution

was relatively slower (i.e., through physical observations of these figures) than that of

the control group. The possible explanation is that pilots in the experimental group may

have spent more time examining the features of their display to support and/or influence

their decision-making or decision considerations to execute collision resolutions.

(a)

(b)

(c)

(d)

(e)

of 303

5.3.2.2 Aircraft’s heading: change of flight plan

In this section, we examined several measures to assess how FCAS features influence

pilots’ performance to resolve conflict in a free-flight environment, as compared with a

control display. Pilots’ performance includes maintaining the desired flight path that

produces minimal change in heading. The independent t-test indicated no difference

between the experimental and control groups on heading change, as shown in Table 5-3.

Table 5-3 Descriptive statistics associated with pilots’ performance on heading measures

Table 5-3 presents the results of the t-test analyses on perceived separation performance

measures for both the Starboard and Head-On approaches. As with Starboard approach,

an independent sample t-test assuming unequal variance was conducted to determine if

there was a statistically significant difference between the mean heading change

performance scores of the experimental and control groups at significance level of 0.05.

The results of the two-sample t-test for unequal variances indicates that the mean

heading change performance score for the experimental group (M=29.76, SD= 8.68, n=

13) is significantly different (lower) than the mean heading change performance score

for control group (M=31.75, SD=8.96, n= 8), [ t (19) = .50, p > 0.05].

Heading Measures

Mean (SD)

95% CI

Cohen’s d Control

Group Experimental Group

t df

p

Lower limits

Upper limits

Starboard Approach

31.75 (8.96)

29.76 (8.68)

0.50 19 .62 -6.28 10.24 0.11

Head-On Approach

31.13 (4.16)

27.62 (3.50)

1.91 17 .07 2.903 12.26 0.42

Port Approach

32.38 (3.78)

24.69 (5.71)

3.61 19 .03 -0.022 7.041 0.62

of 303

The analysis reveals that the main effects of the protective cone and relative velocity

vector of FCAS may increase pilot confidence in flying and maintaining self-separation,

which allows them to minimise their heading change manoeuvres. This suggests that the

Ownship position is closer to the intended flight path, with the support of the protective

cone. These results show that the experimental group has a relatively more favourable

manoeuvre towards minimising heading change performance as compared to the control

group; hence, reject the null hypothesis (H-2).

The results in Table 5-3, regarding the Head-on approach, revealed that the control

group did not exhibit a higher mean change of flight path score (M=31.13, SD= 4.16, n

= 8) than the experimental group (M=28.26, SD= 3.00, n= 13), [ t (19) = 1.91, p >

0.07]. This also suggests that the Ownship position is closer to the intended flight path;

hence, reject the null hypothesis (H-2). However, for the Portboard approach, the group

means for the same sample suggested that pilots in the experimental group (M =24.62,

SD = 5.71, n=13) have a lower mean than that of the control group (M =32.38, SD=

3.78, n=8). The assumption of normality was assessed by the Shapiro-Wilks test (p >

0.05) and group means was found to be normally distributed. The assumption of the

homogeneity of variance was also tested and found to be insignificant via Levene’s F

test, F (.94), p = .303. Using an alpha level of 0.05, an independent sample t-test was

conducted to evaluate whether the experimental and control groups differed

significantly. The t-test was significant at [t (19) = -3.36, p = 0.03 (two tailed)]. Based

on these t-test results, the study rejects the null hypothesis (H-2). Therefore, it can be

concluded that there was a significant difference between the two group means in

heading scores. The effect size, as estimated by Cohen, is d = 0.63, which corresponds

to a percentile standing of 73%. By comparing the two groups, the percentile suggested

that the average participant in the control group would have scored 73% lower than that

of a participant in the experimental group that was initially equivalent.

As discussed in Chapter 4, horizontal conflict resolution can be achieved if one or both

aircraft manoeuvre to cause VR to rotate by an angle, so that the relative velocity vector

line lies tangentially along either side of the protective cone. Larger magnitude of

means (i.e., average change of an aircraft’s heading) for the control group suggest that

the pilots could have exceeded the required heading change by more than –2µ (β-α)

of 303

(Bach, Farrell and Erzberger, 2009). Similar situations are likely to occur with TCAS.

The most frequent alert issued by TCAS is to “Adjust Vertical Speed”. This alert may

be misinterpreted by pilots as to either “reduce or increase” the current vertical speed to

avoid a collision. Thus, it may lead pilots to burst the safety altitude.

5.3.2.3 Manoeuvre preference

A pilot should be able to choose an appropriate manoeuvre required to avoid air traffic

in a free-flight environment. However, this freedom is constrained by environmental

factors, such as the Intruder, and aircraft performance. Table 5-6 and Table 5-7 show

the frequency and percentage of the participants’ preferences, as classified by the group.

A Port approach illustrates that the Intruder is on a collision course, approaching from

the left-hand side of the screen (in the 4th quadrant), while a Starboard approach

indicates that the Intruder is approaching from the right-hand side of the screen (1st

quadrant). However, with the Head-On approach, both the Ownship and Intruder are

approaching each other. The result suggests that for the control group, an Intruder

approaching from the Starboard may have introduced “right-of-way” ambiguities.

Similar ambiguities may have been introduced when the Intruder was on a converging

course with the Ownship. The present findings are consistent with Cashion et al. (1997),

who maintain that the probable explanation for this finding is that there were some

differences in the type of manoeuvring decisions made by the participants. These

differences were also related to the interpretation of the Visual Flight Rules right-of-

way procedures, but only consistent when the Ownship is allowed to manoeuvre, or

have the right-of-way to avoid collision. Another possible explanation for these

ambiguities may lie in a pilot’s set of personalised techniques or in having insufficient

information to perform collision avoidance effectively (Degani and Wiener, 1994).

The results in Table 5-4 reveal that the control group right-of-way interpretation was

followed consistently for the Head-on approach (87.5%) of right manoeuvre, but

slightly less consistently for both the Port and Starboard approaches. The participants

had to assume the right-of-way manoeuvre for the Port (75.0%) and Starboard (62.5%)

approaches.

of 303

Table 5-4: Frequency and percentage of pilot preferences for the control group (N=8)

Port Approach Starboard Approach Head-on Approach

Left

Manoeuvre

Right

Manoeuvre

Left

Manoeuvre

Right

Manoeuvre Left

Manoeuvre

Right

Manoeuvre

n % n % n % n % n % n %

2 25 6 75 5 62.5 3 37.5 1 12.5 7 87.5

In the experimental group, the results of pilots' avoidance manoeuvres in the starboard

approach indicate that the group avoided collision by flying behind the Intruder, as

compared with the control group. However, participants in the experimental group with

the FCAS aid them to be more consistent avoiding. The previous findings of Cashion et

al. (1997) with a conventional display are not consistent with the present findings.

Thepossible explanation of these consistencies is that the pilots in this group used the

protective cone’s feature to perform collision resolution, as defined by the protective

cone, while the Ownship was initialized to pass in front of the Intruder.

The results of the participants’ responses in Table 5-5 revealed that the experimental

group’s “right-of-way” interpretation was followed consistently for the Head-on

(92.3%) and Starboard (84.6%) approaches, but less consistently for the Port approach

(7.69%). The participants did have to assume the “left-of-way” manoeuvre for the

Head-on (7.69%), Port (75.0%) and Starboard (62.5%) for manoeuvres behind the

Intruder. The results of this current study suggest that, for the experimental group, the

protected cone supported pilots in avoiding the Intruder, which resulted in consistently

coordinated manoeuvre preferences behind the Intruder, as compared with the control

group. Thus, the overall participants’ responses appear to differ on Starboard and Port

approach manoeuvres. This could be due to non-standard entries on how to avoid

conflicts at a conversion angle.

of 303

Table 5-5: Frequency and percentage of pilot preferences for experimental group (N=13)

Port Approach Starboard Approach Head-on Approach

Left Manoeuvre

Right Manoeuvre

Left Manoeuvre

Right Manoeuvre

Left Manoeuvre

Right Manoeuvre

n % n % n % n % n % n %

12 92.3 1 769 2 15.38 11 84.6 1 7.69 12 92.3

5.3.3 Mental Workload

This section presents the results of the pilots’ subjective ratings on the NASA-TLX

scale for mental workload (see Appendix E). The evaluation of pilot mental workload

constraints can be identified during the design process of a new system, or the re-

designing of an existing system, which include constraints, such as task completion

time, interruption, information overload and task difficulty (Andre and Hancock, 1995;

Iqbal, Zheng and Bailey, 2004). Pilots are the core part of a human-machine interaction

therefore, addressing these constraints is critical to the design of an efficient and safe

system (Andre and Hancock, 1995). According to Andre and Hancock, pilot mental

workload is likely to change to accommodate new constraints introduced by the new

system.

Taken together, previous research indicates that mental workload is a representation of

a mental construct that expresses the users’ cognitive demands as a result of performing

a task under specific environmental and operational conditions, coupled with the users’

abilities and capabilities to respond to those demands. Furthermore, a subjective

measure of mental workload (NASA_Task Load Index-NASA_TLX) has been used

extensively to identify specific workload components that take advantage of pilots’

information processing capacity by demanding their attention when performing

vigilance tasks (Wickens and Hollands, 2000).

The results of a t-test comparing mental workload scores of the NASA-TLX dimensions

(i.e., these different components are independent) between the two groups are shown in

of 303

Table 5-6, and the subjective rating is also presented in Table. From Table 5-6, it is

clear that both mental demand and performance scores are statistically significant

between the two groups. The finding shows that the control group does exhibit greater

mental demand than the experimental group, which suggests that the experimental

group has the ability to perform the tasks and activities assigned to them with less

mental demand or effort. This result seems to suggest that the improvement in the

participants’ performances associated with the FCAS did not come at the cost of any

consistent increase in mental demand due to FCAS features, including the protective

cone, as compared to the conventional display used in the control group. Despite the

performance scores being relatively highly rated by the control group, it was suggested

that the participants in this group tried their hardest to get their best performance;

though the frustration level increased. The mental demand result seems to indicate that

the improvement associated with FCAS may have not come at the cost of any consistent

increase in mental demand ( a component of mental workload), which may be due to

FCAS features, as compared to the control display.

To test the hypothesis (H-3) that pilots in the experimental group will have a lower

mental workload (overall) when avoiding conflict than the control group, an

independent sample t-test procedure was used. The t-test was not significant, [t (19) =

1.74, p = 0.098 (two tailed)]. As the study acknowledged, one of the main problems of

using subjective workload assessment technique is that is not sensitive to a low mental

workload (Ameersing and Goonetilleke, 2001). Another possible explanation of lack of

statistical significance of the overall mental workload is probably a consequence of

some limitations in the experimental design (see Section 6.2). Therefore, the current

study retains the null hypothesis of no mean difference. That is, the experimental

group’s mental workload (M = 39.15, SD = 10.22) is not significantly lower than that of

the control group (M = 48.07, SD = 13.17).

of 303

Table 5-6: Results of the different task demands level by NASA-TLX scores

Dimensions

Mean (SD)

95% CI

Cohen’s d

Control

Group

Experimental

Group t

df

p

Lower limit

Upper limit

Mental Demand

53.44

(9.67)

26.92

(3.01)

2.62 8.34 .03 8.91 44.11 .54

Physical Demand

19.43

(3.80)

16.15

(3.46)

.62 19 .54 -7.89 14.45 .14

Temporal Demand

36.93

(5.70)

36.73

(6.23)

.23 19 .98 -18.96 19.36 .01

Performance 32.19

(6.26）

29.61

(5.03)

-3.62 19 .02 -20.60 -5.51 0.61

Effort 69.06

(3.71)

82.12

(1.69)

1.37 19 .18 -14.34 19.48 0.071

Frustration 41.56

(11.52)

18.26

(4.78)

1.87 9.56 .09 -4.73 51.31 .41

The assumption of normality was assessed by the Shapiro-Wilks test (p > 0.05). Mental

workload was normally distributed. Thus, for the control group, the 95% confidence

interval for the skewness score ranges from 0.163 to 1.667 and the kurtosis score ranges

from – 2.05 to 0 .908, with the experimental group measuring a skew of < |1.0| and a

kurtosis of < |2.0|. These values for both skewness and kurtosis are lower than 1.96,

which is adequate to establish the normality of the data used in this study (Cramer and

Howitt, 2004). The assumption of the homogeneity of variance was found to be not

significant when examined via Levene’s F test, F (1.27), p = .274. The estimated

Cohen’s d = 0.76 corresponds to an r- value of .330. The square of the r-value, which is

the percentage of variance in the mental workload ratings, provides an explanation for

pilots in the experimental and control groups, and is 33.0%. However, from the results

of 303

in Table 5-8, it can be seen that both groups experience physical demand, temporal

demand, effort, and frustration during the task. The probable explanation of this finding

is that these groups devote their mental effort to task activities to achieve the desired

performance, and/or maintain it. This finding is supported by the previous findings of

Moray (1986), who proposes that there is a relationship between mental workload and

performance.

5.3.4 Situational Awareness

The purpose of investigating the system’s usability is to understand how pilots feel

about the display configurations under conflict situations for both groups. The results

observed have provided the mean ratings for each of the seven questions. The overall

mean ratings for both groups suggested that pilots in the experimental group produced

higher preferences, as compared to the pilots with a conventional display.

The FCAS is designed to support pilots when they perform manoeuvres to avoid

violating separation standards. The current study conducted analyses to investigate

whether the participants in the experimental group used the Ecological Interface

Design’s display to improve situation awareness while performing collision avoidance

tasks. A questionnaire was used to evaluate the system’s usability (see Appendix E). It

contained seven questions, and the results offered a view of the system’s usability of the

proposed FCAS as a contrast to a control display, as shown in Chapter 4. Answers were

sought regarding if the corresponding subscales of the questionnaire measure similar

aspects between the two groups. This is important as it is of interest to know whether

the revealed results present a merely statistically significant difference, or if there are

more fundamental discrepancies. The questions also sought to determine if the

questionnaire administered to both groups covered the same contents as of the display,

or if they were only covered by FCAS and not by the control display. Cronbach’s alpha

internal consistency indicator was used to estimate the reliability of the five (5)-item

scale of questionnaires. The coefficient, shown in Table 5-7, suggests a good level of

internal consistency, α = 0.77. All items appeared to be worthy of retention, as

presented in the Table. Moreover, with item Q1 deleted, the alpha has risen to 0. 842,

which indicates a high level of internal consistency, thus, supports the use of

independent t-tests for further analyses.

of 303

Table 5-7: Reliability of assessment results

Post-Trial Questionnaire Contents

Reliability Analysis


0.085

Q2 How useful was the system for understanding the location of the Intruder relative to the Ownship?

0.445

Q3 How useful was the system for understanding the direction of travel of the Intruder?

0.698

Q4 How useful was the system for avoiding conflict? 0.693

Q5 How useful was the system for avoiding stall when manoeuvring the aircraft?

0.425

Q6 How useful was the system for providing sufficient information for following a desired flight path?

0.694

Q7 Please rate your overall opinion of the system. 0.649

A five-point scale questionnaire was administered to both the experimental (n =13) and

control (n=8) groups. A higher rating would suggest that participants in this group

should have improved conflict situation awareness, due to a clear presentation of

conflict situations and configurations (H-1). However, a lower rating is expected to be

less preferable because pilots lack a clear presentation of conflict situations and

configurations, such as relative velocity vectors and a protective cone. The data analysis

focused on comparing the mean difference between the experimental and control

groups, a t-test procedure was adopted using SPSS software to compare the mean

differences. Prior to conducting the analyses, the assumption of normality was not

violated as examined via the Shapiro-Wilks test (p > 0.05). The assumption of the

homogeneity of variance was also found to be insignificant as examined via Levene’s F

test, F (3.80), p = .066. To test the stated hypothesis (H-1), an independent t-test

procedure was used, and the results of the mean of the individual questionnaire ratings

for both the experimental and control groups are presented in Table 5-8.

of 303

Table 5-8 Display mean ratings on seven subjective preferences questions for the experimental and control groups

Q3 How useful was the system for understanding the direction of travel of the Intruder?

3.00 (1.02)

4.33 (0.75)

3.45 19 0.003 .52 2.13 .60

Q4 How useful was the system for avoiding conflict?

3.56 (0.73）

4.15 (0.52)

2.15 19 0.045 .02 1.15 .42

Q5 How useful was the system for avoiding stall when manoeuvring the aircraft?

1.73 (1.48)

3.10 (1.23)

0.50 19 0.034 .12 2.6 .45

Q6 How useful was the system for providing sufficient information for following a desired flight path?

2.32 (1.54)

3.81 (0.80)

2.53 9.35 0.031 .17 2.18 .52

Q7 Please rate your overall opinion of the system.

3.51 (1.04)

4.17 (0.44)

2.07 19 0.053 -.01 1.34 .38

9 Experimental group ; Cohen’s effective size (*)

Post-Trial Questionnaire Contents

Mean (SD)

t df p

95%CL

d* Control

Group Experi.9

Group Lower limit

Upper limit


1.55 (0.39)

1.85 (0.90)

1.04 17.62 0.312 -.30 .89 .21

Q2 How useful was the system for understanding the location of the Intruder relative to the Ownship?

4.03 (0.54)

4.25 (0.63)

0.84 19 0.413 -.33 .78 .18

of 303

Individual questionnaires rating mean, confidence intervals, and t-tests will not be

discussed. However, the contribution of this analysis has yielded four meaningful

constructs, Q3, Q4, Q5 and Q6, by examining the statistically significant (p < 0.05)

difference between these two groups.

In the following analysis, the overall questionnaire ratings of the experimental and

control groups were considered. Furthermore, homogeneity of variance was not violated

as assessed via Levene’s Test for Equality of Variances (p = 0.066). The overall

questionnaire ratings for both the experimental and control groups were normally

distributed, as inspected via the Shapiro-Wilks test (p > 0.05). The independent t-test

results revealed that the experimental group’s situation awareness improved (M =

26.65, SD = 1.62) compared to the control group (M = 19.69, SD = 3.43). However, this

difference was statistically significant, t (19) = 6.31, p = 0.00 (two tailed), 95% Cl

[4.65, 9.26], but there is a statistical difference in overall questionnaire ratings between

the experimental and control groups. Therefore, the study rejects the null hypothesis and

concludes that the experimental group with the EID display has improved situation

awareness, as compared with the control group in agreement with H-1. From the stated

information, there is sufficient evidence to suggest that the overall questionnaire ratings

are different for pilots in both the experimental and control groups. This result reveals

that the calculated Cohen’s effect size value (d = 0.79) suggests a medium practical

significance between the experimental and control groups. Using a conventional

approach to interpreting this value (Coe, 2002), the effect size of 0.79 corresponds to

76% as the average percentile standing of the average participant in the experimental

group, relative to the average participant in the control group. In comparison, the

percentile standing of 76% suggests that the average participant in the experimental

group would score 76% higher than a participant in the control group that was initially

equivalent. As expected, the protective cone improved pilots’ situation awareness in the

experimental group.

of 303

5.3.5 Correlation Analysis

In order to conduct the multivariate analysis of variance (MANOVA), a number of

Pearson product–moment correlation coefficients between performance variables [i.e.,

the perceived aircraft heading and separation (as measured by the flight simulator)]

were performed to test the MANOVA assumption that the performance variables would

be correlated, ranging from .20 to .60 (Meyer, Gampst, and Guarino, 2006). The data

showed no violation of normality, linearity or homoscedasticity. Table 5-9 shows the

results of the correlation analyses on the separation, heading workload and situation

awareness.

As presented in Table 5-9, from inspection of table, there was a moderate strength,

negative correlation between heading and separation for Port. The correlated

relationship is statistically significant, (r = -.509, n = 19, p < .05) with high levels of

perceived aircraft heading associated with a lower perceived separation. This finding

indicates that an increased heading change led to minimum reduction of separation (i.e.,

deviation). There is also a moderate strength negative correlation between separation

(Port approach) and separation (Head-on approach), and is statistically significant, (r = -

.566, n = 19, p < .01). This finding demonstrates that an increased in separation during

Port manoeuvres also increases separation (i.e., deviation) in Head-on approach, which

shows that participants who rated themselves more highly on separation in Port

manoeuvres tend to also rate themselves more highly on separation in the Head-on

manoeuvres. A moderate strength positive correlation between separation (Head-on

approach) and Heading (Port approach) was observed and is statistically significant (r =

-.589, n = 19, p < .01). In this case, those participants who rated themselves relatively

low on separation in Head-on manoeuvres would rate their performances relatively high

on heading change on the Port approach.

All observed performance measures (separation and heading) and situation awareness

were not, however, significantly correlated with workload, and this indicates that the

greater time spent was associated with an increased perception of workload.

Considering the overall results on performance and workload, these findings indicate

that pilots clearly discriminated the constructs of display features and workload.

of 303

The results revealed that there were only one strong positive associations between

separation and opinion ratings for Port measures and is statistically significant (r = .600,

n = 19, p < .001). In this case, participants who rated their performance relatively low

on separation in Port manoeuvres would place themselves relatively high on situation

awareness measures as shown in Table 5-9.

Table 5-9 Bivariate Correlations among separation, heading, workload and situation awareness (N=21)

Dependent variables 1 2 3 4 5 6 7 8

1. Port Approach: Separation

r - .075 .566** -.509* -.098 -.321 -.190 .600** p .746 .007 .018 .674 .156 .410 .004

2. Starboard Approach: Separation

r - .089 -.022 -.029 -.005 .001 .241 p .700 .924 .899 .982 .996 .292

3. Head-on Approach: Separation

r - -.589** -.206 -.094 -.012 .265 p .005 .370 .687 .959 .246

4. Port Approach: Heading

r - .240 .232 .154 -.334 p .294 .312 .504 .139

5. Starboard Approach: Heading

r - .390 .099 -.274 p .081 .669 .230

6. Head-on Approach: Heading

r - .396 -.360 p .076 .109

7. Workload

r - -.315 p .165

8. Situation awareness (Overall_Opinion)

r - p

Note: Correlations marked with an asterisk (*) were significant at p < .05, **. Correlation is

significant at the 0.01 level (2-tailed). The degrees of freedom for a correlation are n-2.

of 303

5.3.5.1 Multivariate Analysis of Variance (MANOVA)

Multivariate effect

In contrast to ANOVAs, MANOVA differs in that it takes into account multiple

independent and dependent variables (i.e., experimental and control group). The

performance dependent variables selected for analysis are heading and separation in

order to find a linear combination of these variables. MANOVA was originally

performed to avoid amplifying the Type 1 error rate (Cramer and Bock, 1966). Because

a test can be conducted only once to investigate the relationship between independent

and dependent variables, the risk of committing a Type I error is minimised in the

follow-up ANOVAs.

Preliminary assumption testing was conducted for normality and multivariate outliers

not to be violated. The Levene’s F test was used to determine the equality of error and a

variances test was used to examine the homogeneity of variances of each dependent

variable. The Levene’s test indicated homogeneity of between-group variance for

separation measures (significance <.05), but not for heading measures (significance >

0.05). Therefore, the homogeneity for separation is violated.

To address this violation, an independent one way ANOVA (with Brown–Forsythe’s F

and Welch’s F statistics) was used. There is still a highly significant difference in

separation measures (i.e., deviation) across the groups (i.e., experimental vs. control),

Welch: F (1, 8.480) = 10.037 p = .012. The violation of homogeneity of variance poses

no threat to the validity of the current study’s results (Erceg-Hurn and Mirosevich,

2008). Another assumption of the MANOVA that must be satisfied is the covariance

matrices of the dependent variables, which are the same across the experimental and

control groups. Box's M test of equality of covariance matrices tests assumptions. Box's

M (8.097) is not significant: F (3, 7394.20) = 051. Therefore, the current study has not

violated the assumption of equality of covariance matrices between the experimental

and control groups and, therefore, Pillais’ Trace is an appropriate test to use for the

purposes of MANOVA.

of 303

Thus, the hypothesis of equal homogeneity of variance-variance-covariance matrices

cannot be rejected because the p value is greater than 0.05. Therefore, the null

hypothesis that the covariance is not homogeneous was rejected.

To test the hypothesis that there would be a difference between the experimental and

control groups, a one-way MANOVA was conducted. The overall multivariate effect is

statistically significant via Pillais’ Trace = .54, F (2, 18) = 10.35, p < .001, the

multivariate η2 = .97 which indicates that approximately 97% of multivariate variance

of both separation and heading change are associated with group factors. This supports

the conclusion that the protective cone effect is significant. The multivariate effect size

(d) was estimated at 0.87, and Power (1-β err prob) is at 95 (i.e., excellent). The

statistically significant MANOVA allowed a follow up with both the univariate and

discriminant analyses. As stated in the previous paragraph, the multivariate effect was

found to be statistically significant. Therefore, the univariate analyses were conducted

to provide information about the effect of the independent variable on heading and

separation measures separately. Univariate independent one-way ANOVAs suggest that

the protective cone has a significant effect on heading change, F (2, 27) = 9.73, p < .05,

d = 0.87 and on separation measures, F (2, 27) = 5.23, p < .05, d = 3.6. Thus, the current

study can conclude that the protective cone has aided pilots in the experimental group to

less frequently violate the minimum separation standard, as compared with the control

group who were without an aid.

The effects of flying hours on dependent variables

As there was a significant difference in the number of hours flown between the control

and experimental groups, we added flying hours as a covariate to the multivariate

analysis. This analysis was to ensure that pilot experience was not confounding our

results.

o Multivariate (before covariate adjustment)

These results were reported in a series of tables prior to covariate adjustment. The first

part of the MANOVA analysis gives the results of the multivariate tests. These results

suggest that there was a significant effect of group on a linear combination of the

dependent variables (i.e., heading, separation, situation awareness and mental

of 303

workload). The overall result suggests that there is a highly significant multivariate

effect across the group for the combined dependent variables. From inspection of the

Table 5-10, Pillai’s Trace =.822; F (8, 12) = 6.934; p = 0.02. In this case, the dependent

variables are all significant (p < .05). Therefore, we can conclude that Group did have a

significant effect on the four different dependent variables.

Table 5-10 Multivariate Tests of Significance (before to covariate adjustment)

However, the second part of the results section gives univariate tests for the effects of

group membership on each of the different dependent variables. Tabulated data (not

reported here) revealed that univariate outcome before covariate adjustment indicated

that group membership had a significant effect (p-values): Port_ Separation (p = .001);

Head-on_Separation (p = .000); the Head-on _Heading (p = .051);

Overall_Pilot_Opinion (situation awareness) (p = .005), and Port_ Heading (p = .003).

Only three univariate effects were not significant for this effect: Starboard_Separation

(p = .529); Starboard_Heading (p = .622); and Mental Workload (p = .098).

o Multivariate (after covariate adjustment)

The results after adjusting for the covariate of flying hours can be seen in the Table 5-

11. The MANCOVA indicates that the multivariate outcome is weaker subsequent to

applying covariate adjustment. That is, it appears that the covariate has increased some

of the error variance. There is no significant multivariate effect of flying hours on the

set of dependent variables: Pillai’s Trace =.547; F (8, 11) = 4.995; p = 0.08 as showed

in the table below. Therefore, based on this analysis, we conclude that flying hours did

not have a significant effect on the four different dependent variables across the group.

Effect Value F Hypothesis df Error

df Sig. Partial Eta Squared

Intercept Pillai's Trace 1.000 35630.069b 8.000 12.000 .000 1.000

Group Pillai's Trace .822 6.934b 8.000 12.000 .002 .822

of 303

Table 5-11 Multivariate Tests of Significance (after to covariate adjustment)

Effect Value F Hypothesis

df Error df Sig. Partial Eta Squared

Intercept Pillai's Trace 1.000 6476.601b 8.000 11.000 .000 1.000

Group Pillai's Trace .784 4.995b 8.000 11.000 .008 .784

Hours Pillai's Trace .547 1.660b 8.000 11.000 .214 .547

Tabulated data (also not reported here) revealed that the univariate outcome after

applying covariate adjustment indicates that the overall effects of flying hours on each

of the different dependent variables is weaker (compare this with the outcome prior to

applying covariate adjustment) across the group membership: Port_ Separation (p =

.001); Port_ Heading (p = .029); Head-on_Separation (p = .002); Head-on_Heading (p =

.013); Overall_Pilot_Opinion (situation awareness) (p = .010). Similar to the

MANOVA analysis, three univariate effects were not significant for this effect:

Starboard_Separation (p = .530); Starboard_Heading (p = .505); and Mental Workload

(p = .193). Therefore, the inclusion of the covariate taking into account flying

experience has changed the overall results. On inspection of the table below, there is no

significant interaction between “dependent variables” and “hours”. These results

suggested that the requirement for Homogeneity of Regression slopes has not been

violated, which is precisely what the current study has expected as shown in Table 5-

12.

of 303

Table 5-12 Homogeneity of Regression slopes: Dependent Variables vs Hours, by Group

Tests of Between-Subjects Effects: source ( Group * Hours)

Dependent Variables

Type III Sum of Squares

df Mean Square

F Sig.

Port approach

Separation

.361 1 .361 2.122 .163

Starboard approach

Separation .016 1 .016 .079 .782

Head-on approach

Separation .009 1 .009 .504 .487

Port approach

Heading 63.215 1 63.215 3.156 .094

Starboard approach

Heading 184.670 1 184.670 2.496 .133

Head-on approach

Heading 14.579 1 14.579 1.164 .296

Mental Workload 72.415 1 72.415 .519 .481

Situation Awareness

( Pilot Opinion) 36.976 1 36.976 2.970 .103

of 303

5.3.5.2 Discriminant Analysis (DA)

The MANOVA was followed up with discriminant analysis (DA), to help interpret the

statistically significant MANOVA effect. The DA is a multiple linear regression that

predicts an outcome. The eigenvalues, Wilk’s Lambda, and standardized discriminant

function coefficients were consulted, which revealed one discriminant function. In

Table 5-13, the independent variable was associated with two levels (experimental and

control groups), one eigenvalue, and canonical correlations were extracted by the

MANOVA. The eigenvalue of 2.692 accounted for the proportion of variance

explained.

Table 5-13 Eigenvalues table

Eigenvalues

Function Eigenvalue % of variance Cumulative % Canonical correlation

1 2.692a 100 100 .854

a. First 1 canonical discriminant functions were used in the analysis.

The canonical correlation value suggests a high correlation, r = .854, which indicates

that the model explains 73.0% of the variances in the canonical discriminant functions

of the variation in the grouping variable (i.e., experimental and control groups). The

results of the discriminant analysis were consulted to determine whether a pilot was in

the control, or experimental group. The predictor variables were separation and

heading. There are significant differences in the mean for these predictors on the

dependent variables (i.e., separation and heading). Though the log determinants show

the same trend, Box’s M indicated that the assumption of equality of covariance

matrices was not violated. The overall Wilks’ Lambda suggests the significance of the

discriminant function, λ =.27, χ2 (2, N=21) = 20.90, and the test function indicates a

highly significant function (p < .005), indicating that overall, the predictors

differentiated among the two performance groups as presented in the Table 5-14.

of 303

Table 5-14 Wilks’ lambda table

Wilks’ Lambda

Test of function(s)

Wilks’ Lambda Chi-square df Sig.

1 .271 20.897 6 .002

The proportion of total variability that was not explained is 27.0% and, as such, based

on this sample size, it can be concluded that there is statistically significant

discriminating power in the variables included in the model. The Discriminant Equation

(i.e., discriminant analysis) can therefore be formulated. To formulate the discriminant

function (equation), the un-standardised coefficients are used to assess each

independent variable’s unique contribution to the discriminate function, and thus

provide information on the relative importance of each variable as follows:

- (5.1)

Where D is the dependent variable, “a” is the constant obtained via “Canonical

Discriminant Function Coefficient”, bn is the corresponding output for un-standardised

coefficients, xn is the independent variables. Here we have two predictor factors via

separation and heading change.

The standardised discriminant coefficients (or weights) were consulted to assists in the

interpretation of the statistically significant MANOVA effect

The decision rule classification is expressed in equation 5.2:

(5.2)

Where lower centroid is for the control group, higher centroid is for the experimental

group, and n1 (the control group), n2 (the experimental group) are the mean for both

groups, respectably.

The model, as expressed in equation 5.3, will predict if a pilot has the potential to be in

the experimental or control group. From the canonical discriminant function

coefficients Table 5-15,

of 303

- (5.3)

Table 5-15 Canonical discriminant function coefficients (Unstandardised)

Function

1

Port Heading -.067

Port Separation .692

Head-on heading -.0.194

Head-on separation 6.148

Starboard Separation .449

Starboard Heading .049

(Constant ) -99.709

A further way of interpreting discriminant analysis results is by estimating the group

centroids (i.e., canonically derived group mean) for the two levels, as presented in the

Table 5-16. Because the participant numbers in the two groups are not equal [i.e.,

experimental (13) and control (8)], the study uses weights on the centroids to assess the

diving point. As presented in the table below, the results reveal that the control group

has a mean of -1.989 while the experimental group produces a mean of 1 .224.

of 303

Table 5-16 Unstandardized canonical discriminant functions evaluated at group means

Functions at Group Centroids

Function

1

Control group -1.989

Experimental group 1.224

Therefore, participants with performance measures close to a centroid are predicted to

belong to that group, as indicated in Equation 5.4.

–

(5.4)

Furthermore, the classification results (Tabulated data is not reported here) of the

predicted group membership revealed that 78.9% of respondents were classified

correctly into ‘experimental’ or ‘control’ groups. Pilots in the control group were

classified with slightly better accuracy (87.5%) than pilots in the experimental group

(76.97%). The classification results additionally revealed that out of the eight (8) pilots

classified as the control group, only one (1) case had been wrongly classified as

belonging to the experimental group (7.7%). With the experimental group consisting of

13 pilots, only three (3) cases were wrongly classified as belonging to the control group

(23.1%). Therefore, the accuracy of the model may be considered as adequate and that

the model has the capacity to predict whether a participant would be in an experimental

or control group. The analysis of the qualitative data will be discussed in the next

section.

of 303

5.3.6 Pilots’ Appraisement of FCAS Interviews were conducted with twenty-one (21) participants to elicit their mental

models of collision avoidance manoeuvres. The analysis of the qualitative data in this

section examined pilots’ interactions with the display. Appropriate passages from the

pilots’ comments (from interview transcript excerpts) were used to present results, in

terms of a pilot's cognitive behaviour. The participants’ transcripts, with their case

number placed in brackets, were included in the analysis. For example, C1 refers to a

participant’s case number in the control group, while E4 refers to a participant’s case

number in the experimental group. Situation awareness and cognitive control behaviour

were measured by recording comments made after the participants completed the

exercises. This section will present and interpret some representative excerpts, as well

as analyse the frequency of pilots’ responses to the questions posed below:

1. “What is your opinion about the system you just tried?”

2. “What features of the system support you to avoid collision?”

3. “What is your opinion regarding relative motion?”

Answers to these questions were analysed and the contents were then transcribed using

themes and relationships, as described by Van Someren, Barnard and Sandberg (1994,

p. 26). The overall distribution of the experimental and control group responses were

coded into six (6) themes, specifically, cognitive control behaviour, protective cone,

system’s usability, traffic situation awareness, clutter and suggestions. The total number

of responses was presented as a percentage. The following sections are representative

excerpts of some of the appraisal comments made by pilots for both the experimental

and control groups.

5.3.6.2 Pilots Cognitive Behaviour

This section addresses the following two specific interview questions posed:

“What is your opinion about the system you just tried?”

“What features of the system support you to avoid collision?”

of 303

The cognitive-control hierarchy of flying has been proposed by Rasmussen’s (1986)

skills, rules, and knowledge (SRK) taxonomy, as discussed in Chapters 2 and 3. The

current study examines the participants’ appraisal comments on the cognitive control

hierarchy (i.e., skill-based, rule-based and knowledge-based behaviour) related to

decision-making.

One of the pilots (E6) from the experimental group made reference to the influences of

approaching head-on, stating that each aircraft tends to alter its heading to the right.

“Right-of-way” rules, as practice, reflected on issues within the context of their flying

training program as evidenced by participant number E6:

“….When it comes to the aircraft with a direct head on collision at my 12 o’clock flying

directly at me, I will automatically turn to the right as we were taught at ground school.

That is when two aircraft are head on, both of you should turn to the right. That is why I

automatically turn to the right in that situation” (E6)

The following two comments by the same participant relate to reliability, safety, and

sources of information on decision-making. These participant’s comments reflected on

how he considered the alternatives and the importance of those alternatives before

making a decision as demonstrated below:

“If there is an aircraft at my 10 o’clock that is flying towards my 12 o’clock path, I

would l turn left to get behind the aircraft, I’m more secure…..,if turn left,. I’m sure I

will leave aircraft behind, but if I turn right there is still a long period time where the

aircraft is in your vicinity. I felt more secure being able to get out the way, straight

away, and knowing that I’m behind it. that is what I like about it. And the same thing

goes for the opposite direction.

When the aircraft is on my right let’s say 2 o’clock, I will turn to the right to get behind

the aircraft, rather than turning left and flying alongside it. I don't like .. turning left or

increasing air speed….., if I’m behind, I know that I’m out, that way I feel more

secure…” (E6).

of 303

A fundamental feature of flying practice on which pilots often reflect is activities during

their training that are compatible with their mental models. This is clearly illustrated in

the following narrative by a participant in the experimental group:

“We naturally go to the right because we always go towards to the right that is our

training” (E3).

Subsequently, the narrative below demonstrates acuteness of professional judgment and

understanding of strategies planning, as well as which approaches might work

effectively to avoid conflict traffic. The narrative suggests that the participant

considered the available options and weighed those options before making a decision:

“…If the traffic is approaching from the right or in front, I chose to change my heading

to the right without altering my speed. However, in the case of traffic approaching from

the left, I chose to reduce my speed and allow the conflict traffic to move ahead of

me.”(C1)

This finding supports the use of a display developed by the National Aeronautics and

Space Administration (NASA), or any other collision avoidance system display

available to date. For example, with the control system, the pilots are likely to reduce

speed or choose a lateral path stretch as the best solution (Ballin, Sharma, Vivona,

Johnson and Ramiscal, 2002). However, when changing the aircraft speed, it is not clear

which directions the participants are likely to manoeuvre to avoid the collision.

Conversely, if their current course was not direct, they would look for modifications

that minimise the distance to the destination (Johnson, Battiste, and Bochow, 2003). In

addition to reflecting on influences to training practice, which originated from flying

school initiatives, participants made reference to how system problems would have an

impact on interaction with the FCAS. For example, one of the participants provided

evidence of interaction with FCAS with this sort of behaviour:

“…when I first started to use it I didn’t know what to do, or what was happening…”

(E8)

of 303

5.3.6.3 Protective Cone Function

This section presents and addresses the following two specific interview questions

posed:


“What is your opinion regarding relative motion?”

As discussed in Chapter 2, in the context of this study, decision-making is primarily

concerned with which side of the relative velocity vector is used, in relation to the loss

of separation. The experimental findings provide evidence that showing a protective

cone and relative velocity vector significantly improves pilot decision-making in

collision avoidance. The comments by eight of the participants in the experimental

group, with respect to the use of the cone in deciding which way to turn to avoid

collision, were generally consistent.

The following narrative demonstrates how understanding the appropriateness of using

the protective cone supported this particular participant in avoiding conflict. The

participant’s comments considered the available options and weighed those options

before making a decision. This finding suggests that the protective cone aids the

participant in coping with both the complexity and interrelationships of physical

objects, which influence the participant’s learning and development:

“Yes I thought it is quite intuitive. It does give you the information you need to make the

right decision. Obviously if you go in front the aircraft you going to make a long turn,

and if go behind the aircraft it's shorter turn, so depending where you’re tracking or

heading to and various other factors, like maybe where our aircraft is located that kind

of thing , you may choose to go in front or behind. Obviously one is a little bit longer

than other. This gives you the cone as well. The wider the cone the bigger your turn has

to be and shorter the cone the shorter the turn has to be..”(E7)

of 303

The following two comments also suggest that these participants identified flight

constraints. These findings can, in part, be understood when perceiving these constraints

as to whether this decision can make an impact on distance of return. A shorter distance

could mean that pilots could reduce fuel consumption. Again, these participants also

considered the available options and weighed those options before making a decision:

“… so the line [line of sight] that protrudes to the middle of the cone is especially useful

in determining which way to go, given that if the red line of your projected cone is

already on the other side of the middle line, you already know which way to go so that

you don’t have to cover much distance” (E4).

“First of all, I think the most important thing or useful thing about the project is the

cone: especially when the cone is split in half, with a dotted line in the middle. It tells

me my relative track versus the relative vector of the Intruder's position…” (E12)

Several improvements on the current situation in collision avoidance are signalled in the

narrative below, which the pilots attributed to the protective cone and having engaged in

the process of reflecting on collision avoidance through action. This participant

indicated his abilities to systematically evaluate and search for reasons behind the

outcomes of the manoeuvre to avoid collision:

“…once the I knew what I was looking for, like, the red arrow to put over on the blue

line, once I understood to do that, then everything else, I think achieved its purpose of

avoiding the collision”(E8)

In the same way, three issues arise from the comment given by another participant in

the experimental group (see below excerpt). First, the participant appears to have been

wholly supported by the protective cone, in terms of reducing her/his mental workload.

Second, the participant might not have struggled to change aircraft direction at any

time, supported with the appropriate guidelines of the protective cone. Third, the

participant considered the options available to make a decision. Fourth, the interaction

with the system is intuitive.

of 303

“…you can choose if you want to go in front the aircraft or in behind that aircraft so it

is not really mentally challenging and it easy to decide whatever you feel you want to

do really on impulse”(E9)

One particularly evident finding, combining school training, rule-based behaviour, and

safety issues, is captured in the following narrative:

“The use of the protected zone and the relative speed it actually very useful. We are

trained that one must go to the right hand side of the traffic, depending whereabouts

they are coming from………… on the relative speed vector line…….. Involves safe”

(E14)

Equally important is the following finding that exemplifies a sequential pattern of

thinking used by the participant to deliberate over strategies he or she might use to

avoid collision:

“……..to avoid a collision, it does not really matter if you are going off track. You have

to go off track to avoid collision, so to me if you can avoid a collision just by a little bit,

it's good but, if you have go a long way it does not really matter”(E3)

It was suggested by one participants that the logical reasoning of dealing with the

Starboard or port approach, as the participant engages in the process of problem

identification and means-ends analysis to trigger the decision-making processes to avoid

collision. According to the participant:

“As for the other ones, it is kind of looking at the position of my aircraft in comparison

to the middle section of the cone and whether I can get ahead or behind the person,

where they were relative to my aircraft and to the point of the cone that is the main

deciding point really….”(E3)

Considerable improvements on the current situation to collision avoidance are stated in

the below narrative, which the pilots attributed to the protective cone of having engaged

in the process of reflecting on collision avoidance through action. The participants

expressed that their ability to systematically consider, evaluate and search for reasons

behind the outcomes of manoeuvre to avoid collision had improved.

of 303

“….so the only thing you need to do is a little bit of thinking. Because it does not

actually tell you any specific direction which you need turn to avoid. You have to pretty

much use your mind out to figure it out. In most cases it is not a difficult decision to

make depending how far away the aircraft is, because the further away it is the more

time you have to have to use your mental capacity to figure which direction to turn

depending on other aircraft's speed and distance”(C3)

In contrast, the following narrative suggests that, with the non EID, the participant

struggled to find an appropriate approach to avoid collision. The excerpt seems to

suggest that the participant was not able to access additional information to avoid

collision. In other words, flight constraints were not visible to the participant. This

finding indicates that the participants’ perceptions, and modifications to their plan of

action in a systematic way, help to guide the development of their strategies for

purposes of avoiding a collision. This might be explained by the use of trial and error

learning where participants’ behaviour seems to imply higher mental processes:

“…the other thing I realised about the instrument was that pilot has to stop and turn,

turn and stop, and then visualise whether I am colliding [with the Intruder] or not, and

stop and turn again, these are the only two disadvantages I found with this system…”

(C4)

According to two comments made by the participants who attempted to explain on the

issues consistent with limitations of the use of limited information, as recognised by

previous researchers (Rome et al., 2000). These comments validate the issue of how the

use of non-EID does not make flight constraints visible to pilots required to avoid

collision: “…it had relatively limited information, all that you have to do is basically

turn. You have to move away from the aircraft itself….” (C5)

In the following narrative, the participant in the control group reflects on how

judgments at angles are difficult and hard to predict, based on the movement of

Intruders in space:

of 303

“…have an idea where to turn, but you did not know, the maximum back angle that you

are meant to go to turn to, or you don’t know or you are not sure if you made certain

action if will it exceed a certain point where it may lead you back into the collision

traffic again…”(C6)

5.3.6.4 Traffic Situation Awareness (TSA)

This section addresses the three specific interview questions posed:



“What is your opinion regarding relative motion?”

The pilots were provided with a poster presentation of the display to re-familiarise

themselves with the different display symbology and the types of conflicts they

encountered during the experimental runs. Participants’ comments are related to having

a mental picture of the Intruder’s location, intention in relation to the Ownship position,

and flight constraints within the protective cone. This approach provided participants

with the ability to form a coherent picture of the conflict situation. Participant

comments demonstrated the importance of acquiring information from the surrounding

aircraft about the current conflict situation. From the findings, comments by five of the

participants in the experimental group reported that they had been influenced by the

way the system was able to display traffic situation awareness. The following extracts

are the most common types of reflective conversation found in the interview transcripts

in relation to this. The development of a 2D relative motion display is a fundamental

goal of the current study, with considerable emphasis being placed on using the 2D

inside-out over outside-in motion frame of reference. The following narrative supports

the use of the 2D inside-out display:

“Everything is in relative motion, so you turn everything else turn as well. It gives you a

better sense of where everybody is after you turned, because everything is moving

around now, moving with different relative speeds and that kind of thing. That is quite

good, yeah...” (E7).

of 303

The following narratives suggest the participants have evaluated and reflected upon the

evidence gathered that they were aware of the current situation to inform future

planning. Moreover, these findings clearly support the proposed model to include

navigational planning as part of the five-stage pilot information processing model. The

narratives also illustrate the use of relative vectors that provide the potential answers to

the research question raised, about factors that might improve participants’ decision-

making:

“…the protective cone lets you know well before……. if there is going to be any

collision, that is good…….plenty of time to react or maybe change your heading” (E7)

“...I like the concept of being able to make conflict resolution so far in advanced that is

certainly good….. it’s more of a deal of knowing where the other traffic was, as well as

which way ...so simple, looking at the motion vectors of the surrounding traffic …trying

to make a decision which way, …helped me not only with the traffic I’m trying to avoid

but any future potential traffic…”(E10)

It is noteworthy to report on how participants perceived and understood the situation in

the question, as influenced by features of the EID. These narratives support the

discussion on the exocentric 2D display with the frame of reference of north up.

“The next thing is the interface, I think it gives the broad picture of where the other

aircraft are and it provides me with good situation awareness of my current position.

(E12)

“I think it is user friendly. It is easy to determine where your Intruder is”… (E4)

“The display gives me an indication of which side the traffic is approaching from” (C1)

5.3.6.5 Clutter

This section addresses the following interview question:


of 303

As discussed in Chapter 2, a display that is cluttered may present pilots with incomplete

mental models. Indeed, comments by participants in the control group relate to having

too many aircraft on the screen at a given time. This finding revealed that cockpit

display traffic information that is cluttered has an impact on the participants as

commented. Previous work suggests that high traffic density does also have an

impact on controller performance (Hilbum, et al., 1997). The EID displays may also

suffer from lower usability due to the clutter and this may have interfered with the

observation of the conflict situation on the screen on first inspection. However, the

interference did not seem to upset interaction with the display, as described in the

following narratives:

“…I thought at times it look a little at bit cluttered, especially when the aircraft are

directly under you. Maybe at a certain distance you may just clear the screen when

there is no collision risk. If there is not collision risk and they are directly under you, it

could be too much on the screen at once…” (E7)

“…features many arrows on the screen…” (E8)

“…sometime it a little bit of clutter...” (E10)

Other participants stated:

“….the screen is quite cluttered; there is a lot on the screen… I have never seen so

much traffic on the screen. Normally when you go flying you’ve got TCAS, you normally

don’t see so much traffic. That was a lot of traffic…”(E11)

“…there is a bit too much information for the pilot or operator for what he is doing at

that time, but then progressively throughout the task , I then actually thought, okay,

actually…”(E14)

Similarly, participants’ comments in the control group relate to having too many aircraft

on the screen at a given time.

“….the first issue was… the screen was upload, very uploaded [information

overload]….”(C4)

of 303

5.3.6.6 System’s Usability

This section addresses a specific interview question:


System usability is widely viewed as the answer to questions regarding how participants

interact with a display. The current study identifies factors where the display may be

difficult to use. Comments on system usability were most common, with eight (8) of the

experimental and three (3) of the control group participants commenting on this issue.

This finding strongly suggests the perspectives of participants using these displays (to

both the new EID display and the conventional display) which appeared to be useful are

summarised as:

“Yeah, other than that I thought everything worked quite well” (E7)

“For the whole system what in my own opinion, yeah, it’s a good system...” (E8)

“The flight collision avoidance system provides easy ways to interpret the indication

once the Intruder is detected by using the blue cone system and then connecting the red

vector up… to avoid collision. It does not require much effort and not much mental

strain is required, and the physical demand is quite low as well.... All in all it provides

safe and results”(E9)

“Alright, so I really enjoyed the system. Actually, it was good, it's good having measure

vectors of other aircraft and be able to see them it terms of conflict resolution. I found it

quite simple... the actual display, the graphic, was fine. It's easy to understand and very

clear…”(E10)

“…but overall it was intuitive, yeah, the human machine interface is intuitive in my

opinion…” (E11)

“So overall the system is a very useful training tool in my opinion and it would be a tool

for the airlines because there is no way for them [pilots] to see and avoid..… depends

on relative direction, relative position so it is much better than the current system for

separation” (E13)

“….yeah, overall, it’s [the system] pretty good” (E14)

of 303

“the system provides a good display of vectors, however, the operator must be familiar

with how these vector need to be interpreted….”(C1)

“…it is alright, the system is quite good” (C4)

Importantly, comments were also made which validate problems associated with the

radar screen-system. As already indicated Chapter 2, the conventional system poses

cognitive challenges for the operators:

“…… it is not so much to do with the difficulty, it basically confusing, it is confusing. In

this setting it's okay, but under high pressure scenarios could be very dangerous” (C2)

“The first collision avoidance system was not very intuitive. A trained pilot would

perhaps able to manoeuvre out of the collision quite easily. This may not be the case,

however under pressure…”(C2)

5.3.6.7 Suggestions for further improvement

This section addresses the following interview question:


Comments by three participants in the experimental group and one participant in the

control group proposed ideas or plans for future considerations. This indicates that these

participants have evaluated and reflected on the type of display currently under

examination to inform of a future design, as demonstrated in the following narrative:

“… giving the directions which way to go or which way is the shortest. There could be

other factors like terrains or aircraft on the way so I don’t know how is going to work,

like the aircraft in my opinion on which is the best way to turn is something you may be

able to include”(E8)

“…but maybe a simple of a matter of being able to occasionally turn off the wind vector

or something at a particular times...” (E10)

“…..If you are able to de-clutter the screen a little bit that will be good...” (E11)

“…yeah it is a good system it can be improved a bit” (C3)

of 303

The next section will discuss previous work on concept revision. The concept revision,

problem is to correct a given, roughly correct concept. This problem will be considered

in the model of Wicken’s (1984) model of human information processing and Parker’s

(1973) model of mid-air collision factors.

5.4. A Proposed Revision of Parker’s (1973) Model

The comments presented by the participants illustrate that they were aware of their

aircraft’s position, relative to another aircraft, with the aid of the protective cone

function, which were the most useful pieces of interface information. They also

commented on how relative motion (i.e., relative speed) features of the display

facilitates in creating situation awareness. There was also evidence that the participants

observed many different features of the flight display.

The performance and situation awareness comments suggest that pilots were able to

avoid a conflict without violating the minimum standard, as supported by both relative

velocity vectors and the protective cone functions. From the EID perspective, these

functions mapped onto the abstraction hierarchy to externalise the pilot’s mental models

in a visual form of two aircraft in a conflict situation (Vicente, 1999). Mental models

are supported by the visualisation of conflict information directly perceived at all levels

of abstraction hierarchy. Conflict situations are easily and clearly noticeable, enabilising

direct manipulation of those functions on display. The visualisation of relations between

constraints for the affordance is fitted by those functions. Thus, pilots’ situation

awareness and decision-making would have allowed them to avoid conflict scenarios

with reduced mental demand. In particular, further consideration was given to the

protective cone to decide which manoeuvres are available to avoid collision (i.e., to fly

in front of, or behind the aircraft as a suggestion). The participants could establish the

range of information, such as relative velocity, and vector position within the cone, so

that minimum deviation from the original flight path could be achieved. For example,

with a head on collision, the participants normally applied the right rule to avoid the

Intruder. The converging conflict was decided, based on which side the relative velocity

vector is in relation to the line-of-sight, to give minimum deviation from the original

flight path.

of 303

A similar finding was reported by William et al. (2011) who found that participants’

comments revealed that information such as relative distance, bearing to Intruder and

relative velocity of Intruder should be included in the redesign of a traffic collision

display such as TCAS. The Mid-Air Collision Factors (MACF) model proposed by

Parker (1973) indicates that many aircraft mid-air collisions at closure are a result of

pilots not being able to see another aircraft in a traffic pattern, due to vision envelope

restrictions, the pattern flown, and the type of manoeuvres involved. Figure 5-6 presents

a proposed adaption of Parker’s (1973) model with the recommended variation of

replacing the stage movement with “relative movement”, as shown in bold. Constraints

such as velocity, acceleration, displacement, time and trajectory can then be imposed on

the relative movement. The word protective cone, shown in bold, will also replace the

stage vision envelope. With the conventional display, it is evident that pilots lack

relative movement information in the field of view during collision avoidance.

In sum, the developments of the FCAS currently proposed by this study include the

means for capturing pilots’ rational and intuitive decision-making during conflict

resolution by evaluating relative movement and protective cone display. It is also

evident that the participants were very enthusiastic about the inclusion of the protective

cone and relative motion functions.

MID-AIR COLLISION

FAILED TO SEE- AND- AVOID

FAILED TO LOOK

Cockpit Duties

Runway Fixation

Distractions

Background

Contrast

Visibility

Relative Movement

LOOKED BUT FAILED TO SEE COULD NOT SEE

Protective Cone

Traffic Pattern

Manoeuvres

Figure 5-6 A revised Parker’s (1973) Mid-Air Collision Factors (MACF) model

of 303

5.5. A Proposed Revision of Wicken’s (1984) Model

The participants’ comments relate to an intuitive decision-making approach, as

discussed in Chapter 2. In particular, the participants’ decision-making was able to

capture the available options and the criteria for evaluating their choices of conflict

manoeuvres. The participants consciously evaluated the options suggested by the

display, based on their relative importance and expected outcomes. They then used this

information to choose the option that had the highest safety margins.

The planning analysis also plays a crucial role in many conflict situations, especially

when the participants have a clear set of criteria and weight (i.e., value) to deal with

options to avoid collision. It is evident that the participants’ comments take into account

choosing a destination, evaluating alternative flight paths, and deciding the specific

course of their plan before making a decision and then implementing this decision

(Kahneman, and Tversky, 1984). These comments further suggest that time also plays a

crucial role in shaping their decision-making processes.

The proposed modification of the model presented in the current study is to support the

design of an interface that supports pilots with strategic planning capabilities to manage

and resolve air conflict. Wicken’s (1984) model refers to a framework of human

information processing. The framework outlined in the literature illustrating decision-

making considered a number of stages to include sensory store (e.g., light, sound,

touch), perception (i.e., the abstract separation of a whole into its constituent parts in

order to study the parts and their relations), generation and assessment of alternatives

(i.e., decision making), working memory, the selection of a course of action (i.e.,

implementation) and attentional resources (i.e., mental concentration, awareness).

Attentional resources as a limiting factor for the last three stages (i.e., working memory

decision making and implementation The findings of the current study suggest that as

participants developed situation awareness, they also considered using criteria (i.e.,

clockwise and anti-clockwise rotation resolution) and value (i.e., a shorter and longer

distance from the intended flight path) as supported by the temporary planning task.

of 303

As previously discussed in Section 2.4, the Wicken’s model of human information

processing should be revised to include a temporal planning task to support the design

of the FCAS with attentional resources as a limiting factor. The resulting model should

be compatible with a pilot’s planning processes (i.e., spatial planning task) for avoiding

air conflicts or obstacles in a free-flight environment, as shown in Figure 5-7.

Accordingly, the figure presents the proposed adaption of Wicken’s (1984) model, with

the recommended variation shown in bold black. The presence of both feedback loops

suggest that the stages of the pilot information processing process are continuous.

In the present context, Wickens’ (1984) model could be modified slightly to

accommodate “bottom up” approaches or “goal driven behaviour”, which are controlled

by external cues (i.e., commands). Bottom-up approaches, however, allow an

individual to directly perceive the incoming information from the environment and

perform an action without the influence of higher cognitive processes (Gibson 1979).

As such, the additional connection link (red arrow) between the working memory stage

(i.e., short-term buffer with a limited capacity for new incoming data and information)

Figure 5-7 A revised Wicken’s (1984) model of pilot information processing

Stimulus

FCAS

Attention resources

Sensory store

Perception Decision

and response selection

Response execution

Commands

Temporal planning Task

Human

Automation

Spatial planning

Task Working Memory Long-term

Memory

of 303

and the perception stage is necessary because the model, in its original form, does not

have a link between the perception and the working memory stage, as shown in Figure

5-7. In regard to environmental constraints, bottom-up processing links those

constraints until the top-level function of the system at the abstraction hierarchy is

formed and complete.

In top-down approaches, however, the link (green arrow) between the long-term

memory stage and the working memory stage support the approach. With this approach,

knowledge or expectations are used to guide the processing of information under the

influence of higher cognitive processes. The overall result of the system’s current

situation is formulated, specifying but not detailing the physical form of the system at

the lowest level of the abstraction hierarchy. Typically, the approach is supported by

mental models, plans and rules. However, mental models may fail to make information

clearer on the fundamental mechanisms or to provide sufficient information to establish

the accuracy of the physical system. As previously stated in Section 2.5, mental models

are considered a transition from a temporary device to logical reasoning (Johnson-Laird,

1983). Johnson-Laird views mental models as a temporary working memory that

supports simple or hastily executed information, without detailing the essential features

of the system, for the purposes of immediate reasoning tasks.

In the EID framework, both top-down and bottom up approaches are involved in

perception that supports an interface design (Rasmussen and Vicente, 1992). As

discussed in Section 2.6, pilot perception and action should play an important role in the

design of interfaces. For example, the yoke system used in the current study for

navigational tasks maintains the communication of spatial and temporal aspects of

perception- action in close relation with each other at skill-based level as shown in

Figure 5-7.

of 303

The next section examines a number of implications of mental models theory for the

display of conflict avoidance information. Mental model theory is an effort to model the

human ability to understand the representation of the external reality of an unknown and

unpredictable phenomenon that causes an event to result one way rather than another

(Gentner and Stevens, 1983; Johnson-Laird, 1983).

5.6. Implications of Mental Models Theory

This section examines a number of the implications of mental models theory for the

display of conflict avoidance information. Mental model theory is an effort to model the

human ability to understand the representation of the external reality of an unknown and

unpredictable phenomenon (Gentner and Stevens, 1983; Johnson-Laird, 1983). In its

simplest form, the mental model represents the internal relationships of a system based

on experience and perceived actions. The user's mental model is the source of the user's

assumptions about the effects of actions, which can support navigation through the

“system” (i.e., its menu structure), navigation of the aircraft and/ or planning of what

actions (also how and when) will be taken to reach a desired goal, and contributes in the

interpretation of feedback (Van Der Veer, 1989). This suggests that an accurate and

comprehensive mental model would be consistent with the device or interface such as

the FCAS developed by conceptual model designers. What is particularly significant in

the use of geometric properties of the protective cone (i.e., vector, lines, and crosses) of

complex systems such as the FCAS is that they can communicate many different

functional relations. Pilots can suggest a desirable or necessary course of action in the

operation of the FCAS. Interpretations of diagrams (i.e., protective cone) with vectors

used a subset of heading change including rotates should support pilots’ mental models.

However, in the current study, complete and correct mental model of FCAS has not

been demonstrated consistently by the participants in the experimental group.

Therefore, having a mental model of a system may depend on the user’s understanding

concerning what features the system consists of, how these features are connected,

arranged or integrated, and how the features may behave and interact with each other in

the future, such as the use of relative velocity vectors in FCAS. But, according to

Norman (1983), a well-captured or powerful mental model displays a similar function

to that of a system or image, as presented to the users.

of 303

Other studies, such as Kieras and Bovair (1984), suggest that a mental model is relevant

only if it is applicable to performing the task. Therefore, in the present context of a task,

mental models can be both applicable and inapplicable to the task and has been

demonstrated by the participants in the control group.

The comments made by the participants in the control group suggest that they applied

trial and error techniques to avoid conflicts, which demonstrates the inapplicable use of

mental models to the task (i.e., functional models). This issue is supported by the results

of Kieras and Bovair’s in which they compared operating the device using a mental

model to having to learn how to operate the device by trial and error. In this situation,

the procedures were reasonably difficult to identify and recognise implying that a higher

mental process is involved (i.e., structural models). However, it is less difficult using

the mental model information (i.e., functional models).

The fundamental practical question is how to choose the right mental models for the

user to have in a free-flight environment where procedures or rules are yet to be defined

by the aviation community. Here, specific questions are posed to both flight training

(i.e., how to communicate complete and accurate mental models via flight manuals that

explicitly represent the intended conceptual model to the users) and the design of a

system (i.e., how to design a system easily understood by the users). Thus, two

implications of mental models theory for interface design are offered. First, the design

of user interactive interface display should be integrated into the implementation of

practice. For example, from a novice characteristics perspective, the focus should be on

the mental models participants bring to practice, not their individual or group

behaviours. Furthermore, a clear distinction between what the participants know and

what they need to know about the task to be performed is necessary. Secondly,

conceptual model designers should work from an exhaustively complete knowledge of

theory (i.e., cognitive principles) not just from guidelines, rules or procedures for

designing conceptual models. This acknowledges the importance of current knowledge

of the world, represented in mental models, for the acquisition of new knowledge and

skills. Studies, such as Gentner and Stevens (1983) have demonstrated that the learning

of a new system or procedure occurs as participants’ mental models acquire refinement

and accuracy of information.

of 303

5.7. Summary of Main Research Findings and Analysis

This chapter presented the main findings and analyses of the current research in relation

to collision avoidance. The study involved experimental group with the FCAS, and a

control comparison group with conventional display. Quantitative methodology

comprising questionnaires and simulation data were used to measure the dependant

variables. The results indicate that the experimental group has less mental demand than

the control group and, importantly, that there is a greater improvement in task

performance in the experimental group compared to the control group. However, the

overall mental workload suggests that there is no significant difference between both

the experimental and control groups, and did not support the original hypothesis (H-3).

The simulation data has contributed insight into some of the important self-separation

concerns. As hypothesised, with the FCAS, performing manoeuvres to avoid separation

conflicts generated larger minimum separation standards as compared with the control

group. The analysis of pilots’ manoeuvres preferences has also yielded two meaningful

constructs, a Port approach and Head-on approach on the potential use of EID. The

study revealed that higher scores on the Portboard approach and Head-On approach

seem to be the most significant difference between the experimental and control groups.

The study also identified pilots’ inconsistency in avoiding conflict with a Starboard

approach in both groups.

The recorded interviews were utilised to obtain qualitative data. The participants in this

study used a FCAS and conventional display to explore, in an interview, the factors that

affected their decision-making processes and the ways in which they coped with

collision avoidance difficulties. The experimental group comments indicated that the

successful completion of conflict resolutions is influenced by the factors such as line-of-

sight; relative velocity vectors; the rate of change of the protective cone, and the angle

of rotation. These have provided them with enhanced situation awareness while

performing collision avoidance tasks as hypothesised. The participants also reported

that they consciously evaluated the options suggested by the display based on their

relative importance and expected outcomes. They then used this information to choose

the option that had the highest safety margins.

of 303

Finally, the current study has proposed a revision of two models by Wicken and Parker.

The next chapter provides the conclusions, and makes recommendations based on the

thesis findings.

of 303

Chapter 6. Conclusion and Contribution

The application of the Work Domain Analysis (WDA) and attentional theories allow the

development of the FCAS. FCAS has contributed important insights into the main

research question, as stated in Chapter 1. The current study began with a holistic

approach in relation to collision avoidance resolution in a free-flight environment. The

free-flight concept ensured that pilots had the freedom and responsibility to maintain

separation, as compared to the current Air Traffic Management, is predicted to lead to

the introduction of new devices, procedures and tasks. These new issues such as new

cognitive workload call for the development of a new collision avoidance system. This

chapter summarises the main research findings in relation to the research question and

draws important conclusions. In addition, this chapter will highlight the theoretical,

methodological and practical contributions of the current study to the aviation industry.

Finally, limitations of the current study will be discussed and suggestions will be made

for future recommendations are also presented in light of the current research.

of 303


of 303

The current study provides a number of potential positive contributions to the current

airspace traffic management (ATM) related issues. At present, the ATM is suffering

from a shortage of airspace in relation to the allocation of flight levels. The current

ATM can only assign one flight level to one aircraft at a time, that is, no other aircraft

can fly within the buffer zone or depend on the navigational precision of the aircraft. As

air traffic continues to increase, air traffic controllers and pilots will be faced with a

change of mental workload however, existing collision avoidance systems are not

sufficient to cope with this significant increase. With the free-flight environment, the

current buffer zone, in which no other aircraft can fly, will be “eliminated” or

maximised as it is not necessary and does not depend on the aircraft’s navigational

capabilities. The buffer zone does, however, depend on the relative motion. Thus, the

goal of free-flight is to delegate the responsibility for maintaining the separation of

aircraft to pilots. In the free-flight environment, a pilot’s role is to change to the

airspace and decision manager. The inevitable increase in air traffic will impact

considerably on the pilot’s responsibility, as they are faced with a new set of mental

demands and varying tasks. Consequently, new ways of maintaining separations and/or

situation awareness are required.

In light of these challenges, this thesis sought to answer the following research question:

Which factors associated with the environment, aircraft constraints and

perception of collision significantly influence decision making by pilots when

manoeuvring their aircraft, such that the protected zone is not violated?

To answer this research question, Cognitive Work Analysis (CWA) and attention theory

allowed the examination of the various dimensions of a collision avoidance system.

With human-related approaches, a few researchers have examined the use of an

Ecological Interface Design approach, which is related to CWA to support pilot’s flight

collision avoidance manoeuvres.

of 303

The CWA is composed of a work domain analysis (WDA) and Cognitive Task Analysis

(CTA). The WDA states the importance for improved visualisation of spatial and

system awareness, and its framework supports the inclusion of both relative motion and

protective cone functions. The WDA was used to map flight constraints (i.e., separation

and flight-path deviation) in resolving air traffic conflicts on the FCAS display. Each

level of WDA is related through a structural mean-ends link to support pilots with a

visual representation of constraints-based conflict and possible conflict resolutions.

Importantly, pilots are aware of their current position and action possibilities. The CTA

identifies skills, rules, and knowledge-based behaviour of decision making processes in

terms of how pilots’ goals, flying activities, and tasks relate to the physical constraints

of the environment. This underpins the concept of the Ecologically Interface Design

(EID).

A visual representation of these constraints is presented in the form of conflict geometry

of vectors in order to support pilot’s flight collision avoidance manoeuvres by changing

the aircraft’s heading and speed. Perceptual grouping in visual search for features is

assumed to be effortless, quick and pre-attentive at a glance (Treisman, 1982). FCAS

features, such as the protecting cone and relation motion function, are incorporated to

predict the time and distance to reach the first loss of separation, which indicates the

need for collision avoidance in a free-flight environment. Thus, the FCAS display

properties (i.e., line-of-sight, relative velocity vectors, the rate of change of the

protective cone, and the angle of rotation) are used to discriminate between display

features in the context of conflict resolution geometries. This suggests that FCAS

features are grouped to provide more meaningful collections of information. Grouping

helps pilots to focus their attention on a single object within the environment, especially

so that they could preferentially select from a complex display of constraints

information, with a view to limiting or clarifying receptivity by narrowing the range of

stimuli. In addition, the interaction with the FCAS is predicted to be effortless and

quick, and to allow accurate mapping of pilots’ mental models of the aircraft’s

performance. Therefore, the following hypotheses were established to fulfil the

objectives of the study. Pilots using the FCAS, compared with the conventional display,

will:

of 303

1. Have an improved situation awareness compared with the situation awareness


2. Violate separation constraints less frequently than the horizontal minimum


3. Have a lower mental workload compared with the mental workload experienced


In an attempt to investigate collision avoidance related problems in a free-flight

environment, the experimental study examined loss of separation between other aircraft,

which may result in collision. Participants (N=21) were recruited from the aviation

industry in Melbourne. All participants (100%) were male between the age of 18 and 80

years. Thirteen participants were allocated to the experimental group and eight (8)

participants were allocated to the control group.

Both qualitative and quantitative distinctions were found between the experimental and

control groups. These distinctions highlighted areas in which similarities and

differences between pilots occurred in terms of performance, mental workload, situation

awareness and mental models. The parameters considered for analyses were headings

and loss of separations for lateral separation.

6.1. Conclusion

The objective of the present study was to develop and evaluate the FCAS. The

evaluation of this system was intended to lead to an improved understanding of the

interaction between pilots and collision avoidance systems in a free-flight environment

and to identify ways in which this knowledge could contribute to pilot safety. This

study have not fully addressed issues such as pilots’ cognition activities however, the

main research question was how pilots’ interact with the FCAS.

of 303

To experimentally assess pilot situation awareness, a questionnaire ratings scale of

situation awareness measurement was used. The technique was extended to include

situation awareness queries covering three levels of situation awareness (Endsley,

1995a). The results revealed significant differences between the experimental and

control groups, in agreement with H-1. The analysed yielded one meaningful construct

in relation situation awareness: “How useful was the system for understanding the

direction of travel of the Intruder?” This construct differentiates between the two groups

in regards to the understanding of what has been perceived (Level 2). A significant

statistical difference revealed that the use of a visualisation technique of protective cone

and relation motion functions improved the experimental group’s situation awareness

compared with the control groups. Furthermore, the analyses of pilot comments in both

the experimental and control groups revealed that pilots focused on relevant information

from the flight collision avoidance display to avoid conflicts. For example, in the

experimental group, this information is related predominantly to situation awareness.

Pilots in this group focused on the information provided by the FCAS that was

appropriate to support them in identifying potential conflicts, and provide resolutions

(divert or go around) to avoid obstacles. However, the post experimental verbal reports

by most of the pilots in the experimental group may be worthy of note at this point. The

narrative comments made by the pilots reflected strong feelings about the ability to

understand an awareness of other aircraft and to maintain self-separation during the

practice and final exercises of the task. That is, some of the participants suggested that

the protective cone function supports the maintenance of minimum separation standard,

but the interface is highly cluttered and may have induced too much relative

information (i.e., vectors). However, the interface did not interfere with maintaining

situation awareness to explain the reason behind significant differences between the two

groups. Thus, the application and evaluation of FCAS has improved pilot situation

awareness (i.e., appeared to be more accurate) by supporting pilots’ perceived relative

and protective cone functions.

of 303

The EID approach allowed the development of affordances in which separation and

flight constraints were displayed in the form of geometric representations to pilots. This

information assists pilots to operate within those constraints and to make better

decisions without exceeding the metrics required for the heading change. The flight

simulations provided a means for investigating and comparing performance of

participants in both groups. In the experimental group, performance was better than the

control group. More specifically, the results of the horizontal separation manoeuvres

found that the pilots in the experimental group deviated less than the control group in

agreement with H-2. A possible explanation might be that flight constraints were not

visible to pilots using a conventional approach. However, participants’ preference in the

current study was heading changes, as supported by Cashion et al, (1997) who showed

that pilots preferred heading to speed.

A study of horizontal separation conducted by Palmer (1983) required pilots to avoid

conflict situations using cockpit traffic displays. The pilots were instructed to perform

conflict resolution manoeuvres to maintain a horizontal separation at a specified value

(i.e., minimum miss distance) from their assigned flight path. Palmer (1983) reported

that pilots exceeded the pre-determined metrics required to avoid collision. In the

current study, pilots commented that the information provided by the FCAS for

collision avoidance was intuitive. This type of intuitive situation was not observed when

pilots used TCAS to support conflicts for a vertical separation. Other resolution

advisories for vertical separations, such as “Adjust Vertical Speed Adjust”, have also

been reported to be unintuitive (Rome et al., 2000). Other important distinctions

observed between the TCAS and FCAS to maintaining minimum separation standards

include the observation that TCAS alerts are more likely to support pilots bursting their

flight level.

of 303

The results of the pilots’ overall mental workload (using the NASA-TLX) with both

displays revealed that there was no significant difference between the experimental and

control groups. Unexpectedly, mental workload measures appeared to be not

significant, which was not hypothesised (H-3). Based on the data collected, the

hypothesis could be revised to state that if the collision avoidance task is increased,

pilots using FCAS will have a lower mental workload compared with mental workload

experienced by pilots using standard instruments.

However, both groups also reported information overload related to display clutter.

Taken together, the results yielded two meaningful constructs. Within the experimental

group, these constructs are a reduction in mental demand and improved performance

level.

In regards to safe separation in a free-flight environment, no previous studies have

examined a pilot’s manoeuvre choice to fly either behind or in front of the Intruder from

the Starboard, Head-on or Port approach, or to investigate the impact on pilots’ decision

making in terms of relative safety of manoeuvres. The collision avoidance task used in

the present study included avoiding the Intruder from three different directions (i.e.,

Starboard, Head-on and Port approaches). The results were consistent on both the Port

and Head-on approaches, suggesting that the protective cone and relative functions have

a significant effect on these approaches. The Starboard approach was a “mirror image”

of the Port approach with the same traffic geometry information presented in a clear,

unambiguous fashion. However, unexpectedly, the Starboard approach results were not

significant. A possible explanation for these unexpected behaviours is that pilots tend

not to comply with the “right of way” rules as the Intruders move toward the Ownship

at a conversion angle (i.e., off the right wing of the Ownship). In view of this, pilots

demonstrated unexpected behaviours in choosing a flight path. These inconsistencies in

their ability to employ strategies in choosing a flight path control to avoid collision may

have contributed to their decision making. Indeed, there is the possibility that the pilots’

strategies were consistent with their cognitive representations of problem-solving skills

that could lead to their best performance. Conversely, the inconsistencies of problem-

solving skills that reflect pilots’ choice and expectations might improve performance.

Theoretically, ecological interfaces are supposed to act as externalised pilots’ mental

models.

of 303

Thus, pilots should be able to effortlessly acquire advanced mental models that require

many hours of flying experience to develop (Vincent, 1999b). This effect may be

explained by the need for more time to control the functioning of the protective cone

and relative motion effectively, as suggested in the pilots’ comments. In addition, pilots

may have used the protective cone and relative motion functions without actually

updating their mental models, as suggested by Borst, Mulderand and Van Paassen

(2010). Clearly, the accurate externalisation of the pilots’ mental model of the FCAS

was not fully supported in the experimental results.

Decision-making is a complex and multi-dimensional phenomenon and forms an

integral part of resolving conflict in a free-flight environment. The FCAS has supported

cognitive control (i.e., skills, rules and knowledge-based behaviour), as indicated in the

pilots’ comments. The comments clearly show the importance of mapping constraint-

based information to FCAS and suggest that the most useful features of the display to

support pilot behaviour were the protective cone, angle of resolution, line-of-sight and

relative velocity vector. Crucially, presenting the relative motion (i.e., constrained-

based information) and protective cone functions result in better decision making. Both

of these functions help pilots with decision making and provide an availability of

options. Hence, rotating the relative velocity vector out of the protective cone to avoid a

conflict automatically requires pilots to closely monitor the angle of resolution with

rerouting instructions with minimum flight path deviation. This gives them a clear and

direct cue as to whether they can fly in front of or behind the Intruder, and to estimate

the relative distance to the Intruder in a timely manner.

In this exploration of the participants’ development of the decision making process, a

number of factors emerged both within, and across the dimensions used to frame this

study, which highlight how participants perceived flight constraints. These include:

analysing line-of-sight, relative velocity vectors, closely monitoring the rate of change

of the protective cone, and evaluating more closely the angle of rotation. It is also

important to note that, with similar displays, such as that of Van Dam, Mulder, and van

Paassen (2008), the primary difference is that the rate of expansion of the protective

cone function is visible, as commented by pilots. Certainly, differences in the specific

criteria used to display the protective cone and relative motion functions on the

of 303

interface could have contributed to differences between the present study and that of

Van Dam, Mulder, and van Paassen (2008) as stated in Chapter 2. The current study

makes an improvement in the field by providing strong support for such functions.

In turn, protective cone and relative motion functions were perceived to have led to

greater understanding, insight and awareness about the principles and procedures of

using the FCAS, and enhanced knowledge of how to accommodate different conflict

approaches and diversity within the pilots’ own flight experiences. The participants in

the control group had more flying hours than those in the experimental group. One

distinction between the experimental and control group was that pilots in the

experimental group used relative motion and protective cone functions to support their

decision making. The control group, on the other hand, may have used a different set of

criteria in formulating their respective collision avoidance judgments. It was evident

that the experimental group appeared to use both a rational and intuitive approach,

regardless of their flying experience, as compared to the control group. The findings of

the current study suggest that the control group used an analytical approach to avoid

conflict. This result is consistent with Klein (2008) who argued that experienced

decision makers use prior experience to quickly categorise situations using their schema

to suggest appropriate courses of action.

A number of conclusions can be drawn from the present results. The exact causes of

better decision making are not known in the literature for a free-flight environment, but

several factors and conditions may play a role in its development. First, the current

findings suggest that key, relevant information provided by FCAS leads to one

conclusion, rather than a mental simulation of numerical values (i.e., the system always

provided one interpretation) - conclusiveness. Second, it appears that information

related to conflict and resolutions may be acquired in the three levels of situation

awareness. Specifically, participants perceived flight constraints on the FCAS to acquire

situation awareness including analysing loss-of-separation, relative velocity vectors,

closely monitoring the rate of change of the protective cone and zone, angle of rotation,

and evaluating the angle of rotation more closely. It was evident from the participants’

comments that these flight constraints were more salient than other features of the

FCAS - completeness. Third, participants’ comments revealed that a pilot might be

of 303

willing to take a risk with safety (i.e., avoid collision) rather than manoeuvring costs

(i.e., manoeuvre with respect to longer distance, before they can turn back to resume

their flight-plan route) - incompatible goals. Fourth is the perception of collision.

With the support of the FCAS, individuals rely on intuitive judgments to evaluate

environmental risks such as Intruders.

The following subsections will review the contributions of this thesis (i.e., theory,

methodology, and practice), highlight the limitations of the current study, and, finally,

outline suggestions for future research.

6.1.1 Theoretical Contributions by the Study

This section draws together discussion from different aspects of pilot cognitive

processes to include decision making and situation awareness in mental models. A new

task in the free-flight environment may be that pilots are required to choose a flight path

and maintain self-separation with minimum intervention from ATCO. However, in a

free-flight environment, pilots have multiple goals as the flight progresses. The goal

activates a suitable mental model to extract specific information from the environment

to form situation awareness of the current situation. The information they process may

be simple or complex, clear or distorted, complete or incomplete, with gaps needed to

be filled. These issues may have an effect on pilots’ decision making as they monitor

and control the aircraft, and interact with other automated display systems. Limitations

of working memory (e.g., processing resources) on processing this information may

affect the fidelity of pilots’ mental models. Mental models may be vulnerable to both

problem complexity and conflicting information, as presented by these systems. This

combined with poor understanding of the situation or loss of situation awareness may

lead to an inappropriate decision, even if the information needed to support the proper

choice of actions is available in the environment (Endsley, 2006).

The current study provides clear evidence that the FCAS could improve pilots’ situation

awareness. Specifically, the FCAS capabilities depend on displaying relative directions

and relative positions of the traffic and obstacles. Pilots are aware of many different

forms of information within the vicinity. More importantly, based on object-based

models of attention, capture occurs because the visual system considered relative

of 303

motion and protective cone functions as one perceptual group. Hence, the design of

FCAS also contributes to pilot’s decision-making in free-flight environments. This

could be an improvement of the possible current system flaws in terms of pilots’ self-

separation in free-flight environments.

Contributions to research on situation awareness

The current study builds on a model proposed by Parker (1973) and proposes that stage

movement should be replaced with relative movement to accommodate constraints such

as velocity, acceleration, displacement, time and trajectory. These constraints should be

visualised to improve situation awareness. The results of the current study demonstrate

that situation awareness is an important component for decision making in self-

separation. It was revealed that acquiring situation awareness in collision avoidance is a

process in which participants gain an initial level of situation awareness in the

familiarisation state of flight constraints, and enhanced this awareness during the flight

manoeuvres.

The study’s central contribution is the demonstration that the flight collision avoidance

system’s (FCAS) enhanced awareness is likely to prepare pilots more methodically, in

terms of assessing conflict situations, understanding consequences, and identifying

conflict resolution manoeuvres well before a collision. The performance and situation

awareness results suggest that pilots were able to avoid a conflict without violating the

minimum standard, as supported by both relative velocity vectors and the protective

cone functions, which were the most useful pieces of interface information.

Contributions to research on decision making

The theoretical framework for automation design, as presented by Wicken’s (1984),

must be revisited in order to further understand the decision making dynamics of pilot

self-separation in a free-flight environment. The framework suggests a number of stages

including sensory storage, perception, attentional resources, decision selection, and

implementation. However, the current study highlights that such a framework is

necessary, but not sufficient for effective display design. An important finding of the

current study is the participant comments on the considerations that are used to inform

of 303

their decisions. For example, participants listed the use of criteria (e.g., clockwise and

anti-clockwise rotation resolution) and value weighting (i.e., a shorter and longer

distance from the intended flight path) which support a less subjective and more

objective evaluation of the alternatives before making a decision with limited

attentional resources. As a result, this current study contributes to the research literature

by depicting the importance of including a stage (temporal planning task) in the

framework for an automation design model that completely and clearly expresses and

represents planning as a process before decision considerations.

6.1.2 Methodological Contributions by the Study

The main methodological contribution of this thesis has been the combination and

application of theoretical concepts of attention, and Work Domain Analysis (WDA).

Previously, these two concepts had only been studied in separation. As a stand-alone

framework, the WDA, does not make it easier to determine how information should be

located or placed, particularly in relation to a reference. Therefore, the applicability of

WDA alone does not necessarily offer the pathways to design a system effectively. As

previously highlighted, despite its comprehensive use in several domains, attention

theory related WDA applications had not yet been documented. Therefore, to support

pilots and assist in designing a system, the current study combined attention theory with

WDA as a framework. The inclusion of attention theory in the framework bridges the

gap by informing WDA to support the development of FCAS.

6.1.3 Practical Contributions by the Study

The current research contributes an important new FCAS system as well as educational

benefits and new applications. In a free-flight environment, pilots may need to acquire

new skills when interacting with new automation systems for collision avoidance. The

main technical contribution demonstrates that pilots can perceive the relative velocity of

the intruder aircraft to supplement the use of the “see and avoid” techniques to avoid

conflicts. It also provides an insight into the use of “right of way” procedures to

facilitate a better understanding of the impact of the new automation system in relation

to daily flight practices in a free-flight environment. For example, the increased reliance

on visualisation of relative motion vectors to support pilots’ decision making processes.

of 303

This can also improve educational efforts (e.g., pilots’ training in a free-flight

environment) within scientific and technological disciplines and create new

opportunities for decentralised collision avoidance information, while still promoting

collaboration between pilots and air traffic controllers.

6.2. Limitations of the Current Study

The current study has made every effort to accomplish as much as is scientifically

possible, given the limited size of the participant sample. The results should therefore

be viewed as suggestive rather than conclusive and it must be noted that there is no

previous research, or limited research, with which to validate the current study’s

findings. Future research could address this issue by replicating the current study with

more than twenty-one participants with a wider range of flight experience. A number of

other limitations have been identified which must be taken into consideration when

interpreting the results of the study. These limitations relate primarily to the ability to

comprehensively simulate real-world situations. First the simulations were conducted in

a controlled environment using a static simulator and, as such, the current study did not

model aircraft flight dynamics for the simulations. Therefore, the experiment did not

involve critical events that could happen in a real-flight scenario. For example, the

proposed display did not present more than one possible flight path of the Intruder to

Ownship. In addition, the scenarios presented in the current study were limited to a pair

wise conflict. Further investigation could address this issue by replicating the current

study with multiple conflict scenarios. Finally, the non-EID display used for the control

group may not have comprehensively reflected the proper working principles of the

TCAS system. Other limitations have been identified in the procedures used to collect

data. For instance, participants were not trained to verbalise certain aspects of the

simulations while performing tasks and this has resulted in incomplete data collection.

There were also no performance metrics based on objective criteria (e.g. time within the

cone, closeness of approach, deviation from flight path).

of 303

6.3. Suggestions for Future Research

The FCAS introduced in this dissertation provides a number of suggestions for future

research. First, research is needed to examine why pilots have difficulty in avoiding

conflict when an aircraft is approaching with a starboard approach. Future research

should therefore focus on examining the relationship between the mental models and

pilot choices (e.g., “right of way”). This has implications for how pilot training must

adapt to take into account individual differences in order to raise awareness of how

these differences may contribute to unsafe or safe flying behaviour in a free-flight

environment. That is, the mental models formed by novice versus expert pilots and how

these differ in terms of quality deserve considerable further research. Indeed, a few

areas still exist that pose a challenge, for example, strategic thinking, that may occur in

a pilot’s decision making in a free-flight environment. As mentioned, understanding of

the process by which pilots’ flying experiences influence their behaviour is of particular

interest. The findings may then promote pilot flight training and simulation

development programs that consider all the skills necessary for flight safety in a free-

flight environment.

To increase airspace capacity, it would be useful to examine pilots’ behaviour by

designing a study in which pilots can fly below the minimum separation standards in

emergency situations that are currently applied with EID to accurately develop their

mental model of the aircraft performance. Such an EID approach would enable the

development of useful ground-based “see-and-avoid” systems for unmanned aircraft as

well as other future decision support tools managed by air traffic management. Future

research must also explore the possibility of designing a conflict avoidance display

showing two or more possible directions of the Intruder in a conflict scenario that will

assist pilots in making plans and decisions well in advance.

of 303

6.4. Closing Remarks

The goal of the free-flight concepts is to ensure that pilots have the freedom and

responsibility to maintain self-separations. The free-flight concept is predicted to

introduce new systems, rules, procedures, and tasks compared to the current Air Traffic

Management (ATM). Indeed, new technologies and rules are being considered to

replace current air traffic management. This will give pilots the ability to change flight

trajectory at any given time. One way to proceed is to define the current airspace as a

“free flight” environment to be operated by a pilot as a “preferred flight path”, without

compromising the established minimum separation standards.

It is also worth emphasising that the use of radar places enormous responsibilities on

controllers and, moreover, the pilots and controllers’ cognitive limitations often led to

errors or accidents. Why do these problems occur when pilots are equipped with a

display to support them in cockpit activities? One of the commonly proposed solutions

has been to increase the use of automation systems in the cockpit. Yet the design of

current support systems lack the necessary information, systemically arranged to

support factors such as mental workload, situation awareness and decision making that

may have contributed to this problem.

A system that brings about distributed, yet shared and cooperative decision making

would be more efficient and robust and thus more result oriented than a centralised

control management system. Once the future air traffic management system is fully

implemented, both pilots and controllers would be simultaneously presented with the

same information regarding aircraft positions, attitudes, and directions. This will

provide cooperative conflict resolution in the free-flight environment. With the aid of

FCAS, pilots would be able to choose and make flight path decisions independently.

of 303

The FCAS proposed here is a simple interface for displaying constraints such as relative

velocity and relative motion functions in an attempt to interactively avoid collision.

Instant feedback is predicted which allows a considerable amount of time for pilots to

build situation awareness, reduce cognitive workload and keep themselves in the control

loop. FCAS has been designed and developed based on an EID approach. This new

approach is a significant departure from conventional approaches regarding how pilots

have previously thought about airborne collision avoidance display. The concept of

protective cone and relative motion functions used in the FCAS is likely to become

more important as the collision avoidance display system increases its complexity. With

these functions, pilots were aware of their capabilities, the aircraft constraints, and were

actively involved in the decision making loop to prevent mid-air collision. This study is

the first attempt at using important earlier research concerning self-separation in a free-

flight environment. Studies have not previously reported the use of the protective cone

and relative-velocity vectors in a collision avoidance display. This thesis has shown that

the application and evaluation of FCAS has improved pilot situation awareness,

performance and better decision-making processes in a free-flight environment.

of 303

Reference

Abeloos, A.L.M. Van Paassen, M.M. & Mulder, M., (2003), An abstraction hierarchy

and functional model of airspace for airborne trajectory planning support.

Conference on human decision making and manual control, Linköping University,

June 2-4, 2003. ISBN 91-7373-686-4.

Aberbach, J. D., & Rockman, B. A. (2002). Conducting and coding elite interviews. PS:

Political Science and Politics, 673-676.

Abraham, C., & Sheeran, P. (2003). Acting on intentions: The role of anticipated regret.

British Journal of Social Psychology, 42(4), 495-511.

Adams, M. J., Tenney, Y. J., & Pew, R. W. (1995). Situation awareness and the

cognitive management of complex systems. Human Factors: The Journal of the

Human Factors and Ergonomics Society, 37(1), 85-104

Adamczyk, P. D., & Bailey, B. P. (2004). If not now, when?: the effects of interruption

at different moments within task execution. In Proceedings of the SIGCHI

conference on Human factors in computing systems (pp. 271-278). ACM.

Ahlstrom, U. (2005). Work domain analysis for air traffic controller weather displays.

Journal of Safety Research, 36(2), 159.

Alexander, A. L., Stelzer, E. M., Kim, S. H., & Kaber, D. B. (2008, September).

Bottom-up and top-down contributors to pilot perceptions of display clutter in

advanced flight deck technologies. In Proceedings of the Human Factors and

Ergonomics Society Annual Meeting (Vol. 52, No. 18, pp. 1180-1184). Sage

Publications.

Alexander, A. L., & Wickens, C. D. (2002). Does Workload Modulate the Difference

between Cockpit Traffic Display Formats?. In Proceedings of the Human Factors

and Ergonomics Society Annual Meeting (Vol. 46, No. 1, pp. 1-5). SAGE

Publications.

Alexander, A. L., & Wickens, C. D. (2003). The effects of spatial awareness biases on

maneuver choice in a cockpit display of traffic information. In 12th International

Symposium on Aviation Psychology, 2003.

of 303

Alexander, A. L., Wickens, C. D., & Merwin, D. H. (2005). Perspective and coplanar

cockpit displays of traffic information: Implications for maneuver choice, flight

safety, and mental workload. The International Journal of Aviation Psychology,

15(1), 1-21.

Allred, K. G., Mallozzi, J. S., Matsui, F., & Raia, C. P. (1997). The influence of anger

and compassion on negotiation performance. Organizational Behavior and

Human Decision Processes, 70(3), 175-187.

Amant, R. S. (1999). User interface affordances in a planning representation. Human-

Computer Interaction, 14(3), 317-354.

Amelink, M. H., van Passen, M. M., Mulder, M., & Flach, J. M. (2003, April). Total

energy-based perspective flight path display for aircraft guidance along complex

approach trajectories. In Proceedings of the 12th International Symposium on

Aviation Psychology (pp. 48-53).

Andre, A. D. (1995). Information requirements for low-visibility taxi operations: What

pilots say. In Proceedings of the Eighth International Symposium on Aviation

Psychology (pp. 484-488).

Andre, A. D., Wickens, C. D., & Moorman, L. (1991). Display formatting techniques

for improving situation awareness in the aircraft cockpit. The International

Journal of Aviation Psychology, 1(3), 205-218.

Andrews, J. W., (1978) "A Relative Motion Analysis of Horizontal Collision

Avoidance", SAFE Journal, 8(2).

Andrews, J. W. & Hollister, W. M. (1997). “Impact of Intent-Based Manoeuvre

Constrains on Alert Rates” Air Traffic Control Quarterly 5(3) 157-179.

Averty, P., Collet, C., Dittmar, A., Athènes, S., & Vernet-Maury, E. (2004). Mental

workload in air traffic control: an index constructed from field tests. Aviation,

space, and environmental medicine, 75(4), 333-341.

Aviation Safety Network (ASN,2010). “ASN Aviation Safety Database” http://aviation-

safety.net/index.php October, 14th 2010.

Bach, R., Farrell, C., and Erzberger, H., (2009)"An Algorithm for Level-Aircraft

Conflict Resolution," NASA/CR-2009-214573.

Baddeley, A. D. (2004). The psychology of memory. The essential handbook of

of 303

memory disorders for clinicians, 1-13.

Bainbridge, L. (1983). "Ironies of Automation." Automatica 19(6),775-779.

Ballin, M. G., Sharma, V., Vivona, R. A., Johnson, E. J., & Ramiscal, E. (2002). A

flight deck decision support tool for autonomous airborne operations. In

Proceedings of the AIAA Guidance, Navigation and Control Conference,

Monterey, CA, USA, AIAA.

Bauer, M. I., & Johnson-Laird, P. N. (1993). How diagrams can improve reasoning.

Psychological Science, 4(6), 372-378.

Bedny, G. Z., Karwowski, W., & Jeng, O. J. (2004). The situational reflection of reality

in activity theory and the concept of situation awareness in cognitive psychology.

Theoretical Issues in Ergonomics Science, 5(4), 275-296.

Bedny, G., & Meister, D. (1999). Theory of activity and situation awareness.

International Journal of cognitive ergonomics, 3(1), 63-72.

Beevis, D. (Ed.). (1992). Analysis techniques for man-machine systems design

(Technical Report AC/243(Panel S)TR/7). Vols. 1 & 2. Brussels: NATO Defence

Research Group.

Begault, D. R. (1993). Head-up auditory displays for traffic collision avoidance system

advisories: A preliminary investigation. Human Factors: The Journal of the

Human Factors and Ergonomics Society, 35(4), 707-717.

Bennett, K. B., Posey, S. M., & Shattuck, L. G. (2008). Ecological interface design for

military command and control. Journal of Cognitive Engineering and Decision

Making, 2(4), 349-385.

Berardi, A., Parasuraman, R. & Haxby, J. (2001) Overall vigilance and Sustained

Attention Decrements in Healthy Aging. Experimental Aging Research 27, 19-39

Berti, S., & Schröger, E. (2003). Working memory controls involuntary attention

switching: evidence from an auditory distraction paradigm. European Journal of

Neuroscience, 17(5), 1119-1122.

Billings, C. E. (1991). "Toward a Human-Centred Aircraft Automation Philosophy."

The International Journal of Aviation Psychology 1(4), 261-270.

of 303

Billings, C E. (1997). Issues Concerning Human-Centred Intelligent Systems: What’s

"human-centered" and what’s the problem? Plenary talk at NSF Workshop on

Human-Centred Systems: Information, Interactivity, And Intelligence (HCS),

Crystal Gateway Marriott Hotel, Arlington, VA (paper of talk).

Bilimoria, K.D. (2000) “A Geometric Optimization Approach to Aircraft Conflict

Resolution” AIAA 2000-4265, AIAA Guidance, Navigation, and Control

Conference, Denver, CO.

Bone, J., Domino, D., Hahn, E., & Narkus-Kramer, M. (2002). Cockpit display of

traffic information (CDTI) enhanced flight rules (CEFR). In Digital Avionics

Systems Conference, 2002. Proceedings. The 21st (Vol. 2, pp. 11B1-1). IEEE.

Bootsma, R. J., & Oudejans, R. R. (1993). Visual information about time-to-collision

between two objects. Journal of Experimental Psychology: Human Perception

and Performance, 19(5), 1041.

Borst, C., Mulder, M., & Van Paassen, M. M. (2010). Design and simulator evaluation

of an ecological synthetic vision display. Journal of guidance, control, and

dynamics, 33(5), 1577-1591.

Borst, C., Mulder, M., van Paassen, M. M., & Mulder, J. A. (2009). Experimental

Evaluation of an Ecological Synthetic Vision Display.

Borst, C., Suijkerbuijk, H. C. H., Mulder, M., & Van Paassen, M. M. (2006). Ecological

interface design for terrain awareness. The International Journal of Aviation

Psychology, 16(4), 375-400.

Brewster, S.A. & Crease, M.G. (1999) Correcting Menu Usability Problems With

Sound. Behaviour and Information Technology 18(3), 165-177.

Broadbent, D. E. (1982) Task combination and selective intake of information. Acta Psychol (Amst) 1982 Jul; 50(3) :253-90.

Brown, M. L., Newsome, S. L., & Glinert, E. P. (1989, March). An experiment into the

use of auditory cues to reduce visual workload. In ACM SIGCHI Bulletin (Vol. 20,

No. SI, pp. 339-346). ACM.

Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal

of mixed methods research, 1(1), 8-22.

of 303

Bryman, A. & Cramer, D. (1997). Concepts and their measurement. In Quantitative

data analysis with SPSS for Windows: A guide for social scientists (pp. 53-68).

Routledge.

Buttigieg, M. A., & Sanderson, P. M. (1991). Emergent features in visual display design

for two types of failure detection tasks. Human Factors: The Journal of the


Burns, C. M., Bryant, D. J., & Chalmers, B. A. (2005). Boundary, purpose, and values

in work-domain models: Models of naval command and control. Systems, Man

and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 35(5),

603-616.

Burns, C. M. & Hajdukiewicz, J. R. (2004). “Ecological Interface Design”. Boca Raton,

FL: CRC Press. ISBN 0-415-28374-4.

Burns, C., Jamieson, G., Skraaning, G., Lau, N., & Kwok, J. (2007, October).

Supporting situation awareness through ecological interface design. In

Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol.

51, No. 4, pp. 205-209). SAGE Publications.

Burns, C.M., Momtahan, K., & Enomoto, Y. (2006). Supporting the Strategies of

Cardiac Nurse Coordinators Using Cognitive Work Analysis. Proceedings of the

50th Annual Meeting of the Human Factors and Ergonomics Society. San

Francisco, CA. October 16-20, 442-446.

Cashion, P., Mackintosh, M. A., McGann, A., & Lozito, S. (1997, October). A study of

commercial flight crew self-separation. In Digital Avionics Systems Conference,

1997. 16th DASC., AIAA/IEEE (Vol. 2, pp. 6-3). IEEE.

Carroll, J. M., & Anderson, N. S. (1987). Mental models in human-computer

interaction: Research issues about what the user of software knows (No. 12). J. R.

Olson (Ed.). National Academies.

Cavallo, V., & Laurent, M. (1988). Visual information and skill level in time-to-

collision estimation. Perception, 17(5), 623-632.

Chakravarthy, A., & Ghose, D. (1998). Obstacle avoidance in a dynamic environment:

A collision cone approach. Systems, Man and Cybernetics, Part A: Systems and

Humans, IEEE Transactions on, 28(5), 562-574.

of 303

Chalmers, B. A., Webb, R. D., & Keeble, R. (2001). Cognitive Work Analysis

Modeling for Tactical Decision Support. A Paper Presented at the 7th ICCRTS

Symposium.

Chen, Z. (2012). Object-based attention: a tutorial review. Attention, Perception, &

Psychophysics, 74(5), 784-802.

Chen, Z.; & Cave, K. R. (2006). "When does visual attention select all features of a

distractor?". Journal of Experimental Psychology: Human Perception and

Performance 32: 1452–1464.

Choi, J. H., & Lee, H. J. (2011). Facets of simplicity for the smartphone interface: A

structural model. International Journal of Human-Computer Studies.

Christodoulou, M. A., & Kodaxakis, S. G. (2006). Automatic commercial aircraft-

collision avoidance in free flight: the three-dimensional problem. Intelligent

Transportation Systems, IEEE Transactions on, 7(2), 242-249.

Clason, D. L., & Dormody, T. J. (1994). Analyzing data measured by individual Likert-

type items. Journal of Agricultural Education, 35, 4.

Cleveland, W. P., Fleming, E. S., & Lee, G. (2011, April). TCAS traffic display

redesign. In Systems and Information Engineering Design Symposium (SIEDS),

2011 IEEE (pp. 209-214). IEEE.

Coe R.. (2002). It’s the effect size, stupid: what effect size is and why it is important.

British Educational Research Association Annual Conference 2002; Sept 12–14;

Exeter, England.

Cohen, J. (1992). A power primer. Psychological bulletin, 112(1), 155.

Colvin, K., Dodhia, R., & Dismukes, R. K. (2005). Is pilots' visual scanning adequate to

avoid mid-air collisions. In Proceedings of the 13th International Symposium on

Aviation Psychology (pp. 104-109).Commission of inquiry. (1990). Air France

Airbus A-320 F-GFKC, Mulhouse-Habsheim airport, June 26, 1988(edited

excerpts reprinted in Aviation Week and Space Technology, June 4, 1990). French

Ministry of Planning, Housing, Transport and Maritime Affairs.

Cooper, G. E., & Harper Jr, R. P., (1969).The use of pilot rating in the evaluation of

aircraft handling quality (No. AGARD-567). Advisory Group for Aerospace

Research and Development NUEILLY- Sur-Seine (France).

http://www.psyc.canterbury.ac.nz/people/cv%27s_papers_&_other_info/Chen_Cave_JEPHPP%20paper.pdf

http://www.psyc.canterbury.ac.nz/people/cv%27s_papers_&_other_info/Chen_Cave_JEPHPP%20paper.pdf

of 303

Coren, S., & Girgus, J. S. (1980). Principles of perceptual organization and spatial

distortion: the gestalt illusions. Journal of Experimental Psychology: Human

Perception and Performance, 6(3), 404.

Corbetta M & Shulman GL. (2002) Control of goal-directed and stimulus-driven

attention in the brain. Nature Rev. Neurosci. 3:201–215.

Cox, R. (1999). Representation construction, externalised cognition and individual

differences. Learning and instruction, 9(4), 343-363.

Cronbach, L. J., (1988). Internal consistency of tests: Analyses old and new.

Psychometrika, 53, 63–70.

Dahlstrom, N., & Nahlinder, S. (2006). A comparison of two recorders for obtaining in-

flight heart rate data. Applied Psychophysiology and Biofeedback, 31(3), 273-279.

Damos, D. L., John, R. S., & Lyall, E. A. (2005). “Pilot Activities and Level of Cockpit

Automation”. The International Journal of Aviation Psychology 15(3): 251-268.

Dargahi, J., & Najarian, S. (2004). Human tactile perception as a standard for artificial

tactile sensing—a review. The International Journal of Medical Robotics and

Computer Assisted Surgery, 1(1), 23-35.

Davies, D.R., & Parasuraman, R. (1982). The psychology of vigilance. London: Academic Press.

de Brito, G., & Boy, G. (1999, September). Situation awareness and procedure following. In CSAPC (Vol. 99, pp. 9-14).

Degani, A., & Kirlik, A. (1995, October). Modes in human-automation interaction:

Initial observations about a modeling approach. In Systems, Man and Cybernetics,

1995. Intelligent Systems for the 21st Century., IEEE International Conference on

(Vol. 4, pp. 3443-3450). IEEE.

Degani, A., & Wiener, E. L. (1994). On the design of flight-deck procedures (NASA

Contractor Report 177642). Moffett Field, CA: NASA Ames Research Center.

Singer, G., & Dekker, S. W. (2000). Pilot performance during multiple failures: An

empirical study of different warning systems. Transportation human factors, 2(1),

63-76.

Delzell, S., Johnson, W. W., & Liao, M. J. (1998). Pilots' spatial mental models for

memory of heading and altitude. In Digital Avionics Systems Conference, 1998.

Proceedings., 17th DASC. The AIAA/IEEE/SAE (Vol. 1, pp. E34-1). IEEE.

of 303

Desthieux, G., Joerin, F., & Lebreton, M. (2010). Ulysse: a qualitative tool for eliciting

mental models of complex systems. System Dynamics Review, 26(2), 163-192.

Desimone, R.; & Duncan, J. (1995). "Neural Mechanism of Selective Visual Attention".

Annual Review of Neuroscience 18: 193–222.

Dey, I. (1993). Qualitative data analysis: A user-friendly guide. Routledge.

Dinadis, N., & Vicente, K. J. (1999). Designing functional visualizations for aircraft

systems status displays. The international journal of aviation psychology, 9(3),

241-269.

Doble, N. A., Barhydt, R., & Hitt, J. M. (2005). Conflict Resolution Performance in an

Experimental Study of En Route Free Maneuvering Operations.

Dominguez, C. (1994). Can SA be defined? In Situation Awareness: Papers and

Annotated Bibliography (U). Armstrong Laboratory, Wright-Patterson AFB: OH.

Donald, F. M. (2008). The classification of vigilance tasks in the real world.

Ergonomics, 51(11), 1643-1655.

Dowek, G., Munoz, C., & Geser, A. (2001). Tactical conflict detection and resolution in

a 3-D airspace (No. ICASE-2001-7). Institute for Computer Applications in

Science and Engineering Hampton VA

Doyle, J., Radzicki, M., & Trees, W. (2008). Measuring change in mental models of

complex dynamic systems. Complex Decision Making, 269-294.

Doyle, J. K., & Ford, D. N. (1998). Mental models concepts for system dynamics

research. System dynamics review, 14(1), 3-29.

Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities

involved in directed skill acquisition (No. ORC-80-2). California Univ Berkeley

Operations Research Center.

Durand, N., Alliot, J. M., & Noailles, J. (1996, February). Automatic aircraft conflict

resolution using genetic algorithms. In Proceedings of the 1996 ACM symposium

on Applied Computing (pp. 289-298). ACM.

Edworthy, J., & Hellier, E. (2006). Alarms and human behaviour: implications for

medical alarms. British Journal of Anaesthesia, 97(1), 12-17.

Edwards, W. (1954). The theory of decision making. Psychological bulletin, 51(4), 380.

Eggemeier, F. T. (1988). Properties of workload assessment techniques. Advances in

Psychology, 52, 41-62.

of 303

Elgin, P.D. and Thomas, R.P. (2004). An Integrated Decision-Making Model for

Categorizing Weather Products and Decision Aids. NASA/TM-2004-212990

Ellerbroek, J., Visser, M., van Dam, S. B. J., Mulder, M., & van Paassen, M. M. (2009).

Towards an Ecological Four-Dimensional Self-Separation Assistance Display.

Emery, F. E., & Trist, E. L. (1960). Sociotechnical systems. In C. W. Churchman, &

Verhulst, M. (Ed.),Management Science Models and Techniques(Vol. 2, pp. 83-

97).Oxford, UK: Pergamon

Endsley, M. R., (1988). Situational awareness global assessment technique (SAGAT).

In Proceedings of the National Aerospace and Electronics Conference. 789-79S.

Endsley, M. R. (1995a). “Measurement of situation awareness in dynamic systems”.

Human Factors, 37(1), 65-84.

Endsley, M. R. (1995b). “Toward a theory of situation awareness in dynamic systems”.

Human Factors 37(1), 32-64.

Endsley, M. R. (1999). Situation awareness in aviation systems. Handbook of aviation

human factors, 257-276.

Endsley, M. R. (2000). “Theoretical underpinnings of situation awareness: A critical

review”. In M. R. Endsley & D. J. Garland (Eds.), Situation awareness analysis

and measurement. Mahwah, NJ: LEA.

Endsley, M. R. (2001). Designing for situation awareness in complex systems.

Proceedings of the Second international workshop on symbiosis of humans,

artifacts and environment, Kyoto, Japan.

Endsley, M. R. (2006). Expertise and Situation Awareness. Handbook of Human

Factors and Ergonomics. G. Salvendy, John Wiley & Sons, Inc.

Endsley, M. R., Bolte, B., & Jones, D. G. (2003). Designing for situation awareness:

An approach to human-centered design. London: Taylor & Francis.

Endsley, M. R. & Kiris, E. O. (1995). “The out-of-the-loop performance problem and

level of control in automation”. Human Factors, 37, 381-394.

Endsley, M. R., & Strauch, B. (1997). Automation and situation awareness: The

accident at Cali, Columbia. In Proceedings of the ninth international symposium

on aviation psychology (pp. 877-881).

of 303

Erceg-Hurn, D. M., & Mirosevich, V. M. (2008). Modern robust statistical methods: an

easy way to maximize the accuracy and power of your research. American

Psychologist, 63(7), 591.

Erzberger, H. (2004). Transforming the NAS: The next generation air traffic control

system. In 24th International Congress of the Aeronautical Sciences, Yokohama,

Japan.

Erzberger, H., & Paielli, R. A. (2002). Concept for next generation air traffic control

system. Air Traffic Control Quarterly, 10(4), 355-378.

Erev, I., Barron, G., & Remington, R. (2004). Right of way in the sky: Two problems in

aircraft self-separation and the auction-based solution. Human Factors: The

Journal of the Human Factors and Ergonomics Society, 46(2), 267-276.

Eberts, R. E. (1993). User Interface Design. Englewood Cliffs, NJ: Prentice Hall

Erdfelder, E., Faul, F., & Buchner, A. (1996). GPOWER: A general power analysis

program. Behavior Research Methods, Instruments, & Computers, 28, 1-11.

Faerber, B. (1999). Advanced flight management system human interface. Air & Space

Europe, 1(1), 42-44.

Federal Aviation Administration. (2000). Air Traffic Control (7110.65M). Washington

DC: U.S. Government Printing Office. Traffic Advisories Para 2-1-21.

Flach, J. M. (1995). Situation awareness: Proceed with caution. Human Factors: The


Flach, J. M., Tanabe, F., Monta, K., Vicente, K. J., & Rasmussen, J. (1998, October).

An ecological approach to interface design. In Proceedings of the Human Factors


Publications.

Fitts, P.M. (1951) “Human Engineering for an Effective Air Navigation and Traffic

Control System,” Ohio State University Research Foundation, Columbus.

Ford, J. K., MacCallum, R. C., & Tait, M. (1986). The application of exploratory factor

analysis in applied psychology: A critical review and analysis. Personnel

Psychology, 39(2), 291-314.

of 303

Ford, S., Daniels, J., Lim, J., Koval, V., Dumont, G., Schwarz, S. K., & Ansermino, J.

M. (2008). A novel vibrotactile display to improve the performance of

anesthesiologists in a simulated critical incident. Anesthesia & Analgesia, 106(4),

1182-1188.

Ford, D. N., & Sterman, J. D. (1998). Expert knowledge elicitation to improve formal

and mental models. System Dynamics Review, 14(4), 309-340.

Ford, S., Daniels, J., Lim, J., Koval, V., Dumont, G., Schwarz, S. K., & Ansermino, J.

M. (2008). A novel vibrotactile display to improve the performance of

anesthesiologists in a simulated critical incident. Anesthesia & Analgesia, 106(4),

1182-1188.

Folk, C., & Gibson, B. (Eds.). (2001). Attraction, distraction and action: Multiple

perspectives on attentional capture (Vol. 133). Access Online via Elsevier.

Forster, M. R. (1999). How do Simple Rules Fit to Reality' in a Complex World?.

Minds and Machines, 9(4), 543-564.

Franklin, J. (2000). Diagrammatic reasoning and modelling in the imagination: the

secret weapons of the Scientific Revolution. In 1543 and All That (pp. 53-115).

Springer Netherlands.

Franconeri, S. L., & Simons, D. J. (2003). Moving and looming stimuli capture

attention. Perception & psychophysics, 65(7), 999-1010.

Fuchs-Frothnhofen, P., Hartmann, E. A., Brandt, D., & Weydandt, D. (1996). Designing

human-machine interfaces to match the user's mental models. Control

Engineering Practice, 4(1), 13-18.

Funk, K. (2009). Prospective Identification of NextGen Flight Deck Human Factors

Issues

Funk, K., Rorbert M. & Barshi, I. (2009).“NextGen Flight Deck Human Factors

Issues”. International Symposium on Aviation Psychology. Dayton, OH.

Funk K., Robert M. & Harvey B. (2008).“Identifying and Addressing Human Factors

Issues of ADS-B and Cockpit Displays of Traffic Information” Presentation at

Human Factors and NextGen: The Future of Aviation University of Texas,

Arlington, Texas.

of 303

Funk, K., Lyall, B., Wilson, J., Vint, R., Niemczyk, M., Suroteguh, C., & Owen, G.

(1999). Flight deck automation issues. The International Journal of Aviation

Psychology, 9(2), 109-123.

Gaver, W. W. (1997). Auditory interfaces. Handbook of human-computer interaction,

1, 1003-1041.

Graham, W., & Orr, R. H. (1970). Separation of air traffic by visual means: an estimate

of the effectiveness of the see-and-avoid doctrine. Proceedings of the IEEE, 58(3),

337-361.

Graham, W. (1989). See and avoid/cockpit visibility. Questek Inc Centerport, NY.

Geer, J. G. (1988). What do open-ended questions measure? Public Opinion Quarterly,

52(3), 365-367.

Gersh, J. R., McKneely, J. A., & Remington, R. W. (2005). Cognitive engineering:

Understanding human interaction with complex systems. Johns Hopkins APL

technical digest, 26(4), 377-382.

Gentner D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive

Science, 7, pp 155-170.

Genter, D., & Stevens, A.L., Ed. (1983).”Mental Models” Hillsdale, New Jersey,

Erlbaum.

George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and

reference. 11.0 update (4th ed.). Boston: Allyn & Bacon

Guest, Greg; Bunce, Arwen & Johnson, Laura (2006). "How many interviews are

enough? An experiment with data saturation and variability". Field Methods,

18(1), 59-82.

Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton-

Mifflin.

Gibb, R., Ercoline, B., & Scharff, L. (2011). Spatial disorientation: decades of pilot

fatalities. Aviation, space, and environmental medicine, 82(7), 717-724.

Gilbert, D. T., & Ebert, J. E. (2002). Decisions and revisions: the affective forecasting

of changeable outcomes. Journal of personality and social psychology, 82(4), 503.

Glenberg, A. M., & Langston, W. E. (1992). Comprehension of illustrated text: Pictures

help to build mental models. Journal of memory and language, 31(2), 129-151.

of 303

Gopher, D. & Donchin, E. (1986). Workload – An examination of the concept.

Handbook of Perception andHuman Performance. Volume 2. Cognitive Processes

and Performance. K.R. Boff, L. Kaufman andJ.P. Thomas, John Wiley and Sons,

Inc: 41-1:41-49.

Greca, I. M., & Moreira, M. A. (2000). Mental models, conceptual models, and

modelling. International Journal of Science Education, 22(1), 1-11.

Hameed, S., Ferris, T., Jayaraman, S., & Sarter, N. (2009). Using informative peripheral

visual and tactile cues to support task and interruption management. Human

Factors: The Journal of the Human Factors and Ergonomics Society, 51(2), 126-

135.

Hamm, R.M. (1988). Clinical intuition and clinical analysis: Expertise and the cognitive

continuum. In J. Dowie and A. Elstein (Eds.), Professional judgment: A reader in

clinical decision making (pp. 78-105). Cambridge, UK: Cambridge University

Press

Hannond, K. R., Hamm, R. M., Grassia, J., & Pearson, T. (1987). Direct comparison of

the efficacy of intuitive and analytical cognition in expert judgment. Systems, Man

and Cybernetics, IEEE Transactions on, 17(5), 753-770.

Hastie, R.. (2001). Problems for judgment and decision making. Annual review of

psychology, 52(1), 653-683.

Hart, S. G. (2006, October). NASA-task load index (NASA-TLX); 20 years later. In

Proceedings of the human factors and ergonomics society annual meeting (Vol.

50, No. 9, pp. 904-908). Sage Publications.

Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index):

Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati

(Eds.), Human mental workload (pp. 139-183). Amsterdam: Elsevier.

Hair, J. F., Jr., Rolph E. A, Ronald L. T., & William C. B. (1998), Multivariate Data

Analysis. 5th ed. Englewood Cliffs, NJ: Prentice Hall.

Hancock P. A. & Joel. S. Warm (1996). Vigilance and Workload in Automated

Systems. R. P. a. M. Mouloa(Eds). N.J, Eribuam, Hillsdale: pp.183-200.

Hancock, P. A., Williams, G., & Manning, C. M. (1995). “Influence of task demand

characteristics on workload and performance”. The International Journal of

Aviation Psychology, 5(1), 63-86.

of 303

Haskell, I. D., & Wickens, C. D. (1993). Two-and three-dimensional displays for

aviation: A theoretical and empirical comparison. The International Journal of


Hawkins, D. M. (1980), “Identification of outliers," Chapman and Hall, London

Helton, W. S., & Warm, J. S. (2008). Signal salience and the mindlessness theory of

vigilance. Acta Psychologica, 129, 18–25.

Herrera, J. M., & Vasigh, B. (2011). A Basic Analysis Of Aging Aircraft, Region Of

The World, And Accidents. Journal of Business & Economics Research (JBER),

7(5).

Heylen, F. M., Van Dam, S. B. J., Mulder, M., & van Paassen, M. M. (2008). Design

and evaluation of a vertical separation assistance display. Interface, 19(22), 17.

Higgins, P. G. (1998, November). Extending cognitive work analysis to manufacturing

scheduling. In Computer Human Interaction Conference, 1998. Proceedings.

1998 Australasian (pp. 236-243). IEEE.

Hill S., Lavecchia H., Byers J., Bitt ner A., Zaklad A. & Christ R. (1992). Comparison

of four subjective workload rating scales, Human Factors, 34, 4, 429-439

Hillstrom, A. P., & Yantis, S. (1994). Visual motion and attentional capture. Perception

& Psychophysics, 55(4), 399-411.

Hilbum, B., Jorna, P.G.A.M., Byrne, E. A., & Parasuraman, R. (1997). The effect

of adaptive air traffic control (ATC) decision aiding on controller mental

workload. In M. Mouloua. & J. M. Konce (Eds.), Human automation

interaction: Research and practice, (pps. 84-91). Mahwah, NJ: Erlbaum

Hirtle, S. C., & Hudson, J. (1991). Acquisition of spatial knowledge for routes. Journal

of Environmental Psychology, 11(4), 335-345.

Ho, D., & Burns, C. M. (2003). Ecological interface design in aviation domains: Work

domain analysis of automated collision detection and avoidance. In Proceedings

of the Human Factors and Ergonomics Society Annual Meeting (Vol. 47, No. 1,

pp. 119-123). SAGE Publications.

Ho, C., Tan, H. Z., & Spence, C. (2005). Using spatial vibrotactile cues to direct visual

attention in driving scenes. Transportation Research Part F: Traffic Psychology

and Behaviour, 8(6), 397-412.

of 303

Hoc, J. M. (2007). Human and automation: a matter of cooperation. In HUMAN 07 (pp.

277-285). Université de Metz.

Hoedemaeker, M., & Brookhuis, K. A. (1998). Behavioural adaptation to driving with

an adaptive cruise control (ACC). Transportation Research Part F: Traffic

Psychology and Behaviour, 1(2), 95-106.

Hoekstra, J. M., van Gent, R. N., & Ruigrok, R. C. (2002). Designing for safety: the

‘free flight’air traffic management concept. Reliability Engineering & System

Safety, 75(2), 215-232.

Hoffmann, E. R., & Mortimer, R. G. (1994). Drivers' estimates of time to collision.

Accident Analysis & Prevention, 26(4), 511-520.

Hollnagel E. (2007). “Flight decks and free flight: Where are the system boundaries?”

Applied Ergonomics 38(4) pp. 409-416.

Hollnagel, E., & Woods, D. D. (1983). Cognitive systems engineering: New wine in

new bottles. International Journal of Man-machine Studies, 18(6), 583-600.

Horrey, W. J., & Wickens, C. D. (2001). Supporting battlefield situation assessment

through attention guidance and diagnostic aiding: A cost-benefit and depth of

processing analysis. (Technical Report ARL-01-16/FED-LAB-01-1). Savoy, IL:

University ofIllinois at Urbana-Champaign, Aviation Research Lab.

Huey, F.M. & Wickens, C.D. (1993). Workload transition: Implications for individual

and team performance. Washington, DC, National Academy Press

Hull, D. G. (2007). Atmosphere, Aerodynamics, and Propulsion. Fundamentals of

Airplane Flight Mechanics, 43-78.

Hunt, E., & Lavoie, A. M. (2011). Quantitative and qualitative research methods, can

they coexist yet?. Recherche en soins infirmiers, (105), 25.

Hutchins, E. (1995) “How a cockpit remembers its speeds”. Cognitive Science 19, pp.

265-288.

Hutchins, S. G. (1997) Decision-making errors demonstrated by experienced naval

officers in a littoral environment. In G. Klein & C. E. Zsambok (Eds.) Naturalistic

Decision Making (pp. 207-215) Lawrence Erlbaum Associates Inc., Mahwah, NJ,

1997.

of 303

Hunter, A., & Randall, T. (2013). Using cognitive work analysis to drive usability

evaluations in complex systems. In Engineering Psychology and Cognitive

Ergonomics. Understanding Human Cognition (pp. 39-48). Springer Berlin

Heidelberg.

Illman, P. E. ( 1993). The Pilot’s Air Traffic Control Handbook, 2nd Edition. TAB

Books. New York, New York.

Iqbal, S. T., Zheng, X. S., & Bailey, B. P. (2004, April). Task-evoked pupillary

response to mental workload in human-computer interaction. In CHI'04 extended

abstracts on Human factors in computing systems (pp. 1477-1480). ACM.

Jamieson, G. A., & Vicente, K. J. (2001). Ecological interface design for petrochemical

applications: Supporting operator adaptation, continuous learning, and distributed,

collaborative work. Computers & Chemical Engineering, 25(7), 1055-1074.

Jaspers, M. W., Steen, T., Van Den Bos, C., & Geenen, M. (2004). The think aloud

method: a guide to user interface design. International journal of medical

informatics, 73(11-12), 781.

Jenkins, D. P., Stanton, N. A., Salmon, P. M. Walker, G. H. & Rafferty, L. (2008).

Using the decision-ladder to add a formative element to naturalistic decision

making research. HFI DTC report no HFIDTC/2/WP4.6.1/2.

Jensen, R. S., Adrion, J., & Lawton, R. S. (1987). Aeronautical Decision Making for

Instrument Pilots. System Control Technology Inc Arlington VA.

John, M. S. (2005). "Interruptions, Distractions, and Attention Management: A Multi-

Faceted Problem for Human Factors." Proceedings of the Human Factors and

Ergonomics Society 49th Annual Meeting: 462.

Johnson-Laird, P. N. (1995). Mental models, deductive reasoning, and the brain. The

cognitive neurosciences, 999-1008.

Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative,

and mixed approaches (p. 34)

Johnson, C. W., & Dell, W. (2003). Limitations of 3D audio to improve auditory cues in

aircraft cockpits. In International Systems Safety Conference. Unionville, VA:

International Systems Safety Society.

Johnson, W. W., Battiste, V., & Bochow, S. H. (2003). A Cockpit Display Designed to

Enable Limited Flight Deck Separation Responsibility.

of 303

Johnson, W.W., Bilimoria, K.D., Thomas, L.C., Lee, H.Q., & Battiste, V. (2003).

“Comparison of human and automation generated conflict resolutions” AIAA

Guidance, Navigation, & Control Conference, August 2003.

Jones, D. G., & Endsley, M. R. (1996). Sources of situation awareness errors in

aviation. Aviation, space, and environmental medicine, 67(6), 507-512.

Jones, K. S. (2003) 'What Is an Affordance?', Ecological Psychology, 15(2), 107- 114

Jones, N. A., Ross, H., Lynam, T., Perez, P., & Leitch, A. (2000). Mental Models: An

Interdisciplinary Synthesis of Theory and Methods. Ecology and Society,16(1),

46.

Jonassen, D. H., & Henning, P. H. (1996). Mental models: Knowledge in the head and

knowledge in the world. In Proceedings of the 2nd International Conference on

the Learning Sciences. Evanston, IL, Northwestern University Press

Joshi, A., Miller, S. P., & Heimdahl, M. P. (2003, October). Mode confusion analysis of

a flight guidance system using formal methods. In 22nd Digital Avionics Systems

Conference DASC’03(p. 2).

Kaber, D. B., Alexander, A. L., Stelzer, E. M., Kim, S. H., Kaufmann, K., & Hsiang, S.

(2008). Perceived clutter in advanced cockpit displays: measurement and

modeling with experienced pilots. Aviation, space, and environmental medicine,

79(11), 1007-1018.

Kaber, D. B., & Endsley, M. R. (1997). Out‐of‐the‐loop performance problems and the

use of intermediate levels of automation for improved control system functioning

and safety. Process Safety Progress, 16(3), 126-131.

Kaczmarek, K. A (2000). “Sensory Augmentation and Substitution.” The Biomedical

Engineering Handbook: Second Edition. Ed. Joseph D. Bronzino Boca Raton:

CRC Press LLC.

Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American

psychologist, 39(4), 341.

Kaufman, A. S., & McLean, J. E. (1998). An investigation into the relationship between

interests and intelligence. Journal of clinical psychology, 54(2), 279-295.

Kessel, C. J., & Wickens, C. D. (1982). The transfer of failure-detection skills between

monitoring and controlling dynamic systems. Human Factors: The Journal of the


of 303

Klein, G. A., Calderwood, R., & Clinton-Cirocco, A. (1986). Rapid decision making on

the fireground. Proceedings of the Human Factors and Ergonomics Society 30th

Annual Meeting, 1, 576–580.

Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision

making. Ablex Publishing Corporation.

Klein, G. A. (1989). Strategies of decision making. Klien Associates Inc Yellow

Springs OH.

Klein, G. A. (2008). Natural Decision Making, Human Factors, Vol. 50, No. 3, June

2008, pp. 456–460.

Klein, G. A., Orasanu, J., Calderwood, R., & Zsambok, C.E. (1993) Decision Making in

Action: Models and Methods. Ablex Publishing Co., Norwood, NJ.

Kieras, D. E. (1990). The role of cognitive simulation models in the development

of advanced training and testing systems. In N. Frederiksen, R. Glaser, A.

Kieras D. E. & Bovair S. ( 1984) The role of a mental model in learning to

operate a device. Cognitive Science 8: 255-73.

Kieras, D., & Polson, P. G. (1983, December). A generalized transition network

representation for interactive systems. In Proceedings of the SIGCHI conference

on Human Factors in Computing Systems (pp. 103-106). ACM.

Kim, S. H., Prinzel, L. J., Kaber, D. B., Alexander, A. L., Stelzer, E. M., Kaufmann, K.,

& Veil, T. (2011). Multidimensional measure of display clutter and pilot

performance for advanced head-up display. Aviation, space, and environmental

medicine, 82(11), 1013-1022.

Kochan, J.A., Jensen, R.S. & Chubb, G.P. (1997). A new approach to aeronautical

decision-making: The expertise method (DOT/FAA/AM-97/6). Washington, DC:

Federal Aviation Administration, Office of Aviation Medicine.

Koren, Y., & Borenstein, J. (1991, April). Potential field methods and their inherent

limitations for mobile robot navigation. In Robotics and Automation, 1991.

Proceedings., 1991 IEEE International Conference on (pp. 1398-1404). IEEE.

Krantz, D. H., & Kunreuther, H. C. (2007). Goals and plans in decision making.

Judgment and Decision Making, 2(3), 137-168.

http://en.wikipedia.org/wiki/Gary_A._Klein

of 303

Kuchar, J. K., Andrews, J., Drumm, A., Hall, T., Heinz, V., Thompson, S., & Welch, J.

(2004). A Safety Analysis Process for the Traffic Alert and Collision Avoidance

System (TCAS) and See-and-Avoid Systems on Remotely Piloted Vehicles. In

AIAA 3rd" Unmanned Unlimited" Technical Conference, Workshop and Exhibit,

September 20-23 2004, Chicago, Illinois.

Kuchar, J. E., & Drumm, A. C. (2007). The traffic alert and collision avoidance system.

Lincoln Laboratory Journal, 16(2), 277

Kuchar, J. K., & Yang, L. C. (2000). A review of conflict detection and resolution

modeling methods. Intelligent Transportation Systems, IEEE Transactions on,

1(4), 179-189.

Kuusela, H., & Paul, P. (2000). A comparison of concurrent and retrospective verbal

protocol analysis. American Journal of Psychology, 113(3), 387-404.

Krozel, J., & Peters, M. (1997, December). Strategic conflict detection and resolution

for free flight. In Decision and Control, 1997., Proceedings of the 36th IEEE

Conference on (Vol. 2, pp. 1822-1828). IEEE

Krahmer, E., & Ummelen, N. (2004). Thinking about thinking aloud: A comparison of

two verbal protocols for usability testing. Professional Communication, IEEE

Transactions on, 47(2), 105-117.

Kramer, A., & Jacobson, A. (1991). Perceptual organization and focused attention: The

role of objects and proximity in visual processing. Perception & Psychophysics,

50 , 267–284

LaBerge, D., & Brown, V. (1989). Theory of attentional operations in shape

identification. Psychological Review, 96,101-124.

Lachter, J., Forster, K. I., & Ruthruff, E. (2004). Forty-five years after Broadbent

(1958): still no identification without attention. Psychological review, 111(4),

880.

Lang, P. J., Davis, M., & Ohman, A. (2000). Fear and anxiety: animal models and

human cognitive psychophysiology. Journal of affective disorders, 61(3), 137-

159.

Lau, N., & Jamieson, G. A. (2006). Ecological interface design for the condenser

subsystems of a boiling water reactor. In Proceedings of the 16th World Congress

on Ergonomics.

of 303

Lau, N., Veland, O., Kwok, J., Jamieson, G. A., Burns, C. M., Braseth, A. O., &

Welch, R. (2008). Ecological interface design in the nuclear domain: An

application to the secondary subsystems of a boiling water reactor plant simulator.

IEEE Transactions on Nuclear Science, 55, 3579-3596.

Latorella A. K. (1998). “Effects of Modality on Interrupted Flight Deck Performance:

Implications for Data Link” Proceedings of the Human Factors and Ergonomics

Society 42nd Annual Meeting.

Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten

thousand words. Cognitive science, 11(1), 65-100.

Lawson, R. (2006). The science of cycology: Failures to understand how everyday

objects work. Memory & cognition, 34(8), 1667-1675.

Layton, C., Smith, P. J., & McCoy, C. E. (1994). Design of a cooperative problem-

solving

system for en-route flight planning: An empirical evaluation. Human Factors, 36(1),

94-119.

Lee, J. D., McGehee, D. V., Brown, T. L., & Reyes, M. L. (2002). Collision warning

timing, driver distraction, and driver response to imminent rear-end collisions in a

high-fidelity driving simulator. Human Factors: The Journal of the Human

Factors and Ergonomics Society, 44(2), 314-334.

Lee, D. S., & Yoon, W. C. (2004). Coupling structural and functional models for

interaction design. Interacting with computers, 16 (1), 133-161.

Lee, J., & Moray, N. (1992). Trust, control strategies and allocation of function in

human-machine systems. Ergonomics, 35(10), 1243-1270.

Lee J. D., & See K. A., (2004). “Trust in Automation: Designing for Appropriate

Reliance” Human Factors, 46(1), 30-80.

Lee, D. N., (1976). A theory of visual control of braking based on information about

time-to-collision. Perception, 5(4), 437-459.

Lee R. J., & Nicewander, W. A. (1988). Thirteen ways to look at the correlation

coefficient. The American Statistician, 42(1), 59-66.

Leung, W. C. (2001). How to design a questionnaire. Student British Medical Journal,

9, 187-9.

of 303

Lind, M. (1999, September). Making sense of the abstraction hierarchy. In Proc.

Proceedings of the seventh European Conference on Cognitive Science

Approaches to Process Control (pp. 195-200).

Lintern, G. (2000). “An Affordance-Based Perspective on Human-Machine Interface

Design” Ecological Psychology, 12(1), 65- 69

Lintern, G. (2005). Integration of Cognitive Requirements into System Design.

Proceedings of the 49th Human Factors and Ergonomics Society Annual Meeting.

(pp. 239-243). Santa Monica, CA: Human Factors and Ergonomics Society

Lintern, G., Waite, T., & Talleur, D. A. (1999). Functional interface design for the

modern aircraft cockpit. The International Journal of Aviation Psychology, 9(3),

225-240.

Linderman, K., Schroeder, R. G., Zaheer, S., & Choo, A. S. (2003). Six Sigma: a goal-

theoretic perspective. Journal of Operations management, 21(2), 193-203.

Lindeman, R.W., Yanagida, Y., Sibert, J.L., & Lavine, R. (2003). Effective Vibrotactile

Cueing in a Visual Search Task. Proc. of the Ninth IFIP TC13 Int'l Conf. on

Human-Computer Interaction (INTERACT 2003), Sep. 1-5, 2003 Zuerich,

Switzerland, 89-96.

Lipshitz, R., Klein, G., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic

decision making. Journal of Behavioral Decision Making, 14(5), 331-35

Livack, G. S., McDaniel, J. I., Battiste, V., & Johnson, W. W. (2000). The human

element in Automatic Dependent Surveillance-Broadcast flight operations. In

Digital Avionics Systems Conference, 2000. Proceedings. DASC. The 19th (Vol. 2,

pp. 5D4-1). IEEE.

Loft, S., Sanderson, P., Neal, A., & Mooij, M. (2007). Modeling and predicting mental

workload in en route air traffic control: Critical review and broader implications.

Human Factors: The Journal of the Human Factors and Ergonomics Society,

49(3), 376-399.

Lundgrén-Laine, H., & Salanterä, S. (2010). Think-aloud technique and protocol

analysis in clinical decision-making research. Qualitative Health Research, 20(4),

565-575.

of 303

Luximon, A., & Goonetilleke, R. S. (2001). Simplified subjective workload assessment

technique. Ergonomics, 44(3), 229-243.

Lysaght, R. J., Hill, S. G., Dick, A. O., Plamondon, B. D., & Linton, P. M. (1989).

Operator workload: Comprehensive review and evaluation of operator workload

methodologies (No. TR-2075-3). Analytics Inc Willow Grove PA

Mackintosh, M., Dunbar, M., Lozito, S., Cashion, P., McGann, A., Dulchinos, V., &

Valenti Clari, M. (1998, December). Self-separation from the air and ground

perspective. In Air Traffic Management R&D Seminar.

Mani, K., & Johnson-Laird, P. N. (1982). The mental representation of spatial

descriptions. Memory & Cognition, 10(2), 181-187.

Marino, A. C.; & Scholl, B. J. (2005). "The role of closure in defining the "objects" of

object-based attention". Perception & Psychophysics 67 (7): 1140–1149.

McCann, R. S., & Foyle, D. C. (1996). Scene-linked symbology to improve situation awareness. In AGARD CONFERENCE PROCEEDINGS AGARD CP (pp. 16-16). AGARD.

McDonald, N. (2007, July). “Human integration in the lifecycle of aviation systems”. In

D. Harris (Ed.). Proceedings of the 7th international conference on Engineering

psychology and cognitive ergonomics, Beijing, China. Springer-Verlag Berlin,

Heidelberg: pp 760-769

McFarlane, D. C. (2002). "'Comparison of Four Primary Methods for Coordinating the

Interruption of People in Human-Computer " Human-Computer Interaction 17(1),

63 - 139.

McKinley, R. A., Gallimore, J., Lanning, C., & Simmons, C. (2005). Tactile cueing for

target acquisition and identification (No. AFRL-HE-WP-TR-2005-0009). Air

Force Research Lab Wright-Patterson AFB OH Human Effective Directorate

McLeod R W, and Boss H E, (1983), "Optic-flow and cognitive factors in time-to-

collision estimates" Perception 12(4) 417– 423.

McMenamin, (1995). Operational decision making: The impact of time and

information. Naval War College, Newport, RI.

Meister, D. (1999). The history of human factors and ergonomics. CRC Press.

Meredith, C., & Edworthy, J. (1995). Are there too many alarms in the intensive care

unit? An overview of the problems. Journal of advanced nursing, 21(1), 15-20.

http://perception.research.yale.edu/papers/05-Marino-Scholl-PandP.pdf

http://perception.research.yale.edu/papers/05-Marino-Scholl-PandP.pdf

of 303

Metzger, U., & Parasuraman, R. (2001). The role of the air traffic controller in future air

traffic management: An empirical study of active control versus passive

monitoring. Human Factors: The Journal of the Human Factors and Ergonomics

Society, 43(4), 519-528.

Merz, A.W. (1991) “Maximum -Miss Aircraft Collision Avoidance” Dynamics and Control 1, pp. 25-34

Miller, G. A. (1956). "The Magical Number Seven, Plus or Minus Two: Some Limits

on Our Capacity for Processing Information." The Psychological Review 63: pp.

81-97.

Miller, C. A., & Parasuraman, R. (2007). Designing for flexible interaction between

humans and automation: Delegation interfaces for supervisory control. Human


75.

Miller, A. (2004) “A work domain analysis framework for modeling intensive care unit

patients” Cognition Technology Work 6 pp. 207–222.

Mitchell, C. M., & Miller, R. A. (1986). A discrete control model of operator function:

A methodology for information display design. Systems, Man and Cybernetics,

IEEE Transactions on, 16(3), 343-357.

Molloy, R., & Parasuraman, R. (1996). Monitoring an automated system for a single

failure: Vigilance and task complexity effects. Human Factors: The Journal of the


Mogford, R. H. (1997). Mental models and situation awareness in air traffic control.

The International Journal of Aviation Psychology, 7(4), 331-341.

Montello, D. (1997). The perception and cognition of environmental distance: Direct

sources of information. Spatial Information Theory A Theoretical Basis for GIS,

297-311.

Morris, C. C. (2005). Midair collisions: Limitations of the see-and-avoid concept in

civil aviation. Aviation, space, and environmental medicine, 76(4), 357-365.

Moray, N. (1979). “Mental Workload: Its Theory and Measurement”. NewYork:

Plenum.

of 303

Moray, N. (1986), “Monitoring Behavior and Supervisory Control,” in Handbook of

Perception and Human Performance, Vol. II, Cognitive Processes and

Performance, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds., Wiley, New York,

pp. 40–1 to 40-51.

Moray, N. (1998). "Identifying mental models of complex human -machine systems."

International Journal of Industrial Ergonomics 22, 294-297.

Moray, N. (1999). “Mental Models in Theory and Practice” Attention and

Performance XVII cognitive regulation of performance: Interaction of Theory and

Application D. K. Gopher, Asher. Cambridge, Mass, MIT Press.

Moray, N., & Haudegond, S. (1998, October). An absence of vigilance decrement in a

complex dynamic task. In Proceedings of the Human Factors and Ergonomics

Society Annual Meeting (Vol. 42, No. 3, pp. 234-236). SAGE Publications.

Moray N., Inagaki T & Itoh, M.(2000) “Adaptive Automation, Trust and Self-

confidence in Fault Management of Time-Critical Tasks” Journal of Experimental

Psychology: Applied 6(1), 44-58.

Morio, J., Lang, T., & Tallec, C. L. (2012). Estimating separation distance loss

probability between aircraft in uncontrolled airspace in simulation. Safety Science.

Morrison, B. W., Wiggins, M. W., Bond, N. W., & Tyler, M. D. (2009, June).

Examining cue recognition across expertise using a computer-based task. In

Proceedings of the 9th International Conference on Naturalistic Decision Making:

pp. 91-98.

Mosier, K. L., & Fischer, U. M. (2010). Judgment and decision making by individuals

and teams: Issues, models, and applications. Reviews of human factors and

ergonomics, 6(1), 198-256.

Mou, W., McNamara, T. P., Rump, B., & Xiao, C. (2006). Roles of egocentric and

allocentric spatial representations in locomotion and reorientation. Journal of

Experimental Psychology: Learning, Memory, and Cognition, 32(6), 1274.

Mozer, M. C., & Vecera, S. P. (2005). Space-and object-based attention. Neurobiology

of attention, 130-134.

of 303

Muthard, E. K., & Wickens, C. D. (2001). Change detection in a flight planning task

environment: An examination of the confirmation bias and its relation to decision

making errors. Technical Report ARL-01-18/NASA-01-9). Savoy, IL: University

of Illinois at Urbana-Champaign, Aviation Research Lab.

Nachreiner, F. (1995). Standards for ergonomics principles relating to the design of

work systems and to mental workload. Applied Ergonomics, 26 (4), 259- 263.

Naikar, N., Hopcroft, R., & Moylan, A. (2005). Work domain analysis: Theoretical

concepts and methodology (No. DSTO-TR-1665). Defence Science and

Technology Organisation Victoria (AUSTRALIA) Air Operations DIV.

Naikar, N., Moylan, A., & Pearce, B. (2006). Analysing activity in complex systems

with cognitive work analysis: concepts, guidelines and case study for control

task analysis. Theoretical Issues in Ergonomics Science, 7(4), 371-394.

Naikar, N., Pearce, B., Drumm, D., & Sanderson, P. M. (2003). Designing teams for

first-of-a-kind, complex systems using the initial phases of cognitive work

analysis: case study. Human factors, 45(2), 202-217.

Nersessian, N. J. (1992). How do scientists think? Capturing the dynamics of

conceptual change in science. Cognitive models of science, 15, 3-44.

Neisser, U. & Becklen R. (1975) “Selective looking: Attending to visually specified

events” Cognitive Psychology 7, 480 – 494

Niessen, C., & Eyferth, K. (2001). A model of the air traffic controller's picture. Safety

Science, 37(2), 187-202.

Nikolic, M. I., Orr, J. M., & Sarter, N. B. (2004). Why pilots miss the green box: How

display context undermines attention capture. The International Journal of

Aviation Psychology,14(1), 39-52.

Noble, D. (1998). Distributed situation assessment. In Proc. FUSION (Vol. 98, pp. 478-

485).

Norman, D. A. (1983) “Designs Rules Based on Analysis of Human Error”

Communications of the ACM (26) 4.

Norman D. A (1983a) “Some Observation of mental models” Gentner, D., & Stevens,

A. L. (Eds.). (1983). Mental models. Psychology Press.

Norman, D. A. (1986). Cognitive engineering. User centered system design, 31-61.

of 303

Norman, D. A. (1990). “ The “Problem” of Automation :Inappropriate Feedback and

Interaction, not Over Automation” In D. E. Broadbent, A. Baddeley & J. T.

Reason (Eds.), Human factors in hazardous situations (pp. 585-593). Oxford:

Oxford University Press.

Norman, D.A & Bobrow D. G. (1975) “On Data-limited and Resource-limited

Processes” Cognitive Psychology 7, 44- 64.

Nuechterlein, K. H., Parasuraman, R., & Jiang, Q. (1983). Visual sustained attention:

Image degradation produces rapid sensitivity decrement over time. Science,

220(4594), 327-329.

Nullmeyer, R. T., Stella, D., Montijo, G. A., & Harden, S. W. (2005). Human factors in

Air Force flight mishaps: Implications for change. Proceedings of the 27th Annual

Inter-service/Industry Training, Simulation, and Education Conference (paper no.

2260). Arlington, VA: National Training Systems Association.

O’Hare, D. (2000). “The "Wheel of Misfortune’: a taxonomic approach to human

factors in accident investigation and analysis in aviation and other complex

systems." Ergonomics, 43(12): pp. 2001-2019.

Orasanu, Judith, & Fischer, Ute. (1997). Finding decisions in natural environments: The

view from the cockpit. In G. K. Caroline E. Zsambok (Ed.), Naturalistic Decision

Making (343-357). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers

Owen, M. R. (1993). The Memphis Precision Runway Monitor Program Instrument

Landing System: Final Approach Study (No. ATC-194). Lincoln Laboratory,

Massachusetts Institute of Technology.

Paielli, R. A. (2003). Modeling maneuver dynamics in air traffic conflict resolution.

Journal of guidance, control, and dynamics, 26(3), 407-415.

Paielli, R. A. (2011). Evaluation of Tactical Conflict Resolution Algorithms for En-

route Airspace. Journal of Aircraft, 48(1), 324.

Paige, J. M., & Simon, H. A. (1966). Cognitive processes in solving algebra word

problems. In B. Kleinmuntz (Ed.), Problem solving: Research, method and theory

(pp. 51-148). Wiley, New York

of 303

Palmer, E. (1983, October). Conflict resolution manoeuvres during near miss

encounters with cockpit traffic displays. In Proceedings of the Human Factors

and Ergonomics Society Annual Meeting (Vol. 27, No. 9, pp. 757-761). Sage

Publications.

Palmer, E. A., Jago, S. J., Baty, D. L., & O'Connor, S. L. (1980). Perception of

horizontal aircraft separation on a cockpit display of traffic information. Human


620.

Pape, A. M., Wiegmann, D. A., & Shappell, S. (2001). Air traffic control (ATC) related

accidents and incidents: A human factors analysis. In Proceedings of the 11th

International Symposium on Aviation Psychology, Columbus, OH, The Ohio State

University.

Parker, C. L. (1973). "General Aviation Air Traffic Pattern Safety Analysis." The

System Safety Society Symposium NASA (NASA-TM- 69455).

Pawlak, W. S., Brinton, C. R., Crouch, K., & Lancaster, K. M. (1996, July). A

framework for the evaluation of air traffic control complexity. In Proceedings of

the AIAA Guidance Navigation and Control Conference, San Diego, CA.

Posner, M. I. (1980). Orienting of attention. Q. J. Exp. Psychol. 32, 3 – 25.

Poupyrev, I., & Maruyama, S. (2003). Tactile interfaces for small touch screens. In

Proceedings of the 16th annual ACM symposium on User interface software and

technology (pp. 217-220). ACM.

Previc, F. H., & Ercoline, W. R. (1999). TheOutside-In'Attitude Display Concept

Revisited. The international Journal of aviation psychology, 9(4), 377-401.

Prinzo, O.V. (2003) “Pilot's Visual Acquisition of Traffic: Operational Communication

From an In-Flight Evaluation of a Cockpit Display of Traffic Information”

International Journal of Aviation Psychology 13(3), 211-231.

Prinzo, O. V., & Hendrix, A. M. (2006). Automatic dependent surveillance-

broadcast/cockpit display of traffic information: pilot use of the approach spacing

application. The International Journal of Aviation Psychology, 16(1), 77-96.

of 303

Prinzel, L. L. J., Kramer, L. J., Comstock, J. R., Bailey, R. E., Hughes, M. F., & Parrish,

R. V. (2002, September). NASA synthetic vision EGE flight test. In Proceedings

of the Human Factors and Ergonomics Society Annual Meeting (Vol. 46, No. 1,

pp. 135-139). SAGE Publications.

Parasuraman, R., Mouloua, M., & Molloy, R. (1996). Effects of adaptive task allocation

on monitoring of automated systems. Human Factors: The Journal of the Human

Factors and Ergonomics Society, 38(4), 665-679.

Parasuraman, R., Molloy, R. & Singh I. L. (1993). Performance consequences of

automation-induced "complacency”. The International Journal of Aviation

Psychology, 3, 1-23.

Parasuraman R. & Riley, V. A. (1997). "Human and Automation: use, Misuse, Disuse,

Abuse." Human Factors 39(2), 230-253.

Parasuraman, R., Sheridan, T. B, &. Wickens, C. D. (2000). "A model for Types and

Levels of Human Interaction with Automation." IEEE Transactions on Systems,

man, Cybernetics -Part A: Systems and Humans 30 (3).

Raj, A. K.; Kass, S. J.; Perry, J. F.(2000) Vibrotactile displays for improving spatial

awareness. Proceedings of the IEA 2000/HFES 2000 Congress, 181-184,

Rasmussen, J. (1982). Human errors. A taxonomy for describing human malfunction in

industrial installations. Journal of occupational accidents, 4(2), 311-333.

Rasmussen, J. (1983). Skills, rules, knowledge; signals, signs, and symbols, and other

distinctions in human performance models. IEEE Transactions on Systems, Man

and Cybernetics, 13, 257-266.

Rasmussen, J. (1985). The role of hierarchical knowledge representation in decision

making and system management. Systems, Man and Cybernetics, IEEE

Transactions on, (2), 234-243.

Rasmussen, J. (1986).”Information Processing and Human-Machine Interaction: An

Approach to Cognitive Engineering” Elsevier Science Inc., New York, NY, USA,

Rasmussen, J. (1990). Mental models and the control of action in complex

environments. In D. Ackermann, D. & M.J. Tauber (Eds.). Mental Models and

Human-Computer Interaction 1 (pp. 41-46). North-Holland: Elsevier Science

Publishers. ISBN 0-444-88453-X

of 303

Rasmussen, J., & Goodstein, L. P. (1985a). Decision support in supervisory control.

Risø National Laboratory.

Rasmussen, J., Pejtersen, A.M., & Goodstein, L.P. (1994). Cognitive Systems

Engineering. John Wiley & Sons Inc.

Rasmussen, J., & Vicente, K. J. (1989). Coping with human errors through system

design: implications for ecological interface design. International Journal of Man-

Machine Studies, 31(5), 517-534.

Rantanen, E. M., Yang, J., & Yin, S. (2006). Comparison of pilots' and controllers'

conflict resolution maneuver preferences. In Proceedings of the Human Factors


Publications.

Reason, J. (1990). “Human error”. Cambridge: Cambridge University Press.

Reason, J. (1997). Managing the Risks of Organizational Accidents. Ashgate

Publishing Ltd, London.

Reason, J. (2000). “Human error: models and management”. British Medical Journal

320,768-770.

Remington, R. W., Johnston, J. C., Ruthruff, E., Gould, M., & Romera, M. (2000). Visual search in complex displays: Factors affecting conflict detection by air traffic controllers. Human Factors, 42, 349-366.

Richardson., M., & Ball., L. (2009). "Internal representations, external representations,

and ergonomics: Towards a theoretical integration." Theoretical Issues in

Ergonomic Science 10(4): 335-376.

Richardson, A. E., Montello, D. R., & Hegarty, M. (1999). Spatial knowledge

acquisition from maps and from navigation in real and virtual environments.

Memory & cognition, 27(4), 741-750.

Rogers, M. L., Cook, R. I., Bower, R., Molloy, M., & Render, M. L. (2004). Barriers to

implementing wrong site surgery guidelines: a cognitive work analysis. Systems,

Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on,

34(6), 757-763.

Rome, F., Cabon, P., Favresse, A., Mollard, R., Figarol, S., & Hasquenoph, B. (2006).

Human factors issues of TCAS: A simulation study. In F. Reuzeau, K. Corker, &

G. Guy (Eds.) Proceedings of International Conference on Human-Computer

Interaction in Aeronautics (pp. 2-7). Seattle, WA.

of 303

Roscoe, A. H. (1992). Assessing pilot workload. Why measure heart rate, HRV and

respiration?. Biological psychology, 34(2), 259-287.

Rouse, W. B., & Morris, N. M. (1986). On Looking Into the Black-Box - Prospects and

Limits in the Search for Mental Models. Psychological Bulletin, 100(3), 349-

363.

Rushton, S. K., & Duke, P. A. (2009). Observers cannot accurately estimate the speed

of an approaching object in flight. Vision research, 49(15), 1919-1928.

Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols.

Memory & Cognition, 17 (6), 759-769.

RTCA (1995). “Final report of the RTCA board of directors’ select committee on free

flight”. Washington, DC: RTCA, Incorporated.

Rudisill, M. (1995). Line Pilots’ Attitudes about and Experience with Flight Deck

Automation: Results of an International Survey and Proposed Guidelines.

Proceedings of the Eighth International Symposium on Aviation Psychology.

Columbus, OH, The Ohio State University Press. (In press).

Rupert, A. H. (2000). An instrumentation solution for reducing spatial disorientation

mishaps. Engineering in Medicine and Biology Magazine, IEEE, 19(2), 71-80.

Rushby, J. (2002). Using model checking to help discover mode confusions and other

automation surprises. Reliability Engineering & System Safety, 75(2), 167-177.

Salmon, P. M., Regan, M., Lenné, M. G., & Stanton, N. A. (2007). Work domain

analysis and intelligent transport systems: implications for vehicle design.

International journal of vehicle design, 45(3), 426-448.

Salmon, P., Jenkins, D., Stanton, N., & Walker, G. (2010). Hierarchical task analysis vs.

cognitive work analysis: comparison of theory, methodology and contribution to

system design. Theoretical Issues in Ergonomics Science, 11(6), 504-531.

Sanders, M. S., & McCormick, E. J. (1987). “Human factors and Systems”. Human

factors in engineering and design. McGraw-Hill.

Sanderson, P. M., Naikar, N., Lintern, G., & Goss, S. (1999). Use of cognitive work

analysis across the system life cycle: requirements to decommissioning.

Proceedings of the 43rd Annual Meeting of the Human Factors and Ergonomics

Society, Houston, TX. Pp.318-322

of 303

Sarter, N. B. (1996). Human errors are symptoms of a mismatch between pilots,

machines and the operating environment. ICAO Journal, 51(8), 9.

Sarter, N.B., (2000) .The need for multisensory feedback in support of effective

attention allocation in highly dynamic event-driven environments: The case of

cockpit automation. International Journal of Aviation Psychology 10 (3), 231–

245.

Sarter, N.B. (2006). Multimodal Information Presentation. International Journal of

Industrial Ergonomics (Invited contribution to Special Issue on “New Insights in

Human Performance and Decision Making”), 36(5), 439-445.

Sarter, N. (2008). Investigating Mode Errors on Automated Flight Decks: Illustrating

the Problem Driven, Cumulative, and Interdisciplinary Nature of Human Factors

Research. Human Factors (Special Golden Anniversary Issue), 50(3), 506-510.

Sarter, N. B., & Alexander, H. M. (2000). Error types and related error detection

mechanisms in the aviation domain: An analysis of aviation safety reporting

system incident reports. The international journal of aviation psychology, 10(2),

189-206.

Sarter, N. B., Mumaw, R. J., & Wickens, C. D. (2007). Pilots' monitoring strategies and

performance on automated flight decks: An empirical study combining behavioral

and eye-tracking data. Human Factors: The Journal of the Human Factors and

Ergonomics Society, 49(3), 347-357.

Sarter, N. B. & Woods, D. D. (1991). Situation awareness: A critical but ill-defined

phenomenon. International Journal of Aviation Psychology, 1, 45-57.

Sarter N. B. & Woods D. D. (1992), ‘Pilot interaction with cockpit automation:

operational experiences with the flight management system’, International

Journal of Aviation Psychology, 2(4), 303-321.

Sarter, N. B. & Woods, D. D. (1995). How in the world did we ever get into that mode?

Mode error and awareness in supervisory control. Human Factors, 37, 5-19.

Sarter, N. B., Woods, D. D., & Billings, C. E. (1997). Automation surprises. Handbook

of human factors and ergonomics, 2, 1926-1943.

Scheiner, S.M (2001). MANOVA: multiple response variables and multispecies

interactions. Design and Analysis of Ecological Experiments 2nd ed. (Scheiner &

Gurevitch) Oxford Univ. Press, Oxford.

of 303

Schneider, F. E., & Wildermuth, D. (2003, October). A potential field based approach

to multi robot formation navigation. In Robotics, Intelligent Systems and Signal

Processing, 2003. Proceedings. 2003 IEEE International Conference on (Vol. 1,

pp. 680-685). IEEE.

Scholl, B.J., (2001). Objects and attention: the state of the art. Cognition 80, 1–46

Schriver, A. T., Morrow, D. G., Wickens, C. D., & Talleur, D. A. (2008). Expertise

differences in attentional strategies related to pilot decision making. Human


878.

Schuman, H., & Presser, S. (1979). The open and closed question. American

sociological review, 692-712.

Schutte, P. C., & Trujillo, A. C. (1996, October). Flight crew task management in non-

normal situations. In Proceedings of the Human Factors and Ergonomics Society

Annual Meeting (Vol. 40, No. 4, pp. 244-248). SAGE Publications.

Schuch, H.P. (1992). The influence of flight experience on midair collision risk

perception. Accident Analysis & Prevention, 24(4), 655-660

Scott, J. J., & Gray, R. (2008). A comparison of tactile, visual, and auditory warnings

for rear-end collision prevention in simulated driving. Human Factors: The


Seagull, F. J., & Gopher, D. (1994, October). Expanding the envelope of performance:

Training pilots to use helmet mounted displays. In Proceedings of the Human

Factors and Ergonomics Society Annual Meeting (Vol. 38, No. 1, pp. 11-15).

SAGE Publications.

Seagull, F. J., Xiao, Y., Mackenzie, C. F., & Wickens, C. D. (2000). Auditory alarms:

from alerting to informing. In Proceedings of the Human Factors and Ergonomics

Society Annual Meeting (Vol. 44, No. 1, pp. 223-226). SAGE Publications.

Sharma, R., Pavlovic, V. I., & Huang, T. S. (1998). Toward multimodal human-

computer interface. Proceedings of the IEEE, 86(5), 853-869.

Shelley, M. (2006). Empiricism. In F. English (Ed.), Encyclopedia of educational

leadership and administration. (pp. 338-339). Thousand Oaks, CA: SAGE

Publications, Inc.

of 303

Sheridan, T. B. (1983). Supervisory control of remote manipulators, vehicles and

dynamic processes: Experiments in command and display aiding. Massachusetts

Inst of Tech Cambridge Man-Machine Systems Lab

Sheridan, T. B. (1999). “Human supervisory control” In: Handbook of Systems

Engineering and Management (A.P. Sage & W.B. Rouse Eds.), pp. 591-628. J.

Wiley & Sons, New York, USA.

Sheridan, T. B. (2000). HCI in supervisory control: Twelve dilemmas. In Human error

and system design and management (pp. 1-12). Springer London.

Sheridan, T. B. (2008). "Risk Human Error, and System Resilience: Fundamental

Ideas." Human Factors 50(3): pp. 418-426.

Sheridan, T.B. (2009). “Human Factors Research Needs for NextGen-Airportal Safety”

NASA/CR–2009-21537.

Shepard, R. N., & Hurwitz, S. (1984). Upward direction, mental rotation, and

discrimination of left and right turns in maps. Cognition, 18(1), 161-193.

Sherry, L., & Polson, P. G. (1999). Shared models of flight management system vertical

guidance. The International Journal of Aviation Psychology, 9(2), 139-153.

Sherry, A., & Henson, R. K. (2005). Conducting and interpreting canonical correlation

analysis in personality research: A user-friendly primer. Journal of Personality

Assessment, 84(1), 37-48.

Shipp, S. (2004). The brain circuitry of attention. Trends in cognitive sciences, 8(5),

223-230.

Shneiderman, B. (1982). The future of interactive systems and the emergence of direct

manipulation. Behaviour and Information Technology , 1(3):237–256

Shomstein, S. (2012). Object‐based attention: strategy versus automaticity. Wiley

Interdisciplinary Reviews: Cognitive Science, 3(2), 163-169.

Shorrock S. T. & Isaac A. (2010) Mental Imagery in Air Traffic Control. The

International Journal of Aviation Psychology, 20(4), 309-324.

Shorrock, S. T., & Kirwan, B. (2002). Development and application of a human error

identification tool for air traffic control. Applied Ergonomics, 33(4), 319-336.

Sinreich, D., Gopher, D., Ben-Barak, S., Marmor, Y., & Lahat, R. (2005). Mental

models as a practical tool in the engineer's toolbox. International journal of

production research, 43(14), 2977-2996.

of 303

Simpson, P. A. (2001). Naturalistic decision making in aviation environments (No.

DSTO-GD-0279). Defence Science and Technology Organisation Victoria

(Australia) Aeronautical and Maritime Research Lab

Sklar, A. E., & Sarter, N. B. (1999). Good vibrations: Tactile feedback in support of

attention allocation and human-automation coordination in event-driven domains.

Human Factors: The Journal of the Human Factors and Ergonomics Society,

41(4), 543-552.

Slovic, P. (1987). Perception of risk. Science, 236(4799), 280-285.

Simon, H. A. (1975). The functional equivalence of problem solving skills. Cognitive

Psychology, 7(2), 268-288.

Simon, H. A. (1993). Decision making: Rational, nonrational, and irrational.

Educational Administration Quarterly, 29(3), 392-411.

Smallman, H. S., John, M. S., Oonk, H. M., & Cowen, M. B. (2001). Information

availability in 2D and 3D displays. IEEE Computer Graphics and Applications,

21(5), 51-57.

Smith, K. & Hancock, P.A. (1995). Situation awareness is adaptive, externally directed

consciousness. Human Factors, 37, 137-48.

Smith, C. A., Clegg, B. A., Heggestad, E. D., & Hopp-Levine, P. J. (2009). Interruption

management: A comparison of auditory and tactile cues for both alerting and

orienting. International Journal of Human-Computer Studies, 67(9), 777-786.

Snook, S. A. (2000). Friendly Fire, Princeton University Press.

Soto, D., & Blanco, M. J. (2004). Spatial attention and object-based attention: a

comparison within a single task. Vision research, 44(1), 69-81.

Staggers, N., & Norcio. A. F (1993). "Mental Models: Concepts for Human -computer

interaction Research." International Journal Man-Machine Studies 38: 587-605.

Stanard, T., Flach, J. M., Smith, M., & Warren, R. (1996). Visual information use in

collision avoidance tasks: the importance of understanding the dynamics of action.

In Human Interaction with Complex Systems, 1996. HICS'96. Proceedings., Third

Annual Symposium on (pp. 62-67). IEEE.

Stanton, N. A., Chambers, P. R. G., & Piggott, J. (2001). Situational awareness and

safety. Safety Science, 39(3), 189-204.

of 303

Stokes, A. F., Kemper, K., & Kite, K. (1997). Aeronautical decision making, cue

recognition, and expertise under time pressure In C. E. K. Zsambok, G. (Ed.),

Naturalistic decision making (pp. 183-196). Mahwah, NJ: Erlbaum.

Stokes, A. F., Kemper, K. L., & Marsh, R. (1992). Time-stressed flight decision

making: A study of expert and novice aviators (Tech. Rep. No. ARL-93-1/INEL-

93-1). Savoy, IL: University of Illinois, Aviation Research Laboratory.

Stoner, H. A., Wiese, E. E., & Lee, J. D. (2003, October). Applying ecological interface

design to the driving domain: the results of an abstraction hierarchy analysis. In

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

(Vol. 47, No. 3, pp. 444-448). SAGE Publications.

Tang, H., Denery, D., Erzberger, H., & Paielli, R. (2008). Tactical separation algorithms

and their interaction with collision avoidance systems. In AIAA Guidance,

Navigation and Control Conference and Exhibit, Honolulu, Hawaii.

Taneja, N., & Wiegmann, D. A. (2001). Analysis of mid-air collisions in civil aviation.

In Proceedings of the Human Factors and Ergonomics Society Annual Meeting

(Vol. 45, No. 2, pp. 153-156). SAGE Publications.

Taylor, H. A., & Tversky, B. (1992). Spatial mental models derived from survey and

route descriptions. Journal of Memory and language, 31(2), 261-292.

Telfer, R.A. & Ashman, A. F. (1986). Pilot Judgment Training an Australian Validation

Study. Unpublished Research Report, Univ. of Newcastle, New South Wales.

Thomas, J. C., & Kellogg, W. A. (1989) “Minimizing Ecological Gaps in Interface

Design,” IEEE Software, pp. 78-86.

Thomas L. C. & Rantanen E. M. (2006) “Human factors issues in implementation of

advanced aviation technologies: a case of false alerts and cockpit displays of

traffic information” Theoretical Issues in Ergonomics Science, 7(5), 501 – 523.

Thomas, L. C., & Wickens, C.D., (2006). “Display Dimensionality, Conflict Geometry,

and Time Pressure effects on Conflict Detection and Resolution Performance

Using a Cockpit Display of Traffic Information” The International Journal of

Aviation Psychology, 16(3), pp.315-336.

Thompson, S. D., & Sinclair, K. A. (2008). “Automatic Dependent Surveillance–

Broadcast in the Gulf of Mexico”. Lincoln Laboratory Journal, 17(2), 1-15.

of 303

Thorndyke, P. W., & Hayes-Roth, B. (1982). Differences in spatial knowledge acquired

from maps and navigation. Cognitive psychology, 14(4), 560-589.

Thornton, C., Braun, C., Bowers, C., & Morgan, B. B. (1992). Automation effects in the

cockpit: A low-fidelity investigation. Proceedings of the Human Factors Society

36th Annual Meeting: Vol. 1 (pp. 30-34).

Todd, P. M., & Gigerenzer, G. (2001). Putting naturalistic decision making into the

adaptive toolbox. Journal of Behavioral Decision Making, 14(5), 381-383.

Todd, P. M., & Gigerenzer, G. (2003). Bounding rationality to the world. Journal of

Economic Psychology, 24(2), 143-165.

Trafton, J. G., Altmann, E. M., Brock, D. P., & Mintz, F. E. (2003). Preparing to resume

an interrupted task: Effects of prospective goal encoding and retrospective

rehearsal. International Journal of Human-Computer Studies, 58(5), 583-603.

Treisman, A. (1982). Perceptual grouping and attention in visual search for features and

for objects. Journal of Experimental Psychology: Human Perception and

Performance, 8(2), 194.

Treisman, A., & Souther, J. (1985). Illusory words: The roles of attention and top-

down constraints in conjoining letters to form words. Journal of Experimental

Psychology: Human Perception and Performance, 12, 3-17.

Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention.

Cognitive psychology, 12(1), 97-136.

Tversky, B. (2001). Spatial schemas in depictions. In M. Gattis (Ed.), Spatial schemas

and abstract thought. Pp. 79-111. Cambridge: MIT Press.

Tversky, B, Zacks, J., Lee, P. U., & Heiser, J. (2000). Lines, blobs, crosses, and arrows:

Diagrammatic communication with schematic figures. In M. Anderson, P.

Cheng, and V. Haarslev (Editors). Theory and application of diagrams. Pp. 221-

230. Berlin: Springer

Turatto, M., Galfano, G., Gardini, S., & Mascetti, G. G. (2004). Stimulus‐driven

attentional capture: An empirical comparison of display‐size and distance

methods. Quarterly Journal of Experimental Psychology Section A, 57(2), 297-

324.

of 303

Teichner, W. H. (1972). Predicting Human Performance III: Detection of a Simple

Visual Signal as a Function of Time of Watch (No. NMSU-ONR-TR-72-1). New

Mexico State Univ Las Cruces Dept of Psychology

Uhlarik, J., & Comerford, D. A. (2002). A review of situation awareness literature

relevant to pilot surveillance functions. Kansas State Univ Manhattan Dept of

Psychology.

Valerie J. (2008). “Measures of Situation Awareness”. In Human performance,

Workload, and Situation Awareness Measures Handbook, CRC Hoboken.

Van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. (1994). The think aloud

method: A practical guide to modelling cognitive processes. London: Academic

Press.

van den Haak, M., De Jong, M., & Schellens, P. J. (2003). Retrospective vs. concurrent

think-aloud protocols: testing the usability of an online library catalogue.

Behaviour & Information Technology, 22(5), 339-351.

Van Gent R., Hoekstra, J., & Ruigrok, R. (1997), “Free-flight with Airborne Separation

Assurance,” Report of the National Aerospace Laboratory (NLR), The

Netherlands. NLR-TP-98286.

Van Dam, S. B. J. M. Mulder, & van Paassen, M. M. (2008a) “Ecological interface

design of a tactical airborne separation assistance tool,” IEEE transactions on

Systems, Man, and Cybernetics part A: System and Human, 38(6).

Van Dam, S.B.J.; Steens, C.L.A.; Mulder, M. & van Paassen, M.M (2008b),

"Evaluation of two pilot self-separation displays using conflict situation

measurements," Systems, Man and Cybernetics, 2008. SMC 2008. IEEE

International Conference on , vol., no., pp. 3558, 3563.

Van der Veer, G. C. (1989). Individual differences and the user interface. Ergonomics,

32(11), 1431-1449

van Marwijk, B. J., Borst, C., Mulder, M., Mulder, M., & van Paassen, M. M. (2011).

Supporting 4D trajectory revisions on the flight deck: Design of a human–

machine interface. The International Journal of Aviation Psychology, 21(1), 35-

61.

of 303

Verma, S., Corker, K., & Jadhav, A. (2004). Modeling Visual Behavior of Pilots in the

Human Performance Model-Air MIDAS. In In Proceedings of the Sixth

International Conference Cognitive Modeling Mahwah.NJ:Lawrance Earbaum.

Vernon, D., Reising, C., & Sanderson, P. M. (2002). Ecological interface design for

Pasteurizer II: A process description of semantic mapping. Human Factors: The


Verrillo, R. T. (1993). “The effects of aging on the sense of touch” In: Verrillo, R.T.

(ed.): Sensory research: Multimodal perspectives. Lawrence Erlbaum Associates,

Inc., New Jersey, 285-298.

Ververs, P. M., & Wickens, C. D. (1998). Head-up displays: Effect of clutter, display

intensity, and display location on pilot performance. The International Journal of


Vicente, K. J. (1999a). Cognitive Work Analysis: Toward Safe, Productive, and Healthy

Computer-Based Work. Mahwah, NJ: Erlbaum and Associates. ISBN 0-8058-

2397-2

Vicente, K. J. (1999b). Ecological Interface Design: Supporting operator adaptation,

continuous learning, distributed, collaborative work. Proceedings of the Human

Centered Processes Conference, 93-97

Vicente, K. J. (2002). Ecological interface design: Progress and challenges. Human

Factors: The Journal of the Human Factors and Ergonomics Society, 44(1), 62-78.

Vicente, K. J. & Rasmussen, J. (1992). “Ecological Interface Design: Theoretical

foundations” IEEE Transactions on Systems, Man and Cybernetics, 22, 589-606.

Vortac, O. U., Edwards, M. B., & Manning, C. A. (1994). Sequences of actions for

individual and teams of air traffic controllers. Human–Computer Interaction, 9(3-

4), 319-343.

Vosgerau, G. (2003). The Spatial Nature of Mental Models. Spatial and Visual

Components in Mental Reasoning about Large-Scale Spaces, 17.

Warm, J. S., Dember, W. N., & Hancock, P. A. (1996) “Vigilance and workload in

Automated Systems” In: R. Parasumman and M. Mouloua (Eds). Automation and

human performance: Theory and applications (pp. 183-200). Erthaum, Hillsdale,

N.J.

of 303

Warm, J. S., & Dember, W. N. (1998). Tests of vigilance taxonomy. In: R.R. Hoffman,

M.F. Sherrick, and J. S. Warm (Eds.). Viewing psychology as a whole: The

integrative science of William N. Dember. (pp. 87-112). Washington, DC:

American Psychological Association.

Watson, M. O., & Sanderson, P. M. (2007). Designing for attention with sound:

Challenges and extensions to ecological interface design. Human Factors: The


Wickens, C.D. (1984). "Processing resources in attention", in R. Parasuraman & D.R.

Davies (Eds.), Varieties of attention, (pp. 63–102). New York: Academic Press.

Wickens, C.D. (1996) Situation awareness: impact of automation and display

technology. In Situation awareness: limitations and enhancement in the aviation

environment. AGARD Conference proceedings CP-575. Neuilly-sur-Seine,

France.

Wickens C.D (2002a). “Situation Awareness and Workload in Aviation” Current

Directions in Psychological Science. 11(4) 128-133.

Wickens, C.D (2002b), “Multiple resources and performance prediction”, Theoretical

Issues in Ergonomics Science, 3(2) pp. 150-177.

Wickens, C. D. (2008). Multiple resources and mental workload. Human Factors: The


Wickens, C. D. (2009). ISAP The keynote address: the psychology of aviation surprise:

An 8 Year Update Regarding The Noticing of Black Swans.

Wickens, C. D., Gempler, K. S., & Morphew, E. (2000). Workload and reliability of

predictor displays in aircraft traffic avoidance. Transportation Human Factors

Journal, 2, 99–126.

Wickens, C. D. & Hollands, J. G. (2000). Engineering Psychology and Human

Performance (3rd ed.). Upper Saddle River, NJ: Prentice Hall. ISBN 0-321-

04711-7

Wickens, C. D., & Kessel, C. (1980). Processing resource demands of failure detection

in dynamic systems. Journal of Experimental Psychology: Human Perception and

Performance, 6(3), 564.

of 303

Wickens, C.D., & Prevett, T.T. (1995). Exploring the dimensions of egocentricity in

aircraft navigation displays. Journal of Experimental Psychology: Applied, 1,110–

135

Wickens, D., C., Mavor A. S., & McGee, J., P., (1997). Flight to the further: Human

Factors in Aircraft Traffic Control. National Academy of Sciences. N. R. C. Panel

on Human Factors in Air Traffic Control Automation, National Academies Press.

Wickens, C. D., Xu, X., Helleberg, J. R., Carbonari, R., & Marsh, R. (2000). The

allocation of visual attention for aircraft traffic monitoring and avoidance:

Baseline measures and implications for freeflight (Technical Report ARL–00-

2/FAA- 00-2).

Wickens, C. D., Hellenberg, J., & Xu, X. (2002). Pilot maneuver choice and workload

in free flight. Human Factors: The Journal of the Human Factors and

Ergonomics Society, 44 (2), 171-188.

Wickens, C.D., Lee, J.D., Liu, Y. & Gordon Becker, S.E. (2004). An Introduction to

Human Factors in Engineering, 2nd edition. Upper Saddle River, NJ: Pearson

Prentice Hall.

Wise, M. N. (1979). The mutual embrace of electricity and magnetism. Science, 203(30), 1310-1318.

Whetten, D. A. (1989). What constitutes a theoretical contribution?. Academy of

management review, 14(4), 490-495.

Whittingham, R. (2004). The blame machine: Why human error causes accidents.

Routledge.

Wiggins, M. W. (1997). Expertise and cognitive skills development for ab-initio pilots.

Aviation training: Learners, instruction and organization(A 98-24905 05-53),

Aldershot, United Kingdom, Avebury Aviation, 1997,, 54-66.

Wiggins, M., & O'Hare, D. (1995). Expertise in aeronautical weather-related decision

making: A cross-sectional analysis of general aviation pilots. Journal of

Experimental Psychology: Applied, 1(4), 305.

Wiggins, M., Stevens, C., Howard, A., Henley, I., & O'Hare, D. (2002). Expert,

intermediate and novice performance during simulated pre-flight decision-making.

Australian Journal of Psychology, 54(3), 162-167.

of 303

Williams, E. (2004). “Airborne collision avoidance system”. In ACM International

Conference Proceeding Series (Vol. 110, pp. 97-110).

Wilson, G. F. (2002). An analysis of mental workload in pilots during flight using

multiple psychophysiological measures. The International Journal of Aviation

Psychology, 12(1), 3-18.

Wilson, M., Shepherd, I., Kelly, C., & Pitzner, J. (2005). Assessment of a low-fidelity

human patient simulator for the acquisition of nursing skills. Nurse Education

Today, 25(1), 56-67.

Wilson, J. R., & Rutherford, A. (1989). Mental models: Theory and application in

human factors. Human Factors: The Journal of the Human Factors and

Ergonomics Society, 31(6), 617-634.

Woods, D. D. (1995). The alarm problem and directed attention in dynamic fault

management. Ergonomics, 38(11), 2371-2393.

Woods, D. D., Patterson, E. S., & Roth, E. M. (2002). Can we ever escape from data

overload? A cognitive systems diagnosis. Cognition, Technology & Work, 4(1),

22-36.

Wright, P., Pocock, S., & Fields, B. (1998). The prescription and practice of work on

the flight deck. In ECCE9, Ninth European Conference on Cognitive Ergonomics

(pp. 37-42).

Xie, B. & Salvendy, G. (2000), “Prediction of Mental Workload in Single and Multiple

Task Environments”, International Journal of Cognitive Ergonomics, 4, 213–242

Xiao, Y., & Seagull, F. J. (1999). An analysis of problems with auditory alarms:

Defining the roles of alarms in process monitoring tasks. In Proceedings of the

Human Factors and Ergonomics Society Annual Meeting (Vol. 43, No. 3, pp. 256-

260). SAGE Publications

Xu, X., & Rantanen, E. M. (2007). Effects of air traffic geometry on pilots' conflict

detection with cockpit display of traffic information. Human Factors: The Journal

of the Human Factors and Ergonomics Society, 49(3), 358-375.

Yantis, S. (1993). Stimulus driven attentional capture. Current Directions in

Psychological Science, 2, 156-161.

of 303

Yantis, S. (2000) Goal-directed and stimulus-driven determinants of attentional control.

In: Attention and performance XVIII (Monsell S, Driver J, eds), pp. 73–103.

Cambridge, MA: MIT Press.

Yantis S & Serences J. T (2003) Cortical mechanisms of space-based and object-based

attentional control. Curr. Opin. Biol. 13:187-193.

Yarbus, A .L. (1967) Eye-Movements and Vision. New York: Plenum Press

Yeh, M., & Wickens, C. D. (2001). Attentional filtering in the design of electronic map

displays: A comparison of color coding, intensity coding, and decluttering

techniques. Human Factors: The Journal of the Human Factors and Ergonomics

Society, 43(4), 543-562.

Young, R. M. (1983). “Surrogates and mappings: Two kinds of conceptual models for

interactive devices”. Mental Models (pp.25-52). Hillsdale, NJ, Erlbaum.

Young, M. S. & Stanton, N. A. (2001). Mental workload: theory, measurement, and

application. In W. Karwowski (Ed.), International encyclopedia of ergonomics

and human factors: Volume 1 (pp. 507-509). London: Taylor & Francis.

Young, S. M., & Stanton, N.A. (2002). "Malleable Attentional Resources Theory: A

New Explanation for the Effects of Mental Under load." Human Factors 44(3):

365-375.

Zalesky, A., Fornito, A., & Bullmore, E. (2012). On the use of correlation as a measure

of network connectivity. Neuroimage, 60(4), 2096-2106.

Zhang, Y. (2008). Undergraduate students' mental models of the Web as an information

retrieval system. Journal of the American Society for Information Science and

Technology, 59(13), 2087-2098.

Zheng, B., Jiang, X., Tien, G., Meneghetti, A., Panton, O. N. M., & Atkins, M. S.

(2012). Workload assessment of surgeons: correlation between NASA TLX and

blinks. Surgical endoscopy, 26(10), 2746-2750.

Zingale, D. & Williams, B. (2009) “Review of Aircraft Self-Spacing Concepts:

Implications for Controller Display Requirements” Federal Aviation

Administration. DOT/FAA/TC-TN-09/03.

Zsambok, C.E & Klein, G. (1997). Naturalistic Decision Making. Mahwah, New Jersey,

USA: Erlbaum.

of 303

Appendices

Appendix A: An email of approval from Swinburne’s Human Research Ethics Committee (SUHREC) at Swinburne University of Technology

Appendix B: Participant’s Flight Experience

Appendix C: Participant Consent Form

Appendix D: General Information for the Participants

Appendix E: Post-Trial Questionnaire

Appendix F: NASA-TLX rating scale

of 303

APPENDIX A

of 303

APPENDIX B

of 303

APPENDIX C

of 303

APPENDIX D

General Information for the Participants

Thank you for coming to take part and helping us with this research.

The aim of the research is the evaluation of a system that may help pilots to avoid

collision when in a Free-flight Environment. In this study, we hope to identify any

problems that pilots have using the system.

Should you agree to take part in this study, you will be asked to undertake navigational

tasks that may involve conflict detection and resolution tasks.

You will be asked to work through these tasks using a system that assists pilots

operating within a free-flight environment.

During the study we will ask you to fill out a Flight Experience form and Post-Trial

Questionnaire. The Flight Experience Form is to collect information regarding your

flying experiences .This will help us interpret the data we collect. The Post-Trial

Questionnaire is to ask you for your opinion about the system you just tried. This will

help us understand the applicability of the system.

We anticipate that the each experimental run will take approximately 15 minutes to

complete.

With your permission, we will make audio and video recordings of your session. We

will also capture your interactions with the interface. We will also record your face on a

web cam. This will help us see how you are interacting with the system. If you do not

wish to have your face recorded, please let the team know and they will turn this feature

off. The data recorded during your session will only be used to prepare a report on the

usability of the interface. You will not be personally identified in any of the data

reported.

of 303

You are free to end your session at any time without giving a reason, or you may tell us

your reasons if you wish.

We ask that you next read and sign our consent form.

If you have any questions about this document or the consent form, please do not

hesitate to ask them.

Again, we thank you for helping us in this project.

If you have any concerns please contact:

Supervisor: Dr Peter Higgins

Faculty of Engineering and Industrial Sciences


PO Box 218, Hawthorn, Victoria 3122

(Tel.) (3) 9214 8029 Email: [email protected]

Please feel free to keep this sheet for your own reference.

Investigator: Yakubu Ibrahim

Faculty of Engineering and Industrial Sciences


PO Box 218, Hawthorn, Victoria 3122

(Tel.) 9214 8160 Email: yibrahim@ swin.edu.au

Please feel free to keep this sheet for your own reference.

of 303

APPENDIX E

POST-TRIAL QUESTIONNAIRE

Please rate your opinion on the system by placing an “X” at the desired point on the scale and elaborate accordingly.

A Discrete Flying Task

In this section, we are asking you to rate your opinion on the system for tracking activities. .

1. The response of the yoke and throttle was too sensitive for me to track the path I

wanted to follow.

Fully disagree Fully agree

Suggestions or comments on this aspect

of 303

Traffic Information

In this section, we are asking you to rate your opinion on the system for traffic information.

2. How useful was the system for understanding the location of the Intruder

relative to the Ownship?

Extremely unuseful Extremely useful


3. How useful was the system for understanding the direction of travel of the

Intruder?



of 303

4. How useful was the system for avoiding conflict?



Stalling Speed

In this section, we are asking you to rate your opinion on the system for stalling speed information by placing an “X” at the desired point on the scale and elaborate accordingly.

5. How useful was the system for avoiding stall when manoeuvring the aircraft?



of 303

Representing Wind as a Vector

In this section we are asking you to rate your opinion on the system for wind as a vector information.

6. How useful was the system for providing sufficient information for following a

desired flight path?



Overall opinion

7. Please rate your overall opinion of the system placing an “X” at the desired

point on the scale below.



of 303

APPENDIX F

Rating Sheet - Pilots’ Workload / NASA Task Load Index

Please place an “X” on each scale at the point that best indicates your experience with the system.

MENTAL DEMAND

How much mental and perceptual activity was required (e.g., thinking, deciding,

calculating, remembering, looking, searching, etc.)? Was the task easy or demanding,

simple or complex, acting or forgiving?

Low High

PHYSICAL DEMAND

How much physical activity was required (e.g., pushing, pulling, turning, controlling,

activating, etc.)? Was the task easy or demanding, slow or brisk, slack or strenuous,

restful or laborious?

Low High

TEMPORAL DEMAND

How much time pressure did you feel due to the rate or pace at which the tasks or task

elements occurred? Was the pace slow and leisurely or rapid and frantic?

Low High

of 303

EFFORT

How hard did you have to work (mentally and physically) to accomplish your level of

performance?

Low High

PERFORMANCE

How successful do you think you were in accomplishing the goals of the task set by the

experimenter (or yourself)? How satisfied were you with your performance in

accomplishing these goals?

Low High

FRUSTRATION LEVEL

How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified,

content, relaxed and complacent did you feel during the task?

Low High

of 303

NASA-TLX RATING SCALE DEFINITIONS

Title Endpoints Descriptions

MENTAL DEMAND

Low /High How much mental and perceptual activity was required (e.g., thinking, deciding, calculating, remembering, looking, searching, etc.)? Was the task easy or demanding, simple or complex, exacting or forgiving?

PHYSICAL DEMAND

Low /High

How much physical activity was required (e.g.. pushing , pulling , turning , controlling, activating , etc.) ?Was the task easy or demanding, slow or brisk , slack or restful or laborious?

TEMPORAL DEMAND

Low /High

How much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred? Was the pace slow and leisurely or rapid and frantic?

PERFORMANCE

Good/Poor

How successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)? How satisfied were you with your performance in accomplishing these goals?

EFFORT Low /High How hard did you have to work (mentally and physically) to accomplish your level of performance?

FRUSTRATION LEVEL

Low /High

How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task?

of 303

Subjective Comparison of Demand Factors:

Examining and explaining the Effects of the System Features on Pilot Performance.

For each of the pairs listed below, circle the scale title that represents the more

important contributor to workload in the display.

Mental Demand or Physical Demand

Mental Demand or Temporal Demand

Mental Demand or Performance

Mental Demand or Effort

Mental Demand or Frustration

Physical Demand or Temporal Demand

Physical Demand or Performance

Physical Demand or Effort

Physical Demand or Frustration

Temporal Demand

or Performance

Temporal Demand or Frustration

Temporal Demand or Effort

Performance or Frustration

Performance or Effort

Frustration or Effort

Development and evaluation of a flight collision avoidance ...

Documents

Transcript of Development and evaluation of a flight collision avoidance ...