Interaction Design for Remote Control of Military Unmanned ...

Linköpings universitetSE–581 83 Linköping+46 13 28 10 00 , www.liu.se

Linköping University | Department of Computer and Information Science

Master’s thesis, 30 ECTS | Design and Product Development

2021 | LIU-IDA/LITH-EX-A--2021/005--SE

Interaction Design for RemoteControl of Military UnmannedGround VehiclesDiana Saleh

Supervisor : Mattias ArvolaExaminer : Stefan Holmlid

http://www.liu.se

Copyright

The publishers will keep this document online on the Internet - or its possible replacement - for aperiod of 25 years starting from the date of publication barring exceptional circumstances.The online availability of the document implies permanent permission for anyone to read, to down-load, or to print out single copies for his/hers own use and to use it unchanged for non-commercialresearch and educational purpose. Subsequent transfers of copyright cannot revoke this permission.All other uses of the document are conditional upon the consent of the copyright owner. The publisherhas taken technical and administrative measures to assure authenticity, security and accessibility.According to intellectual property law the author has the right to be mentioned when his/her work isaccessed as described above and to be protected against infringement.For additional information about the Linköping University Electronic Press and its proceduresfor publication and for assurance of document integrity, please refer to its www home page:http://www.ep.liu.se/.

© Diana Saleh

http://www.ep.liu.se/

Abstract

The fast technology development for military unmanned ground vehicles (UGVs) has ledto a considerable demand to explore the soldier’s role in an interactive UGV system. Thisthesis explores how to design interactive systems for UGVs for infantry soldiers in theSwedish Armed Force. This was done through a user-centered design approach in threesteps; (1) identifying the design drivers of the targeted military context through qualitativeobservations and user interviews, (2) using the design drivers to investigate concepts forcontrolling the UGV, and (3) create and evaluate a prototype of an interactive UGV systemdesign.

Results from interviews indicated that design drivers depend on the physical and psy-chological context of the intended soldiers. In addition, exploring the different conceptsshowed that early conceptual designs helped the user express their needs of a non-existingsystem. Furthermore, the results indicate that an interactive UGV system does not neces-sarily need to be at the highest level of autonomy in order to be useful for the soldiers on thefield. The final prototype of an interactive UGV system was evaluated using a demonstra-tion video, a Technology Acceptance Model (TAM), and semi-structured user interviews.Results from this evaluation suggested that the soldiers see the potential usefulness of aninteractive UGV system but are not entirely convinced.

In conclusion, this thesis argues that in order to design an interactive UGV system, themost critical aspect is the soldiers’ acceptance of the new system. Moreover, for soldiers toaccept the concept of military UGVs, it is necessary to understand the context of use and theneeds of the soldiers. This is done by involving the soldiers already in the conceptual designprocess and then throughout the development phases.

Acknowledgments

Firstly, I would like to thank all of my supervisors at FOI, Anna Häägg, Kristofer Bengtsson,and Björn Johnsson. It has been a pleasure to write my thesis at FOI, and I am grateful foryour genuine support. This includes everything from being actors in my demo videos andcontributing to my user testings to providing me with knowledge about the military contextand reading my reports.

Furthermore, I wish to express my sincere appreciation to my supervisor, Mattias Arvola,who has offered me invaluable guidance on the topic of Interaction Design. His persistenthelp and his words of encouragement have been of the highest value, especially when theroads got rough.

Finally, I would like to thank Stefan Holmlid, my examiner at Linköping University. Hissupport and feedback during the examiner seminars have been helpful in my work.

iv

Contents

Abstract iii

Acknowledgments iv

Contents v

List of Figures vii

List of Tables viii

Terminology and Abbreviations 1

1 Introduction 21.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Theory 42.1 Human-Robot Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Teleoperation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Unmanned Ground Vehicle Teleoperation in Complex Environments . . . . . . 62.4 Interactive System Design in Telerobotics . . . . . . . . . . . . . . . . . . . . . . 7

3 Methods 93.1 Research Through Design Case Study . . . . . . . . . . . . . . . . . . . . . . . . 93.2 User-Centered Design Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3 Exploration Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4 Concept Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.5 Prototyping Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.6 User Evaluation Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.7 Research Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Results 154.1 Exploration Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2 Concept Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.3 Prototyping Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.4 User Evaluation Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5 Discussion 285.1 Implication of the results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285.2 Limitations with the method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305.3 The work in a wider context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

v

6 Conclusion 32

Bibliography 33

vi

List of Figures

2.1 The THeMIS UGV 5th generation. By Milrem Robotics(This file is licensed under theCreative Commons Attribution-Share Alike 4.0 International license. . . . . . . . . . . . 7

3.1 The modified Research through Design used in this thesis . . . . . . . . . . . . . . 93.2 A process picture that gives an overview of the user-centered research approach. . 103.3 The participants watching the monitor of the remote control of the UAV on big

screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.4 The Design Drivers. Context, Function and Qualities . . . . . . . . . . . . . . . . . 123.5 Example of a concept generated using the design drivers . . . . . . . . . . . . . . . 12

4.1 Concept 1 with main function to guard . . . . . . . . . . . . . . . . . . . . . . . . . 174.2 Concept 2 with main function to transport . . . . . . . . . . . . . . . . . . . . . . . . 184.3 Concept 3 with main function to carry . . . . . . . . . . . . . . . . . . . . . . . . . . 194.4 Concept 4 with main function to carry . . . . . . . . . . . . . . . . . . . . . . . . . . 204.5 Concept 5 with main function to spy . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.6 Dashboard top bar design proposal 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.7 Dashboard top bar design proposal 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.8 Dashboard bottom bar design proposal 1 . . . . . . . . . . . . . . . . . . . . . . . . 224.9 Dashboard bottom bar design proposal 2 . . . . . . . . . . . . . . . . . . . . . . . . 224.10 Scenario 1: Identify subjects on map . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.11 Scenario 2: Plan a route . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.12 Dashboard design outcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.13 Latest dashboard design outcome from the Design Detailing phase . . . . . . . . . 234.14 Final prototype of UI Design for remote control . . . . . . . . . . . . . . . . . . . . . 244.15 Results from the Scenario Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

vii

List of Tables

2.1 HRI definitions based on Goodrich and Schultz definition. . . . . . . . . . . . . . . 52.2 Levels of Automation of decision and action selection . . . . . . . . . . . . . . . . . 62.3 Five design elements when developing a military UGV . . . . . . . . . . . . . . . . 7

4.1 Context Identification of Military UGVs . . . . . . . . . . . . . . . . . . . . . . . . . 154.2 Design Drivers of Military UGVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.3 PMI on Concept 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.4 PMI for Concept 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.5 PMI for Concept 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.6 PMI for Concept 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.7 PMI for Concept 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.8 The main functions of the UGV and the specified functions of the user interface. . 224.9 Desired, Interesting and Necessary aspects of the prototype. . . . . . . . . . . . . . 234.10 The average answers to the six questions that revolved around usefulness. . . . . . 254.11 Perceived Usefulness of the Average Participant . . . . . . . . . . . . . . . . . . . . 254.12 Opportunities and limitations with military UGVs . . . . . . . . . . . . . . . . . . . 274.13 Potential use cases with military UGVs . . . . . . . . . . . . . . . . . . . . . . . . . 27

viii

Terminology and Abbreviations

FOI The Swedish Defence Research Agency

HRI Human-Robot Interaction

UAV Unmanned Aerial Vehicle

UGV Unmanned Ground Vehicle

1

1 Introduction

The following chapter presents the basis of the work carried out in collaboration with FOI,the Swedish Defence Research Agency. It contains the background and motivation for theresearch study followed by the aim, the research questions, and the delimitations.

1.1 Background and motivation

Today, a soldier in the field carries about 30-40 kg of packing, which in the long run affectsthe soldiers’ endurance and performance. To make it easier for the soldiers, and preservetheir strength and energy, research is conducted on how to use Unmanned Ground Vehicles(UGVs) as a carrying aid. These UGVs for carrying, are not used in the Swedish ArmedForces today but are becoming more of interest as the technology of autonomous vehicles isdeveloping at a rapid pace.

Currently, the intended UGVs are not entirely autonomous and need some form of remotecontrol from an operator. Therefore the remote control must be designed in a way that allowsthe operator to use and maneuver the UGV under the complex conditions prevailing. Thisapplies to both the graphical interface, the selection of information that appears on the screen,and the physical design of the remote control.

The research is performed in collaboration with FOI, the Swedish Defence Research Agency,which is a government authority under the Ministry of Defence and one of Europe’s leadingresearch institutes in defense and security. In this thesis, the end-users are infantry soldiersat the Swedish Armed Forces.

1.2 Aim

The aim of this thesis is to explore the usefulness of interactive UGV systems design forinfantry soldiers on the field. The goal is to investigate, prototype and test interfaces forinteraction between humans and unmanned systems.

2

1.3. Research questions

1.3 Research questions

Based on the aim of this thesis, the main research question is: "How do we design interactiveunmanned ground vehicle systems for infantry soldiers on the field?"

In order to answer the main research question, the question is broken down into three re-search questions that further represent the phases of the project.

1. What characterizes the context of use for infantry soldiers on the field and what is im-portant to consider in the design of an interactive UGV system for this context?

2. What are the infantry soldiers’ needs in relation to an interactive UGV system on thefield?

3. How should the remote control for an interactive UGV system be designed for infantrysoldiers on the field?

1.4 Delimitations

• The end-user involvement with the Swedish Armed Forces has been limited due to theprevailing pandemic situation with Covid-19.

• The aim is not to develop a detailed user interface but rather to explore design andconcepts related to the interaction design of military UGVs.

• Cognitive workload is identified as an important design aspect to consider when de-signing for robotic control. However, this aspect is not measured in the thesis since it isconsidered to be outside the scope of the design project.

3

2 Theory

To be able to understand the field of interactive systems for unmanned ground vehicles inmilitary settings, it is essential to get a broader perspective of human-robot interaction. Thischapter provides a theoretical framework starting from the broad perspective of Human-RobotInteraction to further narrow the scope to Teleoperation in Complex Environments and later diveinto Interactive System Design in Telerobotics.

2.1 Human-Robot Interaction

Human-Robot Interaction (HRI) is described as the study of communication between humansand robotic systems [7]. The field of robotic systems is currently a wide-ranging researchand design activity that, during the last decade, has expanded at a fast pace and in variousfields. This has lead to that there are several approaches to defining the interaction betweenrobot and human. Because the field of robotic systems is wide, these approaches tend to beexplaining specific cases instead of providing a general established HRI framework [44] [43][26].

Scholtz [35], for example, defined HRI from the perspective of the human role. He has identi-fied five roles when interacting with a robot that is: supervisor, operator, mechanic, peer, andbystander.

In contrast, Granda et al. [12] defined HRI focusing around the "robot-interaction" part. Theydefined HRI as five overlapping stages of capability: Bounded Autonomy, Teleoperation, Su-pervised Autonomy, Adaptive Autonomy and Virtual Symbiosis. These stages are not mutuallyexclusive. In the stage of Bounded Autonomy, robotic devices operate with little human in-terference. These robotic devices are usually used in repetitive and pre-programmed duties,such as in factory systems. The fundamental part of the Teleoperation stage is that the oper-ator is responsible for all active information processing. This stage is commonly performedin dangerous situations where the operator can control the robot from a distance. SupervisedAutonomy emphasizes a variable span of autonomous control where the robot can handle par-ticular tasks without an operator. However, when needed, the human operator can intervenewith the robot in real-time. Adaptive Autonomy is characterized by the robot’s capability to

4

2.2. Teleoperation

learn from experience. In this stage, the role of the human is safety monitoring and makingsure what is learned is not harmful. In the final stage, there is Virtual Symbiosis. In this stage,the human and the robot have reached a team relationship that allows them to communicateon a high cognitive level.

Furthermore, Goodrich and Schultz have a definition that focuses on the interaction partbetween the human and the robot. Goodrich and Schultz define interaction as a form of com-munication that varies depending on the distance between the human and the robot [11].Thus interaction is divided into the interaction categories "remote" and "proximate". Remoteinteraction is referring to the fact that the human and the robot are separated spatially andsometimes even temporally (e.g., Mars Rovers that are controlled on earth and hence sepa-rated both in space and in time). Whereby proximate interaction refers to that the humans andthe robots are co-located ( e.g., service/assistant robots could be in the same room as humans)[11].

Goodrich and Schultz further mention that within the two subcategories, remote and proxi-mate interaction, it is useful to distinguish between interactions that require mobility, physi-cal manipulation, or social interaction [11]. Following terms illustrated in Table 2.1 are gener-ated using this explanation: (1) Teleoperation: Remote interaction with mobile robots, (2) Tele-manipulation: Remote interaction with a physical manipulator (3) Robot assistant: Proximateinteraction with mobile robots. This type of interaction may include a physical interaction (4)Peers/companions: Social interaction generally appears to be proximate rather than remote andincludes the cognitive aspects of interaction. In this type of interaction, humans and robotsinteract as peers or companions. These terms are illustrated in Table 2.1 below.

Table 2.1: HRI definitions based on Goodrich and Schultz definition.

HRIRemote Proximate

Mobility Teleoperation Robot assistant:Physical manipulation Telemanipulation Physical robot assistantSocial interaction Telecommunication Peers/companions

2.2 Teleoperation

As previously mentioned teleoperation is considered to be the remote interaction with mo-bile robots. However, it is argued that teleportation itself also could be further divided intosubgroups depending on the degree of interaction. In fact, Milgram et al. introduce teleoper-ation from a director/agent control framework where the director is the human operator andthe agent is the robot [20]. According to this definition, automation means the entire or thepartial replacement of a function previously carried out by a human operator.

Similarly, Parasuraman et al. entail that automation is not all or nothing but a substancethat can vary across a continuum of levels, from the lowest level to the highest level of fullautomation [29]. This definition is illustrated in Table 2.2 below.

5

2.3. Unmanned Ground Vehicle Teleoperation in Complex Environments

Table 2.2: Levels of Automation of decision and action selection

Levels of AutonomyHIGH 10. The computer decides everything, acts autonomously

9. informs the human only if, it, the computer decides to8. informs the human only if asked, or7. executes automatically, then necessarily informs the human, and6. allows the human a restricted time to veto before automatic execution, or5. executes that suggestion if the human approves, or4. suggests one alternative3. narrows the selection down to a few, or2. The computer offers a complete set of decision/action alternatives, or

LOW 1. The computer offers no assistance: human must take all decisions and actions.

The lowest level of autonomy depicted in Table 2.2 shows a state of manual teleoperationwhile the highest level of autonomy shows a form of complete autonomous robotics. Con-sidering these levels of autonomy, Tseng and Mettler further discuss that the human operatorcan switch between modes of different levels of autonomy depending on the task the robotis performing [37]. In the task of searching, the operator could, for example, switch betweenmanually controlling the robot or order the robot to do a pre-planned systematic scan of asubarea [21].

2.3 Unmanned Ground Vehicle Teleoperation in Complex Environments

A UGV is referring to a vehicle that can operate without the presence of an on-board humandriver [42]. In popular speech, UGVs are usually referred to as autonomous vehicles. As oftoday, the development of vehicles with a certain degree of autonomy has become a matter ofcourse for major vehicle manufacturers and research centres [4]. The research outcome showsthat the industry of automotive has opted for the self-driving technology from a civil pointof view and has become a reality in simple environments [45]. However, complex scenariossuch as the military sector still have a long way to go [41] [19]. Simple environments refer toa structured environment with features such as streets, signs, traffic lights, road marks and anetwork of communication [16]. Whereby, a complex environment refers to an unstructuredenvironment that is regularly off-road with elements of assorted obstacles [22].

When developing an autonomous vehicle, following design elements have been identified:perception, decision-making and action [19]. Out of these elements, research has shown a greaterfocus towards exploring perception and decision-making since they are considered to be morecomplex in comparison to action [19]. Perception refers to the perception of the vehicle such asits sensor technology that allows for localisation and notification. Decision-making refers to thetasks related to making decisions when the vehicle is navigating, driving and maneuvering[19].

In addition to the design elements for autonomous vehicles, developing a military UGV re-quires a broader scope of elements. In this case, five main elements have been identified.These elements are considered to be the platform model, the control/monitoring station, the sen-sors, the environment and the missions [22]. These five elements are depicted in Table 2.3. Asseen in the table, The platform model refers to how the UGV is designed to be transported.The aspect could take advantage of a taxonomy for military UGV based on size. The sen-sors refer to the sight and perception of the military UGV such as computer vision systemsand global positioning systems. This is used to identify for example obstacles. The task of thecontrol/monitoring station is to manage the needed information by remotely sending/receivingcommands to the UGV. Furthermore, environment and missions refer to the physical conditionsand the tasks the UGV is performing.

6

2.4. Interactive System Design in Telerobotics

Table 2.3: Five design elements when developing a military UGV

Platform model

How the UGV is designed to be transported.Light-sized - transported by personnel or in a backpack.Mid-sized - transported to the desired location by another largervehicle.Heavy-sized - classified as ordinary vehicles that can beteleoperated.

Control/monitoringstation Information managing by remotely sending/receiving com-

mands to the UGV.The sensors Sight/perception of the UGV. E.g. computer vision systems and

global positioning systems.The environment The physical conditionsThe missions The task of the UGV. E.g. navigating

Looking at the usage of UGVs it is clear that the unmanned ground platforms, of varioussizes and functionalities, are growing rapidly where mainly countries such as the US, GreatBritain, Russia, and China are investing considerable resources [30]. However, the UGV thatis relevant in this thesis is the mid-sized UGV model THeMIS illustrated in Figure 2.1. Cur-rently, several countries including the United States (Marine Corps), France, Norway and theNetherlands, are conducting pilot activities with THeMIS, mainly for logistics and intelligence,surveillance, and reconnaissance [30].

Figure 2.1: The THeMIS UGV 5th generation. By Milrem Robotics(This file is licensed under theCreative Commons Attribution-Share Alike 4.0 International license.

2.4 Interactive System Design in Telerobotics

An interactive system is defined as a combination of systems that users interact with in orderto achieve a specific goal and the components of an interactive system is called the user inter-face [36]. The user interface serve as a form of interaction between two systems that involvesinputs and outputs where at least one of the systems is a human user [5]. When designinga user interface, it is essential to remember that the user interface is equally important in alllevels of autonomy. Even if the robot is capable of operating without any assistance from an

7

2.4. Interactive System Design in Telerobotics

operator (the user), it still needs to communicate to the operator how and what it is perform-ing during a task. This is of particular interest when the robot fails to complete a mission orencounters problems during the task. Hence, as robots become independent, interfaces areused less for control, and more for monitoring and diagnosis [10].

Whether the user interface is used to monitor or control the task, it still has a fundamentaltask of providing the human operator with good tactical system awareness in order to allow formission effectiveness. This is especially important in emergency situations that include uncer-tainty factors that can influence the level of a disaster [31]. The role of the human operator is,therefore, (1) to make sense of the information provided, (2) share a collective understandingof the reality, and (3) in the same time evaluate the consequences of the event using all avail-able resources [31]. Therefore, the design of the interactive system automatically becomes acrucial aspect due to its strength to influence situation awareness and performance [9].

For many years, the history of interactive systems for control of ground vehicles used in themilitary has been permeated by laptop-based-systems. Up to this date, there are only a fewexamples of handheld interactive systems and even fewer interactive systems that have ad-dressed to be portable [40]. The area of interactive UGV systems that are addressed to beportable is thereby in its precursor. These first-of-a-kind systems are argued to benefit fromconceptual design processes by enabling the users to envision the system [17]. Moreover, itis mentioned that more empirical research is needed to design user interfaces for robotic sys-tems to investigate and explore design features’ effects [27]. It is emphasized that achievingsuccessful HRI requires a focus on cooperation acceptance and cognitive workload as well as cov-ering performance and satisfaction. Designers of robots should carefully choose design featuresto balance specific effects and implementation costs concerning tasks, work design aims, andemployee needs in the specific work context [27].

In summary, there is a need for a more user-centered approach when designing robots inorder to find a balance of design features in regards to user needs, work design aims, and thespecific context of the environment. When designing for robotic control following aspects areimportant to take into consideration: (1) the level of control sharing between human-agents(2) human factors and cognitive workload (3) the specific work context.

8

3 Methods

In order to understand and explore the area of interactive systems design for remote controlof military UGVs, a user-centered approach was carried through with the methodology of aResearch Through Design Case Study. The design process included four phases: Exploration,Concept Generation, Prototyping, and User Evaluation. This chapter provides a methodolog-ical background of the activities performed in this thesis. The results of those activities arepresented in the succeeding chapter.

3.1 Research Through Design Case Study

The research in this study was inspired by a Research through Design approach which meansthat the research is developed through design activities [13]. The design activities in thisresearch was performed as a user-centered case study [34] that is illustrated in Figure 3.1.The figure shows a simplified research through design process in relation to the phases inthis specific research case study.

Phases in this research study

Background research New information, new/modified theoryObject is created Object, process is

analysed

Exploration

User Evaluation

Research through Design

Concept Generation

Prototyping

Figure 3.1: The modified Research through Design used in this thesis

9

3.2. User-Centered Design Approach

3.2 User-Centered Design Approach

The case study followed a user-centered design, which is a design approach that aims toimprove the understanding of user needs and the iteration of the design with the aid of ac-tive user involvement [18]. The user-centered design approach was valuable as the field ofremote-controlling unmanned ground vehicles in military environments is relatively new.This means that a purely theoretical research study would have led to limited results. Anoverall view of the design process can be shown in Figure 3.2 below.

Exploration Concept Generation Prototype User Evaluation

"What characterizesthe context of use for infantry soldiers on the field and what is important to consider in thedesign of an interac-tive UGV system for this context?".

"What are the infan-try soldiers’needs in relation to an interactive UGV system on the field?"

"How should the remotecontrol for an inter-active UGV system be designed for infantry soldiers on the field?"

ResearchQuestion

- Qualitative Context observation

- Explorative Inter-views

- Concept Genera-tion with Design Drivers

- Itemised Response and PMI

- Low-fidelity Proto-typing

- Prototype Evalua-tion

- Demonstration Video

- Technology Accept-ance Model

- Task-based Testing

How?

To understand the user context.

To understand the user needs and hence the Design Drivers in an interac-tive UGV system.

To create a tangible prototype for user testings.

To test the concept and the acceptance of an interactive UGV system

Why?

Figure 3.2: A process picture that gives an overview of the user-centered research approach.

3.3 Exploration Phase

The aim of the Exploration Phase was to answer the research question: "What characterizesthe context of use for infantry soldiers on the field and what is important to consider in thedesign of an interactive UGV system for this context?". The following subchapter explainsthe methods used in the Exploration Phase to answer the question. The methods used in thephase include Qualitative Context Observation and Explorative Interviews.

3.3.1 Qualitative Context Observation:

The activity of taking field notes on the behavior and actions of individuals is referred to asa qualitative observation [6]. Early in the design process, the report author participated as apassive observant in testing for Unmanned Aerial Vehicles (UAV). UAVs are more developedand frequently used than UGVs, and are thereby suitable objects for observation to learnabout participant response towards unmanned systems.

In the testing, a user scenario was conducted with UAVs to demonstrate the concept of UAVsin the military context as well as identify the desired qualities of the user interface and func-

10

3.3. Exploration Phase

tions of the UAVs. Therefore, the testing aimed to understand the context environment, in-teract with the users, and get an insight into functions required and wanted when designingan interactive system of a UGV. All of the gained insights were noted down and later tran-scribed. The testing set involved a user scenario of finding enemy soldiers using a UAV. Landmines were set up on the field, and two people in the test group role-played as enemy soldiershiding in the ambush. The UAV monitor was connected to a large display (shown in Figure3.3) to demonstrate UAVs’ usability when searching for land mines and hidden enemy sol-diers. The test participants consisted of two focus groups with soldiers from the home guard,10-15 soldiers participated in each focus group.

Figure 3.3: The participants watching the monitor of the remote control of the UAV on bigscreen

3.3.2 Explorative Interviews:

The main goal of the explorative interview [32] was to learn about the end-users. Semi-formalinterviews were carried through with two soldiers from the home guard, one paramedic andone group leader.

The questions asked focused on:

• Introduction and military context (profession, education, experiences)

• User story (Experiences, problems, highlights)

• Futuristic (If you had a magical helper what would it do?)

• Autonomous vehicles

The data obtained by the Qualitative Context Observation and the Explorative Interviewswith the soldiers from the home guard gave valuable insights that were analysed and con-secutively clustered around the themes Context, Functions and Qualities.

11

3.4. Concept Phase

3.4 Concept Phase

The Concept Phase aimed to answer the research question: "What are the infantry soldiers’needs in relation to an interactive UGV system on the field?". The following subchapterexplains the methods used to answer the question. The methods used was Concept Generationwith Design Drivers and Itemised Response and PMI.

3.4.1 Concept Generation with Design Drivers

As this is an iterative process, the outcome from the Exploration Phase, the design driversillustrated in Figure 3.4 below, were used as a basis in the concept generation. This could bereferred to as the first step in reversed engineering [28].

In this type of reversed engineering, the product was seen as a “black box” and the designdrivers were seen as the input while the output was the generated holistic concepts. An ex-ample of the holistic concepts generated from chosen design drivers is illustrated in Figure3.5 below. The intention, of the specific method usage, was to understand the overall productfunction in the comprehensive view and identify different types of concepts in an ‘unbiased’perception of possible product evolution. The design drivers were chosen randomly to opti-mize creativity.

Figure 3.4: The Design Drivers. Con-text, Function and Qualities

Main function

Adaptable

Interpret instructions

CarrySimple

Perceive as if it is in the background

Figure 3.5: Example of a concept generated us-ing the design drivers

3.4.2 Itemised Response and PMI

The Itemised Response and PMI [38] was used to systematically examine positive, negativeand interesting aspects of each holistic concept. During a workshop session, the method wascarried through with two of the client representatives from the Swedish Defence ResearchAgency. The client representatives got the task to list each concept’s positive and negativefeatures. This method was used to overview the most promising directions to develop fur-ther. The importance of client involvement was to identify interesting design fields and betterunderstand the solution space. After the workshop with the client representatives, the char-acteristics of the most promising concepts were broken down and the PMI concept screeningwas analyzed.

3.5 Prototyping Phase

The Prototyping Phase aimed to answer the research question: "How should the remotecontrol for an interactive UGV system be designed for infantry soldiers on the field?". It

12

3.6. User Evaluation Phase

further explains the methods for Low-fidelity Prototyping and the methods used in PrototypeEvaluation.

3.5.1 Low-fidelity Prototyping

Low-fidelity prototypes were created iteratively, by first using paper prototypes and finallyusing Figma as an interface design tool. A low-fidelity prototype is a prototype version that iscomplete enough to demonstrate the value it brings to the users. It contains the "bare bones"set of features to test with users allowing for people-driven innovation [39].

The prototype was iterated continuously by informal guerilla testings. The guerilla testingmethod aims to gather user input by taking a low-fidelity prototype into the public domainand ask for feedback. The method’s simplicity allows ideas to be tested quickly and at a low-cost [3]. In this case, the guerilla testing was performed with the Swedish Defence ResearchAgency employees.

3.5.2 Prototype Evaluation

After the prototype iterations, a semi-formal user testing was planned to evaluate the wire-frame prototype’s functionality. In the user test, five participants were individually asked toperform specific tasks using the system. The participants were researchers at the SwedishDefense Research Agency with a previous background in The Swedish Armed Forces. Theprototype testing involved the following three tasks: (1) Plan a route to a given point, (2)Plan a reroute passing the given point (3) Evaluate the user interface. While performingthe tasks and integrating with the prototype, the participants were continuously asked tothink aloud. This method is called Thinking out Aloud and it means to ask the participants toverbalize their thoughts as they move through the user interface [14] [25]. The numbers ofparticipants needed in Thinking out Aloud tests have been discussed throughout time andNielsen suggests that five test subjects are enough to find the majority of problems [23].

Following the task-based test, a post-session interview based on Nielsen’s ten heuristics [24]was conducted with each participant. The information gained from the method was used inorder to guide the future design and a final design was formed.

Effectiveness, efficiency, and satisfaction are recommended parameters in usability metricsaccording to the ISO/IEC 9126-4 [1] and the post-session interview was used as a tool toevaluate users’ subjective satisfaction with the aspects of the human/computer interface. Incomparison, the task-based activity was used to evaluate effectiveness and efficiency.

3.6 User Evaluation Phase

The User Evaluation Phase aimed to test the final design concept of the remote control withthe end-users. This was done through two studies. Study 1 focused on the concept placed ina scenario, while Study 2 focused on the design of the user interface.

3.6.1 Study 1: Concept

To test the usefulness and the area of use, a demonstration video of the final design wascreated. This demonstration video showed three driving control modes of the remote control.The user testing consisted of 20 participants, aged 20-29, that work at The Swedish ArmedForces. The test was done remotely and started by showing the participants the video of theconcept that demonstrated the control of the UGV. The video showed three control modes.One mode when the UGV is at its highest level of autonomy. Another mode when the controlof the UGV is semi-autonomous meaning that the UGV is following the pilot. Finally, the

13

3.7. Research Ethics

third mode of control, that is manually manoeuvring the UGV. After this concept video wasshown, participants were asked to fill out a questionnaire.

Six questions, see Table 4.10, were asked to measure the perceived usefulness of the system.These questions were taken from the Technology Acceptance Model (TAM) [8].The specificTAM used was a modified TAM by Lewis. In this modified TAM (mTAM)[15], the partici-pants were asked to provide their level of agreement on a 7 point scale (1=Completely dis-agree and 7 = Completely agree). According to Davis [8], usefulness is 1.5 times more impor-tant than ease of use since it has a bigger impact on the users’ attitude towards the system.The TAM questions were followed by three questions regarding three scenarios and how theparticipant would use the UGV in each case. The three scenarios were: Combat, Reconnoitreand Logistics and the control modes to choose between were the same as the ones showed inthe demonstration video. Possibilities for free text answers were added to each question inthe survey to provide the participant with the opportunity to explain their answers.

3.6.2 Study 2: User Interface

Five participants out of the 20 participants from Study 1 took part in Study 2. During Study 2the participants were asked to perform a simple route-planning task using the remote control.This was done by sharing the screen of the wire-frame prototype made in Figma with theparticipant. The participant was asked to verbalise the thinking process during the task. Thistask was followed by a semi-structured interview. The semi-structured interview revolvedaround the application of the system and the areas of use of a UGV in a military context. Theinterviews were done remotely. The main purpose with the individual testing was to evaluatethe user interface of the current design as well as to get in-depth answers of the concept ofUGVs.

3.7 Research Ethics

The participants that have been of relevance in this thesis participated voluntarily. The par-ticipation was confirmed either verbally or written. The participants were explained that thestudy is in connection with a master thesis. They were also explained that their answers willbe saved locally and treated confidentially. Further, it was explained that all processing andperformance reporting will be performed with de-identified data. Analysis and presentationof collected data take place collectively for the entire group’s results and not at the individuallevel.

14

4 Results

The material in this chapter is based on the methodological approach that was presentedin the preceding chapter. Subchapter 4.1. shows the results of the Exploration Phase. Theconcepts are presented in subchapter 4.2, and the prototypes are shown in subchapter 4.3.Finally, subchapter 4.4 describes the outcomes of the user tests with the end-users.

4.1 Exploration Phase

The Exploration Phase was aimed to answer the research question: "What characterizes thecomplex environments in military settings and what is important to design for in the situ-ation?". The data obtained by the methods used in the phase, Qualitative Context Observationand the Explorative Interviews, with the soldiers from the home guard gave insights regardingcontext identification, shown in Table 4.1 below. Context is identified as both physical context,which refers to the physical environment that the UGV is driving in, and the psychologicalcontext, which refers to the cognitive mindset of the soldier likewise UGV pilot.

Table 4.1: Context Identification of Military UGVs

ContextPhysical Psychological

¨ Rough terrain ¨ Exhausted soldiers¨ Mobile environment ¨ Stressful¨ External noise ¨ Time pressure¨ Urban envrionment ¨ Tired soldiers¨ Dirty environment¨ Dangerous environment¨ Changing light condition

Furthermore, functions and qualities were identified and described as Design Drivers. SeeTable 4.2. The design drivers focus mainly on the design drivers of the UGV rather than thedesign drivers of the user interface of the control of the UGV.

15

4.2. Concept Phase

Table 4.2: Design Drivers of Military UGVs

Functions Qualities¨ To spy ¨ Smart¨ To deliver ¨ Loyal¨ To carry ¨ Controllable¨ To transport ¨ Adaptable¨ To localize ¨ Secure¨ To offer protection ¨ Discreet¨ To lead ¨ Perceive as if it is in the background¨ To mislead ¨ Portable¨ To guard ¨ Independent¨ To watch ¨ Durable¨ To lead different platforms simultaneusly¨ To chase¨ To pursue¨ To plan¨ To give feedback¨ To interpret instructions

In addition to the Design Drivers and the Context Identification, the results from the ExplorationPhase also identified the end-users. The outcome showed a need to design for home guardsoldiers since they are usually less trained than the rest of the Swedish Armed Force.

4.2 Concept Phase

After the Exploration Phase, the aim was to conceptualise the outcome and understand theuser needs. Therefore, the Concept Phase aimed to answer the research question: "What arethe user needs of the intended soldiers?". Based on the result from 4.1 five concepts weregenerated. Following five figures with connecting tables illustrate the concepts and theirpositive, negative and interesting aspects.

Figure 4.1, describes a concept with the main function to guard and the qualities to be con-trollable and to be perceived as if the UGV is in the background. This concept works as anintegrated user interface system consisting of a watch and a remote controller. The functionof the watch is to notify the pilot through vibrations when needed. This could for example bewhen the UGV notices enemies approaching. The multi-modal function would thereby allowthe pilot to rest when active control is not needed. Table 4.3 describes the positive, negativeand interesting aspects with Concept 1.

16

4.2. Concept Phase

Guard

Main function Perceive as if it is in the background

Controllable

Figure 4.1: Concept 1 with main function to guard

Table 4.3: PMI on Concept 1

Positive Negative Interesting¨ Multi-modal ¨ Could be easy to manipulate. What

does it see and what does it think thatit is seeing.

¨ Geofencing, assignzones

¨ Allows to be pas-sively alert

¨ Objects are easier to identify thanliving things

¨ Prototype for guard-ing

17

4.2. Concept Phase

The second figure, Figure 4.2, describes a concept with the main function to transport and thequality to be independent. This concept focuses on the complete autonomy of the UGV andpictures the UGV as a provider of necessities. Table 4.4 describes the positive, negative andinteresting aspects with this concept.

Transport

Main function

Independent

Figure 4.2: Concept 2 with main function to transport

Table 4.4: PMI for Concept 2

Positive Negative Interesting¨ Automated logistics for food is notthat interesting

¨ Follow route A to B

¨ "Find me" - function

18

4.2. Concept Phase

Concept 3, see Figure 4.3, describes a concept with the main function to carry, the subfunctionto offer protection and the qualities to be controllable and perceive as if it is in the background. Theidea of this UGV concept is to act as carrying aid for wounded soldiers. It further allows thehuman operator to attach the remote control to the UGV and use it as a steering wheel. Table4.5 describes the positive, negative and interesting aspects with the concept.

Carry

Main function Perceive as if it is in the background

Controllable

Offer protection

Figure 4.3: Concept 3 with main function to carry


Positive Negative Interesting¨ Controlling with gesture ¨ What happens to the UGV-

control if the pilot gets shot¨ A function to destroy itself

¨ Click function, allowingto maneuver the vehicle onthe UGV

¨ It needs to be able to weighthe load

¨ The remote control andwhat is shown on the screen

¨ Minimal amount of log inis needed¨ Following something¨ Finding things

19

4.2. Concept Phase

Concept 4, in Figure 4.4, distinguish a concept with the main function to carry, the subfunc-tion to interpret instructions and the qualities to be simple, adaptable and perceive as it is in thebackground. The main function of this concept is similar to Concept 3 (Figure 4.3) howeverthe concept tend to be different due to the desired functions. These functions put the humanoperator in a passive control where the UGV is following the operator with the connectedboots. The control of the UGV is through tactical responses. Table 4.6 describes the positive,negative and interesting aspects with the concept.

Main function

Adaptable

Interpret instructions

CarrySimple

Perceive as if it is in the background

Figure 4.4: Concept 4 with main function to carry


Positive Negative Interesting¨ Gestures ¨ You are always moving.

Will it recognise what it isgoing to follow

¨ Learn the ground surface

¨ Adapts to the pilot¨ Connected system¨ Gives feedback

20

4.2. Concept Phase

The last concept, presented in Figure 4.5, describes a concept with the main function to spyand the qualities to be loyal and smart. In this concept, the emphasis is on the communicationbetween the UGV and the human operator. The idea is that the interaction system is somekind of helmet of the human operator that provides the operator with a mental connectionwith the UGV. This enables the human to "feel" what the UGV is sensing. If the UGV identifiesdanger for instance then the human operator might "feel fear". Table 4.7 that describes thepositive, negative and interesting aspects with the concept.

To spy

Main function Loyal

Smart

Figure 4.5: Concept 5 with main function to spy


Positive Negative Interesting¨ Image interpreter ¨ Consumables. You do not want

emotional connection¨ Learn the ground sur-face

¨ The robot learns howthe soldiers in thegroup think and canmap out solutions

All in all, the result from the Concept Phase indicated that the clients are mainly interestedin the remote control device and the graphical user interface of the screen, see Concept 3(Figure 4.3, Table 4.5). However, there is an open-mind towards the multi-modal aspectsseen in Concept 1 (Figure 4.1, Table 4.3), Concept 3 (Figure 4.3, Table 4.5) and Concept 4 (Figure4.4, Table 4.6). The table below, Table 4.8, concludes the main functions of the UGV and thespecified functions the user interface of the control system that should take into considerationwhen prototyping.

21

4.3. Prototyping Phase

Table 4.8: The main functions of the UGV and the specified functions of the user interface.

Main functions Specified functions of the user interface¨ To spy ¨ Show position of vehicle related to the pilot¨ To guard ¨ Allow manual steering mode to maneuver UGV¨ To carry ¨ Show battery level of vehicle¨ To plan ¨ Show speed of vehicle¨ To give feedback ¨ Allow to change sight mode: IR, map, camera view

¨ Include a follow-me button¨ Create a navigation route¨ On/off - button¨ Allow steering mode to maneuver camera view¨ Be able to navigate on map

4.3 Prototyping Phase

The outcome of the Concept Phase served as an input for the Prototyping Phase. This phase,aimed to answer the research question: "How should the remote control for an interactiveUGV system be designed for infantry soldiers on the field?".

The first designs of the graphical user interface of the remote control was based on the func-tions identified in Table 4.8. The inhouse guerilla testing of the wireframed prototype resultedin a fast prototype iteration of the graphical user interface. The evolution of the prototype ispresented in Figure 4.6 to Figure 4.13.

ERROR MESSAGE!

20

5

2 2 H 3 3 M I N

KM/H

KM

n

ew

s

Figure 4.6: Dashboard top bar designproposal 1

20

5

2 2 H 3 3 M I N

KM/H

KM

n

ew

s

Figure 4.7: Dashboard top bar designproposal 2

ERROR MESSAGE!

5 KM

n

ew

s

20 KM/H 2 2 H 3 3 M I N

Figure 4.8: Dashboard bottom bardesign proposal 1

km/H20km5 H min14 33

BUTTON BUTTON

descriptive text

Figure 4.9: Dashboard bottom bardesign proposal 2

22

4.3. Prototyping Phase

ERROR MESSAGE!

20

5

2 2 H 3 3 M I N

KM/H

KM

n

ew

s

Figure 4.10: Scenario 1: Identify sub-jects on map

1 2

3

ERROR MESSAGE!

20

5

2 2 H 3 3 M I N

KM/H

KM

n

ew

s

Path 3Path 2Path 1

Choose path

Figure 4.11: Scenario 2: Plan a route

km/H13:37 20km5 H min14 33CURRENT STATUS

BUTTON BUTTON

min5 m1500fastest route on track

02/12/2020

BUTTON5864.329,1533.688 5865.253,1537.886

ir

Figure 4.12: Dashboard design out-come

km/H13:37 20km5 H min14 33CURRENT STATUS

BUTTON BUTTON

min5 m1500fastest route on track

02/12/2020

BUTTON5864.329,1533.688 5865.253,1537.886

ir

Figure 4.13: Latest dashboard designoutcome from the Design Detailingphase

The Prototype Evaluation resulted in the design changes themed into the three categories:

1. Desired features - Not a priority, but a wish

2. Interesting aspects - Aspects to further investigate

3. Necessary aspects - Aspects that need to be handled with highest priority.

Table 4.9: Desired, Interesting and Necessary aspects of the prototype.

Desired Interesting Necessary¨ 50/50 camera-map screenoption

¨ Compass on map, relativeto me as a pilot. Compasson camera, relative to theUGV

¨ Cancel/Stop button

¨ In Swedish instead of En-glish if it is designed for theSwedish Armed Forces

¨ Make the “Feedback-of-UGV-message-box”clickable for more informa-tion of the process

¨ Scalemeter on map

These design aspects were taken into consideration and the final prototype design shown inFigure 4.14 was created:

23


Maneuver UGV Camera View

� � � � � � � � � � ��

� � � � ��

3. levels of control

� � � � � � � � � � � � � � � � ��

� � � � � � � � � � � � � � � � � � � ��

� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

� � � � � � � � � � � � � � ��

� � � � � � � � � � � � ��

1. Follow Pilot. Semi-autonomous. UGV is following pilot.

2. Plan Route.Highest level of autonomy. UGV is monitored by pilot.

3. Manual Mode.UGV is maneuvered actively by pilot.

� � � � � �

� �

� � � � � � � � � � � � �

Maneuver UGV

Find Pilot.Locate Pilot on map

Find UGV.Locate UGV on map

Battery level of the remote control

Selection button

Figure 4.14: Final prototype of UI Design for remote control

4.4 User Evaluation Phase

The User Evaluation Phase aimed to test the final prototype design with its end-users. Theprocess was divided into two studies. The goal of the first study was to evaluate the conceptand its future use cases. While the second study focused on testing the end-user interactionwith the user interface.

4.4.1 Study 1: Concept

The result of Study 1 is divided into 2 parts; Technology Acceptance Questions and ScenarioQuestions.

Technology Acceptance Questions

Table 4.10 show the average values of the six TAM questions. The mean numbers show aslight shift towards the negative spectrum, as the Likert scale is from 1-7 and the mean (µ) re-

24


sults vary from 3.7-3.95. The standard deviation (σ) measures the variation in the participantanswers. With the lowest value of 1.34 in question 1 and the highest value of 1.53 in Question6. The average perceived usefulness (M) is 46.25 from a scale of 0 to 100, with a standarddeviation (SD) of 20.36 and a 95% confidence interval (CI) shown in Table 4.11. Table 4.11 isfollowed by the free text answers that have been summarised.

Table 4.10: The average answers to the six questions that revolved around usefulness.

Questions µ σ1. Using [this product] in my job would enable me to accomplish tasks more quickly. 3.8 1.442. Using [this product] would improve my job performance. 3.65 1.463. Using [this product] in my job would increase my productivity. 3.95 1.434. Using [this product] would enhance my effectiveness on the job. 3.7 1.455. Using [this product] would make it easier to do my job. 3.7 1.346. I would find [this product] useful in my job. 3.85 1.53

Table 4.11: Perceived Usefulness of the Average Participant

M 46.25SD 20.36

95% CI [37.33, 55.17]

Question 1: Accomplish tasks more quickly. The free text answers indicated that the par-ticipants could see the benefit with a UGV but not in their field (airbase and security). Theanswers also reported on trust issues towards the involvement of robotic systems. Further-more, participants expressed that it is hard to know if the system would enable to accomplishtasks quicker without physically testing the system.

Question 2: Improve work performance. Participants mentioned that it depends on thetask performed. The system would improve work performance if it is used in the case ofsimple patrol or surveillance missions that allow the soldiers to recover. Participants alsomentioned that sending a UGV to control unknown objects/findings or critical terrain wouldsave time, lives and would allow for taking better decision making. On the other hand, oneparticipant also mentioned the convenience of letting a human perform the work since thehuman know how to do it and how it has been done previously.

Question 3: Increase productivity. In the case of productivity, the participants mentionedthat the system would increase the productivity if it is used for simple tasks (patrol or surveil-lance missions) enabling soldiers to solve more important missions in the meantime. Sincethe soldiers work with different types of systems they find it of high importance to maintainknowledge about managing the UGV in order to strive for productivity using the system.

Question 4: Increase effectiveness. In the case of effectiveness, the participants pointed outthat they believe the human is able to detect important details of a higher precision comparedto a robot.

Question 5: Make it easier to do the job. The free text answers indicated that the soldierssee a potential mainly in terms of safety of personnel.

Question 6: Useful in the job. In the case of usefulness, the participants expressed thatthey could see useful aspects of the system that could be used as complement in their work.For instance the system could be used to cover large areas or indicate/locate the opponent atan early stage. Participants did however mention that they see more needs of the system inother military units.

25


Scenario Questions

Figure 4.15 illustrate the outcome of the Scenario Questions in Study 2. The diagram com-pares the desired control modes in the tested scenarios: Combat, Reconnoitre and Logistics. Thediagram is followed by a summary of the free text questions related to the Scenario Questions.

Figure 4.15: Results from the Scenario Questions.

Scenario 1: Combat In the case of Combat, the majority of the participants (14 participants),decided to send the UGV in front of the group and manually control it. 11 out of these 14 par-ticipants chose this control mode in order to keep the soldiers safe and 3 participants chosethis control mode due to that they did not trust the UGV to take own decisions in uncertainconditions as Combat. 3 participants chose the semi-autonomous mode of letting the UGVfollow the operator. Two out of these three chose this option in order to shift their focus to-wards the terrain/environment instead of looking into the screen. The third one chose thisoption due to the value of having the UGV as protection in an uncertain environment. Fur-thermore, one person chose the option to let the UGV drive in its highest level of autonomyout of the reason that the participant wanted to be able to plan for possible fire attack andin that case, it would be easier if the UGV could handle itself. Another answer given by aparticipant was to send the UGV in another part of the combat field to disturb and createconfusion.

Scenario 2: Reconnoitre The majority of the participants (17 participants) decided to con-trol the UGV manually. Eight participants mentioned that they would use the UGV manuallyin order to place the UGV at a stationary point and be able to control the sight of the UGVmanually. Seven participants chose to control the UGV manually due to safety reasons andlack of trust to the UGV. Moreover, two participants chose the option to let the UGV followpilot. The reasoning mentioned was to keep the UGV close as protection. Finally, one partic-ipant decided to send the UGV in its highest level of autonomy with the reasoning that if the

26


UGV gets hit then the area has been localised and enemy troops have been identified and ifit makes it then the area has been secured.

Scenario 3: Logistics In the situation of sending the UGV to get necessities, the majority (14participants) decided to use the UGV in full-autonomous degree. The participants mentionedthat this would enable the soldiers to focus on more important tasks as well as keeping thesoldiers away from unnecessary danger. One participant chose to manually control the UGVdue to lack of trust to a UGV. Five participants mentioned the importance of involving at leastone soldier in the process since soldiers know what kind of necessities that are needed.

Questions about the UGV

Table 4.12 summarizes the opportunities and limitations that the participants of the studyexpressed with military UGVs. Table 4.13 summarizes the potential use cases of militaryUGVs.

Table 4.12: Opportunities and limitations with military UGVs

Opportunities Limitations¨ Extra protection ¨ Trust issues¨ Another sensor ¨ Exposure of soldiers.¨ Carry heavy packing ¨ Technical failure¨ Use UGV in dangerous situations to save lifes ¨ How do I make sure it does what it says it is doing

Table 4.13: Potential use cases with military UGVs

Potential use cases¨ Distraction¨ Logistics¨ Demining¨ Reconnaissance tool¨ Stationary monitoring

4.4.2 Study 2: User Interface

All the participants managed to perform the task of planning a route successfully. However,two of the participants mentioned the desire of pointing out navigation points manually in-stead of trusting the route made by the system. One participant mentioned that the buttonscould be too small concerning the use of gloves but that it is beneficial that the system hasbuttons instead of touchscreen. Another participant pointed out the button of the compass asunnecessary since soldiers usually have a compass on themselves, while another participantmentioned that it would be good to have the compass on the monitor.

The result from the post-session interview with the five participants showed a pervadingopenness towards UGVs in military contexts. The interviews indicated that the participantshave a concern towards involving robots in decision-making processes and trusting the robotto take the best possible road towards a destination. The participants however mentionedthat they would use UGVs in safe situations. For example to get necessities. One of the par-ticipants mentioned that the UGV could serve as the soldiers extended eyes and ears whichputs a huge weight on the optics of the system. Another aspect mentioned was that the sizeof the UGV would have huge effect on how the soldiers would use the concept.

27

5 Discussion

The main research question of this thesis was: How do we design interactive unmannedground vehicle systems for infantry soldiers on the field? The following chapter will discussthe implications of the result thoroughly by discussing the result per subquestion. It will alsodiscuss the limitations of its methods used, and the work in a wider context.

5.1 Implication of the results

The implication of results will be discussed per sub-research question, as they cover the re-search done in this thesis.

1. What characterizes the context of use for infantry soldiers on the field, and what isimportant to consider in the design of an interactive UGV system for this context?

The first step to designing an interactive UGV system is understanding its context. The out-come of this thesis identified that the context for a military UGV is not necessarily only theenvironment and the physical conditions prevailing [22] but also the psychological context ofthe human operator (Table 4.1). The human operator’s psychological state is described as anaspect that could affect the mental workload, and the decision-making ability [27].

When talking about UGVs, the ultimate state is described as the level of autonomy where theUGV is teaming with the human. Granda et al. refers to this state as the Virtual Symbiosis [12].Nevertheless, one could argue whether the UGV need to be at its highest level of autonomyin order to do its job. In fact, the outcome of the research study, indicated that differentwork contexts and tasks such as Combat, Reconnoitre and Logistics require different levels ofautonomy (see Figure 4.15). In Combat and Reconnoitre, the majority of the participants wishedto have a manual operator role, and in the case of Logistics the majority of the participantswould be more likely to give the UGV a higher level of autonomy. One explanation for thiscould be a lack of trust towards letting a military robot take decisions in uncertain situations.However, this could also be because the system does not exist as of now, which gives theparticipant a hard time truly knowing if they would benefit from full autonomy. This lackof knowledge of their desires in relation to the doubt towards the UGV’s capabilities might

28

5.1. Implication of the results

put the participant in a position thinking that having full control of the UGV is the mostconvenient way to maneuver it.

2. What are the infantry soldiers’ needs in relation to an interactive UGV system on thefield?

The needs of the infantry soldiers were explored in iterations. Firstly, functions and qualitiesof the military UGV were identified as important design drivers that affect the concept ofthe interactive system (see Table 4.2). These design drivers were then conceptualized intoholistic concepts of interactive design systems (see Figure 4.1-4.5) that resulted in a list ofmain functions for the remote control as a part of an interactive UGV system (Table 4.8).The remote control acted as a way to conceptualize the relationship between the human andthe UGV into something tangible that the end-users could relate to in the process to furtherexplore the needs of an interactive system. In fact, one could argue that the research studyorbited around identifying the usage of an interactive UGV system, and thus explore thesoldier’s needs, with the help of the early holistic concepts and the low-fidelity prototype ofthe remote control. This was especially highlighted in the User Evaluation Phase, where thefinal users got to test the interactive UGV system. Moreover, the user evaluation showed thatthe users have different needs compared to specific tasks (Combat, Reconnoitre and Logistics)that the UGV performs (see Figure 4.15). In addition, the results obtained from this researchindicate that the participants would like to test the control system in proximate relation to theUGV to further understand the functionalities and capabilities of the UGV and its responselevel when giving it instructions and successively build up trust towards the new system.The trust issues that the participants were experiencing with the system highlighted that aniterative user-centered approach is required to further introduce the UGV as a member ofa human team. Moreover, the iterative process also served as a starting point in finding thereal end-users within the military troops, which could be infantry soldiers in the Home Guard(see Chapter 4.1).

To answer the question, when designing a first-of-a-kind system such as an interactive UGVsystem in this thesis, it is not easy for the user to know what their needs are nor what the newsystem’s contribution is. However, by introducing the user in a conceptual design process, itbecomes easier for the user to envision the new system [17]. The research in this thesis couldtherefore serve as the first iteration of designing an interactive UGV system. Furthermore, theuser involvement needs to be maintained during the next phases of development in order toachieve an end-product that cooperates with the user needs [27].

3. How should the remote control for an interactive UGV system be designed for infantrysoldiers on the field?

The remote control should allow the operating soldier to feel that the UGV serves as extendedears and eyes (see Chapter 4.4.2). This indicates that the soldiers require to be able to see howthe UGV is performing on a high level, and therefore a graphical user interface is suggested.Moreover, it should also be designed so that the soldiers’ uniforms and gears are not limitingthe use of the remote control (see Chapter 4.4.2). For instance, this could be by making thebuttons big enough to wear gloves and having physical buttons instead of touch. In otherwords, the remote control should be designed in a way that is suitable for the military con-text that involves environments such as rough terrain, dirt and urban environments (4.1).Furthermore, the remote control design should also allow the operating soldier to change thelevel of control based on different contexts of use (see Table 4.13). The results show that asof today, the Swedish Armed Forces are not ready for a fully autonomous UGV. Nonetheless,several potential future use cases where the UGV serves as a complement to the operatorwere identified (see Table 4.13). This shows an openness to involve UGVs in some parts ofthe military work, however, the context of when and how needs to be further investigated. It

29

5.2. Limitations with the method

also show that the highest level of autonomy (e.g. Level 10 in [29] or Virtual Symbiosis in [12])might not be the ultimate goal for a UGV to facilitate in military conditions. In the mentioneduse cases, the common denominator is that the UGV is not intended to be used in situationsthat might put humans in danger, allowing the human operator to serve in other situationsthat the individual finds as "more important".

Finally, in order to design the graphical user interface of the remote control, an essentialaspect identified is whether the participants find the interactive system useful in the firstplace [8]. The average soldier’s perceived usefulness resulted in a value of 46.25 on a scaleof 0 to 100. With a confidence level of 95% the perceived usefulness is between 37.33 and55.17. This indicates that the soldiers see the potential of the usefulness of the UGV but arenot entirely convinced. This could be because the end-users were within the airbase securityand not the intended infantry users. It could also be because the UGV does not exist withinthe Swedish Armed Forces today, which makes it hard to understand the use of the system.

All things considered, to design an interactive system, the most critical aspect is user accep-tance. If the user is not accepting the new system in the first place, then the user interface’sdetail design is useless. To accept the concept of military UGVs it is necessary to understandthe context of use and the needs of the users. This is suggested by involving the end-users in theconceptual design process. Lastly, an interactive UGV system does not necessarily need to befully autonomous in order to achieve autonomy for the users.

5.2 Limitations with the method

The research study has had its limitations, especially in user testing. This is mainly due tothe prevailing situation of Covid-19 that limited the user interaction part. However, the factthat the user testing had to be held remotely also helped to find new ways to test the conceptwith the end-users. This ended up creating a demonstration video of the interactive UGVsystem that now can be used to quickly investigate the concept of UGV with more soldiersin the Swedish Armed Forces and thereby generate a higher validity. Testing a prototype ofa physical remote control without the presence of the object that is supposed to be controlledhas been challenging. However, the demonstration video helped put the users in the mindsetand generate ideas and thoughts of the UGV.

To strive for validity, the selection of test participants was aimed at soldiers in the militaryfield. The intended end-users were infantry soldiers in the Home Guard. However, dueto the pandemic situation, it was harder to get the intended users, and the soldiers in thetesting were instead within the Airbase security service. The positive part of the participants’background was their familiarity with unmanned systems in the form of UAVs.

Moreover, the method of TAM does not have many published benchmarks even though it isa method with a high theoretical foundation. The use of TAM is even more limited within theHRI studies [27]. This makes it difficult to compare the results of "perceived usefulness" withother related work. However, it could be used as a benchmark for future HRI studies or forfuture user testing with other military troops.

5.3 The work in a wider context

As automation is seen more frequently in various human activities such as health care, trans-portation and wars, more questions concerning the social issues with automation are arising[33]. Throughout this thesis work, several questions arose concerning the ethical and socialaspects of UGVs in military settings. Even though the UGV in this thesis is not aimed to killpeople, the capabilities of UGVs is expanding to a level where it has the potential to become

30

5.3. The work in a wider context

fully autonomous in combat situations. This puts humans in the role of reflection. Who hasthe responsibility if the UGV kills another human? The person who designed the UGV or theperson that initiated the war? Do emotionless robots fit on the battlefield? Does the human fiton the battlefield in the first place? Humans have not evolved to function in war conditions,but could UGVs be engineered to function well in them? The questions are not easy to answerand there is certainly no doubt that the subject of the robot’s role in society is a controversialtopic. The ethical consideration of robots in the military has both its benefits and limitation[2].

31

6 Conclusion

The thesis’s purpose has been to explore how to design interactive systems for UnmannedGround Vehicles (UGVs) for users in military contexts. The aim was to create a prototypeof a controller interface that is accepted by the users and meets their user needs through auser-centered design approach.

The sum of the results indicates that the key to designing interactive systems for UGVs isunderstanding the context the UGV and the soldiers will work within. Further, the contextdefines the level of decision-making shared between the human operator respectively theUGV. The interaction with the soldiers in the Swedish Armed Forces shows that the field ofautomation is relatively new; hence more research is needed in regards to trust autonomousvehicles. Early user involvement and keeping the user in the design loop are therefore fun-damental aspects in order to build up the level of trust and consequently the user acceptance.

32

Bibliography

[1] Alain Abran, Adel Khelifi, Witold Suryn, and Ahmed Seffah. “Usability meanings andinterpretations in ISO standards”. In: Software quality journal 11.4 (2003), pp. 325–338.

[2] Ronald C Arkin. “Ethical robots in warfare”. In: IEEE Technology and Society Magazine28.1 (2009), pp. 30–33.

[3] Lina Linnea Aspen. “Guerilla testing of mobile medical applications in clinical settingsExperiences from field testing in four Ugandan health institutions”. In: 2016.

[4] Keshav Bimbraw. “Autonomous cars: Past, present and future a review of the develop-ments in the last century, the present scenario and the expected future of autonomousvehicle technology”. In: 2015 12th international conference on informatics in control, au-tomation and robotics (ICINCO). Vol. 1. IEEE. 2015, pp. 191–198.

[5] Adream Blair-Early and Mike Zender. “User interface design principles for interactiondesign”. In: Design Issues 24.3 (2008), pp. 85–107.

[6] John W Creswell and Cheryl N Poth. Qualitative inquiry and research design: Choosingamong five approaches. Sage publications, 2016.

[7] Kerstin Dautenhahn. “Methodology & themes of human-robot interaction: A growingresearch field”. In: International Journal of Advanced Robotic Systems 4.1 (2007), p. 15.

[8] Fred D Davis. “A technology acceptance model for empirically testing new end-userinformation systems: Theory and results”. PhD thesis. Massachusetts Institute of Tech-nology, 1985.

[9] Sarah-Louise Donovan and Tom Triggs. Investigating the effects of display design on un-manned underwater vehicle pilot performance. Tech. rep. DEFENSE SCIENCE and TECH-NOLOGY ORGANIZATION VICTORIA (AUSTRALIA) MARITIME . . ., 2006.

[10] Terrence Fong and Charles Thorpe. “Vehicle teleoperation interfaces”. In: Autonomousrobots 11.1 (2001), pp. 9–18.

[11] Michael A Goodrich and Alan C Schultz. “Human–Robot Interaction: A Survey”. In:Human–Computer Interaction 1.3 (2007), pp. 203–275.

[12] Thomas M Granda, Mark Kirkpatrick, Tracye D Julien, and Larry A Peterson. “Theevolutionary role of humans in the human-robot system”. In: Proceedings of the HumanFactors Society Annual Meeting. Vol. 34. 9. SAGE Publications Sage CA: Los Angeles, CA.1990, pp. 664–668.

33

Bibliography

[13] Richard Herriott. “What kind of research is research through design”. In: Design Revolu-tions; Manchester School of Art, Manchester Metropolitan University: Manchester, UK (2019).

[14] Monique WM Jaspers, Thiemo Steen, Cor Van Den Bos, and Maud Geenen. “The thinkaloud method: a guide to user interface design”. In: International journal of medical infor-matics 73.11-12 (2004), pp. 781–795.

[15] Urška Lah, James R Lewis, and Boštjan Šumak. “Perceived usability and the modifiedtechnology acceptance model”. In: International Journal of Human–Computer Interaction36.13 (2020), pp. 1216–1230.

[16] Peiqun Lin, Jiahui Liu, Peter J Jin, and Bin Ran. “Autonomous vehicle-intersection coor-dination method in a connected vehicle environment”. In: IEEE Intelligent TransportationSystems Magazine 9.4 (2017), pp. 37–47.

[17] Jonas Lundberg, Mattias Arvola, Carl Westin, Stefan Holmlid, Mathias Nordvall, andBilly Josefsson. “Cognitive work analysis in the conceptual design of first-of-a-kindsystems–designing urban air traffic management”. In: Behaviour & Information Technol-ogy 37.9 (2018), pp. 904–925.

[18] Ji-Ye Mao, Karel Vredenburg, Paul W. Smith, and Tom Carey. “The State of User-Centered Design Practice”. In: Commun. ACM 48.3 (2005), pp. 105–109. ISSN: 0001-0782.URL: https://doi.org/10.1145/1047671.1047677.

[19] Carlos Martínez and Felipe Jiménez. “Implementation of a Potential Field-BasedDecision-Making Algorithm on Autonomous Vehicles for Driving in Complex Envi-ronments”. In: Sensors 19.15 (2019), p. 3318.

[20] Paul Milgram, Shi Yin, and Julius J Grodski. “An augmented reality based teleoperationinterface for unstructured environments”. In: ANS 7th Meeting on Robotics and RemoteSystems, Augusta. Citeseer. 1997, pp. 101–123.

[21] Robin Murphy and J Sprouse. “Strategies for searching an area with semi-autonomousmobile robots”. In: Proceedings of Robotics for Challenging Environments. Citeseer. 1996,pp. 15–21.

[22] Jose E Naranjo, Felipe Jimenez, Manuel Anguita, and Jose Luis Rivera. “Automation kitfor dual-mode military unmanned ground vehicle for surveillance missions”. In: IEEEIntelligent Transportation Systems Magazine 12.4 (2020).

[23] Jakob Nielsen. “Estimating the number of subjects needed for a thinking aloud test”.In: International journal of human-computer studies 41.3 (1994), pp. 385–397.

[24] Jakob Nielsen. Ten usability heuristics. https://www.nngroup.com/articles/ten-usability-heuristics/. Online; accessed 11 February 2021. 2005.

[25] Janni Nielsen, Torkil Clemmensen, and Carsten Yssing. “Getting access to what goeson in people’s heads? Reflections on the think-aloud technique”. In: Proceedings of thesecond Nordic conference on Human-computer interaction. 2002, pp. 101–110.

[26] Linda Onnasch and Eileen Roesler. “A Taxonomy to Structure and Analyze Human–Robot Interaction”. In: International Journal of Social Robotics (2020), pp. 1–17.

[27] Sonja K Ötting, Lisa Masjutin, Jochen J Steil, and Günter W Maier. “Let’s Work To-gether: A Meta-Analysis on Robot Design Features That Enable Successful Human–Robot Interaction at Work”. In: Human Factors (2020), pp. 1–24.

[28] Kevin N Otto and Kristin L Wood. “Product evolution: a reverse engineering and re-design methodology”. In: Research in engineering design 10.4 (1998), pp. 226–243.

[29] Raja Parasuraman, Thomas B Sheridan, and Christopher D Wickens. “A model fortypes and levels of human interaction with automation”. In: IEEE Transactions on sys-tems, man, and cybernetics-Part A: Systems and Humans 30.3 (2000), pp. 286–297.

34

https://doi.org/10.1145/1047671.1047677

https://www.nngroup.com/articles/ten-usability-heuristics/

https://www.nngroup.com/articles/ten-usability-heuristics/

Bibliography

[30] Jouni Rantakokko, Fredrik Näsström, Jonas Nygårds, Kristoffer Bengtsson, and RogierWoltjer. Tekniköversikt autonoma och obemannade system Del 2: Markstriden. Tech. rep. FOI,2019.

[31] Dale Richards and Alex Stedmon. “Designing for human–agent collectives: displayconsiderations”. In: Cognition, Technology & Work 19.2-3 (2017), pp. 251–261.

[32] Jennifer Rowley. “Conducting research interviews”. In: Management Research Review35.3/4 (2012), pp. 260–271.

[33] Lambèr Royakkers and Rinie van Est. “A literature review on new robotics: automationfrom love to war”. In: International journal of social robotics 7.5 (2015), pp. 549–570.

[34] Per Runeson and Martin Höst. “Guidelines for conducting and reporting case studyresearch in software engineering”. In: Empirical software engineering 14.2 (2009), pp. 131–164.

[35] Jean Scholtz. “Theory and evaluation of human robot interactions”. In: 36th AnnualHawaii International Conference on System Sciences, 2003. Proceedings of the. IEEE. 2003,10–pp.

[36] International Organization for Standardization. “ISO 9241–210: 2019 (en) Ergonomicsof human-system interaction—Part 210: Human-centred design for interactive sys-tems”. In: (2019).

[37] Kuo-Shih Tseng and Bérénice Mettler. “Analysis and Augmentation of Human Perfor-mance on Telerobotic Search Problems”. In: IEEE Access 8 (2020), pp. 56590–56606.

[38] Annemiek Van Boeijen, Jaap Daalhuizen, Roos van der Schoor, and Jelle Zijlstra. DelftDesign Guide: Design Strategies and Methods. BIS Publishers, 2014.

[39] Robert A Virzi. “What can you learn from a low-fidelity prototype?” In: Proceedings ofthe Human Factors Society Annual Meeting. Vol. 33. 4. SAGE Publications Sage CA: LosAngeles, CA. 1989, pp. 224–228.

[40] Amber M Walker, David P Miller, and Chen Ling. “User-centered design of an attitude-aware controller for ground reconnaissance robots”. In: Journal of Human-Robot Interac-tion 4.1 (2015), pp. 30–59.

[41] Thomas Weiskircher, Qian Wang, and Beshah Ayalew. “Predictive guidance and con-trol framework for (semi-) autonomous vehicles in public traffic”. In: IEEE Transactionson control systems technology 25.6 (2017), pp. 2034–2046.

[42] Liu Xin and Dai Bin. “The latest status and development trends of military unmannedground vehicles”. In: 2013 Chinese automation congress. IEEE. 2013, pp. 533–537.

[43] Holly A Yanco and Jill Drury. “Classifying human-robot interaction: an updated taxon-omy”. In: 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat.No. 04CH37583). Vol. 3. IEEE. 2004, pp. 2841–2846.

[44] Holly A Yanco and Jill L Drury. “A taxonomy for human-robot interaction”. In: Pro-ceedings of the AAAI Fall Symposium on Human-Robot Interaction. sn. 2002, pp. 111–119.

[45] Julius Ziegler, Philipp Bender, Markus Schreiber, Henning Lategahn, Tobias Strauss,Christoph Stiller, Thao Dang, Uwe Franke, Nils Appenrodt, Christoph G Keller, et al.“Making bertha drive—an autonomous journey on a historic route”. In: IEEE Intelligenttransportation systems magazine 6.2 (2014), pp. 8–20.

35

Interaction Design for Remote Control of Military Unmanned ...

Documents

Transcript of Interaction Design for Remote Control of Military Unmanned ...