Chapter XX

Usability evaluation of touchscreen phone in

emergency context Fernanda Pozza, Vanessa Roncalio, Cristiana Miranda, Maria Lucia

Okimoto Federal University of Paraná - UFPR

Curitiba, Brazil fepozza@gmail.com

 

ABSTRACT  

Emergencies are always unexpected, causing changes in the emotional state of the individual, such as tension and stress. In this situation, a simple task such as making a phone call can prove difficult, considering that the user may not be familiar with the device. Also, interfaces are often not designed for emergency use. This paper reports on a pilot test of the usability of a touchscreen phone in an emergency context, which consisted of three phases: pre-test questionnaire, test and post-test questionnaire. Thus, it was possible to obtain information about the usability of the device and the user´s response in a simulated emergency situation.

Keywords: emergency context, usability, touchscreen phone

1   INTRODUCTION  

According to Paulheim et al. (2009), in their study "Improving Usability of Integrated Emergency Response System," the usability in an emergency situation is gaining more attention from researchers. He also highlights that the unpreparedness of designers and users to this issue is due to the fact that most devices are not designed considering their use during a future emergency scenario. Consequently,

when emergencies occur, the users are not very familiar with a particular interface to respond in a very short time, considering that he or she is under stress and out of their normal responses to the environment (Paulheim et al., 2009).

It is known that emergency situations increase the probability of human errors. The severity of effects, in this case, can lead to serious consequences to human life. Thus, in order to contribute with usability in emergencies, this study is based on the usability test of a touchscreen phone in a simulated emergency setting.

Over the past few years have been brought to market devices with different capabilities and features. Such devices involve increasingly complex tasks, requiring more cognitive ability of the user. Thereby, the interfaces play a key role in communication between human and machine. The touch screen phones belong to this category of devices.

Santos and Maciel (2010) states that cell phones today are designed to not only make calls but to be "mobile multimedia terminals." Thus, they offer more features, storage and processing. Most of the phones interfaces are composed by visual elements on the screen alongside with pressing or sliding buttons. These are the means by which it is possible to interact with the device. Therefore, it is important that the interface is designed to optimize the product´s performance.

According to the Brazilian Standard NBR 9241-11 (2002) usability is the ability of a product to meet certain goals to specific users with effectiveness, efficiency and satisfaction, considering the specific context of use. Hence, the context is an important factor in usability studies, because according to the NBR 9241-11 (2002), the usability can vary significantly in different usage situations.

In order to evaluate the usability of a touchscreen phone, Nokia model 5530, this pilot test was set for the users ‘to make a call', during a simulated emergency context. Were considered in the experiment, the emotional and behavioral aspects of the user and the interface characteristics of the cell phone. However, this article deals only with issues related to the interface of the device being tested.

2   METHODOLOGICAL  PROCEDURES    

2.1   Description  of  the  tasks  

Seven participants (n = 7) were chosen for this study, considering only that they did not own the same model of the phone being tested. Each participant was instructed to ‘make an emergency call’ from a locked screen phone.  

The tasks to be completed by the participants included: to unlock the phone screen (Task 1), to activate the phone's keypad (Task 2); to dial the given number (Task 3), to initiate the call (Task 4), as illustrated in Figure 1.

Figure 1 Tasks to be performed on the phone´s interface during the test

2.2   Description  of  the  room  and  equipment  

Two rooms were prepared for the test isolated from one another. In the first room (Room 1), each participant had their heart rate measured (initial frequency). In this room the participant also answered a pre-test questionnaire about stress perception, and a post-test questionnaire in order to enquire the participant's impressions about the device and the test itself.

The test was conducted in Room 2, equipped with: a laptop connected to speakers and the phone (test instrument) on a table, two video cameras, one facing the participant's face and torso and the second directed to the phone screen to assist in the counting of ‘touches’ the heart rate monitor adjusted on each participant with the digital display positioned on the table. Thus, it was possible to compare the initial heart rate (before testing), intermediate (during test) and final (right after the experiment) of each individual. Besides the equipment, there was an observer next to the participant counting the number of ‘touches’ and another observer further away marking the time for the completion of the task.

The experiment was performed in three steps, as Figure 2:

Figure 2 Phases of the test

2.3   Emergency  context  

An emergency event causes an immediate response in any person, generating increased levels of stress. To simulate the emergency context in this study we opted for an approximation of the person´s emotional state in a real emergency situation. Because there are ethical principles that limit the level of stress the test participant is submitted.

For the induction of stress in the laboratory is commonly used sensory stimuli such as noise or exposure to extreme temperatures or psychological stressors that include demanding cognitive tasks and social assessment (Bollini, Walker, 2002).

We tried to induce some significant levels of stress by combining the following factors: psychological suggestion triggered by a story told to the participant, by the need to perform the task in the shortest time and with the disturbance caused by a sudden and loud siren (71.9 decibels , 0.8 Hz). Each individual heard the following story: ‘you and a friend are in an isolated location. Suddenly, your friend passes out. You try to call for help using your own cell phone when you realize its battery is discharged. Your only option is to use your friend´s touch screen phone. Every minute is crucial. You need to act fast! Pick up the phone, unlock it and call 0000-0000’. The total time for performing the test should not exceed two minutes, being this time set from the time of the expert (10 seconds).

The best known test for induction of stress in the laboratory is the Trier Social Stress Test - TSST (Kirschbaum, Pirke, Hellhammer, 1993), designed to cause anxiety and whose protocol has been adapted to induce stress in other studies. The TSST has an anticipatory period in which the participant is preparing for a speech to be presented to two or three evaluators, besides other monitored tasks.

However it was not intended to replicate the TSST procedures in this study. Instead of creating the anticipatory period reported in TSST the present experiment was set to be unpredictable for the participant, as would occur in a real emergency. In this case, the participant was informed of the task a few seconds before executing it.

2.4   Usability  metrics  

To evaluate the usability of the touchscreen phone were applied these metrics proposed by Tullis and Albert (2008): task success, the time taken to perform it and the efficiency.

For task success the authors mean its completeness, with ease or not (Tullis, Albert, 2008). In this study were considered the time each participant took from the start of the first task to the completion of the last task. According to Albert and Tullis (2008), the efficiency can be measured by both the time and effort. This comprises the cognitive and physical aspects and it can be defined as the user´s actions or steps to complete the action. (Tullis, Albert, 2008). For this metric, we took into account the number of ‘touches’ and the time of each participant in relation to the number of ‘touches’ and time of an expert, considering only

individuals who have succeeded in all tasks. Thus, the number of ´touches´ was counted until the individual accomplish the tasks. For this study, we consider ´touch´ any attempt to unlock the phone and make the call, whether giving quick touches with fingertips, pressing or sliding them over the phone or on the screen.

2.5   Independent  variables  

The pre-test questionnaire reported on the emotional aspects, therefore, not to be covered in this article. In the post-test questionnaire, were asked questions related to opinions of the individual in relation to the test and the device being tested. Thus, Figure 3 highlights (in gray) such issues, proposed on a semantic differential scale.

Figure 3 semantic differential scale applied in the post-test questionnaire

In open-ended questions were asked the following questions: • How do you think it should be the process to unlock the screen of the phone

tested? • What did you think of the emergency simulation? • What did you think of this touchscreen phone?

3   RESULTS

3.1   Participants

The group aged between 22 and 49-years-old, 40% between 22 and 25 and 60%

between 32 and 49. Among the participants there was only one woman. Four

participants had Nokia cell phones; however they were different models of the device being tested. The other three participants had different brands: Foston, Samsung and Gradient.

3.2   Influence  of  emergency  context  

The emergency context in the proposed experiment was emphasized by these elements: a short story told to the participant, by the loud siren during testing and by the urgency to complete the tasks in the shortest possible time. The verification of the influence of these elements can be seen by comparing the initial heart rate with the both the intermediate and final heart rate, as shown in Table 1.

Table 1 Comparison of initial, intermediate and final heart rates (beats per minute)

Participant Initial Intermediate Final 1 85 89 76 2 86 112 97 3 93 98 95 4 95 112 99 5 85 101 94 6 94 89 90 7 85 105 103

It can be noted that with the exception of participant 6, all other participants

showed a higher heart rate during the test. Therefore, we believe that these individuals were influenced by the emergency resources used - history, loud siren and timing.

3.3   Success  and  task  time

Two participants (2 and 5) failed to complete the task "to unlock screen" because they did not find the unlocking button of the device. This action demanded more time for all participants. The task time for each participant, as well as the number of ‘touches’ made by them can be viewed and compared to the time and ‘touches’ of the expert, as shown in Table 2.

Table 2 Total time and number of touches of each participant

Participants Time to accomplish all tasks Number of touches Expert 10’’ 11

1 60’’ 38 2 50’’ 42 3 36’’ 8

4 82’’ 49 5 36’’ 38 6 75’’ 43 7 31’’ 14

It is noted that the lowest time achieved by the participants was 31 seconds

while the expert took 10 seconds. In relation to the number of touches, the third participant was able to make the call under the expert time. However, he used the auto redial instead of following the specified tasks. Therefore, the participant closest to the number of touches of the expert was number seven, with 14 touches.

3.4   Efficiency

To measure the efficiency were taken only participants who were successful in all tasks, as suggested by Tullis and Albert (2008). So in order to validate the pilot test were compared, first, the time for each participant (PT) at the time of the expert (ET), considering ET / PT, as shown in Table 3. In a second phase, were considered the number of touches of the participants (PN) and the number of touches of the expert (EN), EN / PN, as shown in Table 4.  

Table 3 Measured efficiency considering the time to complete all tasks.  

Participants Efficiency (time) 1 0,16 3 0,27 4 0,12 6 0,13 7 0,32

Table 4 Measured efficiency considering the effort to complete all tasks.

Participants Efficiency (effort) 1 0,28 4 0,22 6 0,25 7 0,78

Considering that the third participant made the call using the auto redial instead

of dialing the number, his results were not included in the calculation of the efficiency and effort. Following the formulas described and considering the time and effort of the expert, the efficiency rate in both cases equals to 1. Observing the Tables 3 and 4, it can be argued that the participant 7 came closest to the index of

the expert, with 0.32 for time and 0.78 for effort. Therefore, none of the subjects achieved the efficiency ratio taken as the parameter.

3.5   Results  of  the  independent  variables

The post-test questionnaire allowed the collection of additional data. The responses obtained with the semantic differential were coincident in some aspects and discordant in others, as shown in Figure 4.

Figure 4 Results of the semantic differential scale

By analysing the responses certain inconsistencies became evident. Participants did not consider the device too complex and most of them said it was easy to use it, while most of them agreed partially that they need instructions to use it. They also agreed fully or partially that others will also have difficulty using the product. All of them agreed that the figures on the initial screen are easy to understand. Five participants found it difficult to identify the unlocking button of the device, while the call was considered an easy task for most of them. However, only two participants found the device easy to use in an emergency.

When questioned about how the screen unlocking process should be, we obtained the following: five participants suggested that the unlocking should be integrated into the touchscreen display; one participant said it should be automatic, and another said it did not need to change at all. By comparing these responses to the results of time and effort it can be noted that there is an inconsistency between their performance and opinions.

About the emergency simulation test, the opinions given by the participants are as follows (Table 5):

Table 5 Participants' opinions about the test.

Participants Comments about the test

1 "Well, a little concentration was required" 2 "Interesting" 3 "It enabled an experience that really could happen" 4 “"I really got the impression of an emergency" 5 "Very interesting" 6 "Interesting" 7 "Simple and practical"

Asked about their views on the mobile device tested, the participants pointed out

that (Table 6):

Table 6 Participants' opinions about the device being tested.

Participants Comments about the phone

1 "The touch screen display is a bit weak, it failed sometimes" 2 “Good and practical" 3 "An aesthetically beautiful device, but it took some time to figure

out how to unlock it" 4 "Okay, I've had worse gadgets" 5 "Apart from the unlocking button the phone is apparently good" 6 "Better than my own phone" 7 "With minimal tools for the user"

CONCLUSION

The proposed method of usability in emergencies consisted of an exploratory study of the context of use. It could be inferred that the emergency context altered the performance of users, with the addition of elements in the environment such as a sudden and loud noise through siren and a timing device. The perception of the influence of these factors can be attributed to the increased heart rate during and after the test. However, a comparative study needs to be done in a context of stress free environment. Thus, one cannot state with certainty to in what degree the emergency context caused tension and stress in participants.

From the techniques applied (questionnaires and test) and metrics used (success of tasks, time and efficiency), were gathered sufficient information to assess the

usability of a touchscreen phone, considering the proposed tasks in an emergency context.

In the post-test questionnaire, were obtained inconsistent responses. It is believed that this can be mitigated by asking more objective and specific questions.

Regarding the usability of the touchscreen phone tested, the main difficulty for the participants was to unlock the screen to access their features. This was emphasized in the post-test questionnaire. Such difficulty was due to the lack of signaling and information about this action in the device. From this, it is believed that the manufacturers assume that the phone users have read the manual and therefore know how to use all the phone features. But the manufactures does not consider that the same device might be used in an emergency by someone other than the owner of the phone. To be able to access the features of the mobile phone can be decisive in an emergency, so the question is: are the products designed for an emergency situation?

Thus, knowing the importance that mobile phones have gained in our lives it is recommended that there should be more specific guidelines for usability considering different contexts of use.

REFERENCES  

Bollini, Anna M.; Walker, Elaine F. Efficacy of a laboratory stressor: failure to replicate the Trier Social Stress Test. Accessed February 5, 2012, http://www.jasnh.com/c2.htm.

Associação Brasileira de Normas Técnicas - ABNT. NBR 9241-11: requisitos ergonômicos para trabalho de escritórios com computadores - parte 11 - orientações sobre usabilidade. Rio de Janeiro: ABNT, 2002.

Kirchbaum, Clemens; Pirke, Karl-Marlin; Hellhammer, Dirk H.; The Trier Social Stress Test: a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology, 28:76-81, 1993.

Paulheim, H., Döweling, S., Tso-Sutter, K.H.L., Probst, F., and Ziegert,T. Improving Usability of integrated emergency response systems: the SoKNOS approach. In Proceedings of GI Jahrestagung. 2009, 1435-1349.

Santos, Robson; Maciel, Francimar. Estilos de interação em interfaces para dispositivos móveis. Ação Ergonômica 5: 21-27. Rio de Janeiro, PUC-Rio, 2010.

Tullis, Tom; Albert, Bill. Measuring user experience: collecting, analyzing, and presenting usability metrics. Massachusetts: Morgan Kaufman, 2008.