LNCS 6768 - Accessible Education for Autistic Children: ABA ...

10
C. Stephanidis (Ed.): Universal Access in HCI, Part IV, HCII 2011, LNCS 6768, pp. 511–520, 2011. © Springer-Verlag Berlin Heidelberg 2011 Accessible Education for Autistic Children: ABA-Based Didactic Software Silvia Artoni 1 , Maria Claudia Buzzi 1 , Marina Buzzi 1 , Claudia Fenili 2 , and Simona Mencarini 1 1 CNR-IIT, via Moruzzi 1, 56124 Pisa, Italy [email protected], [email protected], [email protected], [email protected] 2 ASA (Associazione Sindromi Autistiche), Sesto Fiorentino (FI), Italy [email protected] Abstract. Behavior Analysis is the science that studies human behavior. Applied Behavior Analysis (ABA) is a process of systematic application of interventions whose principles are based on Learning Theory. The aim is to improve socially significant behaviors; the interventions are responsible for changing the behavior. ABA is successfully applied for educating autistic children. This paper discusses the design of an eLearning environment, based on ABA principles, for delivering accessible education to autistic children. ABA is a complex methodology and it is crucial to maintain therapy coherence between all the caregivers while allowing them maximum flexibility on the ABA application to better adapt to individual child responses. Keywords: autism, ABA, didactic software, usability, accessibility, design. 1 Introduction Autism Spectrum Disorder (ASD) is a pervasive development disturbance characterized by the difficulty or inability to communicate and consequent impaired social integration. The autism spectrum is very large and its symptoms vary significantly from child to child. Typical behaviors include avoiding eye contact and stereotypical behaviors such as rocking, head rotations or flapping hands. Common issues are: the loss or reduction of functional language (utters sounds that are cries or screams or echolalia), different perception (extreme sensibility to noisy environment). Some cases also present additional cognitive or perception deficits (Comorbidity). The Diagnostic and Statistical Manual of Mental Disorders, DSM-IV TR [1] and the international statistic Classification of Mental and Behavioral Disorders [2] define autism in term of delays or anomalies that appear within 3 years of age, in at least one of following areas: 1) social interaction, 2) communicative and social use of language, 3) symbolic or imaginative play. This amplitude of problems and needs requires a personalized educational approach to best exploit the child’s abilities. Conventional educational techniques are ineffective in autistic children for several reasons, including their different sensorial perception (for instance, difficulty remaining in a noisy environment such as a

Transcript of LNCS 6768 - Accessible Education for Autistic Children: ABA ...

C. Stephanidis (Ed.): Universal Access in HCI, Part IV, HCII 2011, LNCS 6768, pp. 511–520, 2011. © Springer-Verlag Berlin Heidelberg 2011

Accessible Education for Autistic Children: ABA-Based Didactic Software

Silvia Artoni1, Maria Claudia Buzzi1, Marina Buzzi1, Claudia Fenili2, and Simona Mencarini1

1 CNR-IIT, via Moruzzi 1, 56124 Pisa, Italy [email protected], [email protected], [email protected], [email protected]

2 ASA (Associazione Sindromi Autistiche), Sesto Fiorentino (FI), Italy [email protected]

Abstract. Behavior Analysis is the science that studies human behavior. Applied Behavior Analysis (ABA) is a process of systematic application of interventions whose principles are based on Learning Theory. The aim is to improve socially significant behaviors; the interventions are responsible for changing the behavior. ABA is successfully applied for educating autistic children. This paper discusses the design of an eLearning environment, based on ABA principles, for delivering accessible education to autistic children. ABA is a complex methodology and it is crucial to maintain therapy coherence between all the caregivers while allowing them maximum flexibility on the ABA application to better adapt to individual child responses.

Keywords: autism, ABA, didactic software, usability, accessibility, design.

1 Introduction

Autism Spectrum Disorder (ASD) is a pervasive development disturbance characterized by the difficulty or inability to communicate and consequent impaired social integration. The autism spectrum is very large and its symptoms vary significantly from child to child. Typical behaviors include avoiding eye contact and stereotypical behaviors such as rocking, head rotations or flapping hands. Common issues are: the loss or reduction of functional language (utters sounds that are cries or screams or echolalia), different perception (extreme sensibility to noisy environment). Some cases also present additional cognitive or perception deficits (Comorbidity).

The Diagnostic and Statistical Manual of Mental Disorders, DSM-IV TR [1] and the international statistic Classification of Mental and Behavioral Disorders [2] define autism in term of delays or anomalies that appear within 3 years of age, in at least one of following areas: 1) social interaction, 2) communicative and social use of language, 3) symbolic or imaginative play.

This amplitude of problems and needs requires a personalized educational approach to best exploit the child’s abilities. Conventional educational techniques are ineffective in autistic children for several reasons, including their different sensorial perception (for instance, difficulty remaining in a noisy environment such as a

512 S. Artoni et al.

classroom), difficulty concentrating, inability to communicate needs or to accomplish tasks (provoking frustration and “problem behaviors”).

According to several studies [3], [4], early intervention in children affected by autism disorder is more effective for learning and developing social abilities. It is very important to act quickly in the first two years of life to take advantage of the plasticity of children’s brains [5] [6], [7]. Often autism symptoms appear at 18-36 months, and some alarming signals may be observed very early, from birth. Early individuation of autism is certainly an advantage: it is crucial to have an early diagnosis to quickly start behavior therapy [8], [9], [10], [11], [12].

Several studies have shown that early and intensive intervention, beginning in preschool with a load of at least 6-8 h a day, is the only way to produce significant improvement in knowledge and skills, in some cases (high-functional children) with development corresponding to the normotype [13], [12], [14], [15]. Lovaas published the first study (1973) on effectiveness of intensive behavioral intervention in children with autism. The results of this study led the author to develop the ABA model, validated by subsequent research in 1987 [11]. Eikeseth and others showed that ABA treatment, compared to an eclectic type of intervention, produced the most progress in all areas of evolutionary [16]. Sallow and Graupner have recently replicated the studies of the father of ABA, showing that approximately half the children subjected to early and intensive behavioral intervention have reached the age of 7 years with a level of adaptive functioning and intellectual equal to the standard [17]. In the state of Maine Administrators of Services for Children with Disabilities regarded ABA as the only scientifically validated intervention, and concluded that "early diagnosis of autism should be accompanied by the proven effectiveness of an intensive ABA intervention to produce optimal results and benefits of long-term costs" [18]. In Italy the SINPIA guidelines (May 2005) approved ABA as an intervention for teaching autistic children. Thus, early diagnosis of autism is essential to implementing behavioral intervention of this kind. Results of this first study led the author to develop a model of ABA intervention, validated by subsequent investigation in 1987 [11]. Several other approaches are in large part based on methods used in ABA, but most authoritative results in the scientific environment are related to ABA itself [16].

More than 30 years of research have created a highly effective model of intervention through the systematic and intensive application of basic behavioral principles and structured teaching techniques, based on Discrete Trial Teaching (DTT) [10], [19]. ABA therapy requires caregivers to implement precise educational protocols (error-less learning, reinforcement, etc.) and rigorous verification of the child’s learning progress (by filling out structured forms), and has proved to be particularly effective [14], [20], [19].

In short, by conducting a series of sequential trials of increasing difficulty, the child, initially driven by the therapist’s prompts to avoid errors, will gradually be able to operate independently and assimilate concepts/skills. The success of the child’s therapy depends on the coherence of the program, which should be respected not only at school and during the therapy sessions, but also at home with parents and relatives. The family’s participation is fundamental, and parents’ active participation in the ABA program is strongly recommended. Lovaas was the first to suggest that parents play an important therapeutic role in the treatment of children with ASD [6], [11].

Accessible Education for Autistic Children 513

In this paper we describe the participatory design (also involving therapists and children’s parents) of a didactic software program for children with autism spectrum disorder. The ABCD SW project (Autistic Behavior & Computer-based Didactic software) [21] aims at designing and developing Open Source didactic software conforming to the ABA learning method to ensure an usable product that responds to these children’s special needs.

2 Related Work

ABA therapy is based on Augmentative and Alternative Communication (AAC) and Discrete Trial Training (DTT). AAC is a technique that by increasing the user’s perceptions provides an alternative method for communicating, and thus is used in treating learning disabilities and neurological pathologies. Creating new technologies for autistic subjects specifically concerns implementing communication through AAC and the behavior analysis (“problem behaviors” and social interactions [22], [23]).

Recent studies have confirmed the efficacy of electronic therapy based on AAC. Hirano et al. [24] designed and implemented a visual scheduling system (vSked) for planning and organizing children’s activities in the classroom, and observed not only increased efficiency for caregivers but also benefits and improvements in student-student and student-teacher communication.

Recently, Pino and Kouroupetroglou have created an Open Source framework for building adaptable, modular, multilingual, cheap and sustainable component-based AAC products [25]. However at the moment the framework is distributed only by the authors.

Mobile communication tools are a promising field in AAC research. Moving AAC from specialized devices to a standard mobile platform offers many advantages: first of all drastic cost reduction, greater flexibility, simpler and faster customization, small size and ubiquity, and familiar environment (cell phone) for the children. Monibi and Hayes implemented a library of virtual cards for autistic children’s activities on a Nokia N800 (Mocoto prototype). The pre-installed card library may be easily extended with pictures or other digital images. A desktop software program allows setup and customization of activities (e.g. size and number of cards, audio cues) [26].

Sampath et al. propose a system for autism using AAC that allows bidirectional communication between child and caregivers. Specifically, a gateway on a handheld device was built, allowing conversion between pictures and spoken language, enabling the completion of the communication loop (receptive and expressive) [27].

Although many digital products are available for augmentative communication (e.g., GoTalk, Tango, Dynavox, Activity Pad), teachers and therapists have experienced low usability and flexibility; training is required for set-up and customization, making it difficult for parents to use it at home. Furthermore, they are expensive [22].

Putnam and Chong investigate the real use at home of software specifically designed for autism. Results of an on-line survey (spread out throughout associations) filled out by 114 responders shed light on the limited diffusion of specific software (only 8%) while software for cognitive disability was used by 25% [28]. This result

514 S. Artoni et al.

indicates the need to move from research prototypes to engineered software specifically designed for autistic children, possibly according to their preferences.

Participatory design actively involves all stakeholders in the design process to help ensure that a created product meets their needs and that results are highly usable [29]. Participatory design is essential in designing for autistic persons. Hirano et al. successful applied participatory design in developing the vSked system [24]. De Leo and Leroy involved special education teachers in designing software for facilitating communication with severely autistic children via smart phones [30].

A large branch of research is devoted to providing usable tools to assist therapists of children with autism. Kientz, Monibi and Hayes based their studies and their software development on participatory design [22], [26]. Kientz et al. [31] designed and developed two systems for facilitating efficient child monitoring (both progress and behavior): 1) Abaris, the supporting team executing Discrete Trial Training therapy, building indices into videos of therapy sessions and allowing easy data search; 2) CareLog, for collecting and analyzing behavioral data (“problem behaviors”). Furthermore sensors were used to monitor stimming behavior (self-stimulatory movements) to understand the cause of an uncomfortable situation.

Hailpern et al. [32] investigated the use of computers for assessing the behavior of nonverbal children. By defining a set of dependent variables for use in video annotation, called A3, it is possible to systematically analyze the interactions of nonverbal children, with the computer capturing feedback related to attention, engagement and vocal behavior.

Recently, pervasive technologies are investigated for monitoring user behavior. To enhance social skills and abilities of persons with ASD, Kaliouby and Goodwin built a suite of wearable technologies (cameras, microphones, sensors) for capturing, analyzing, and sharing (via wireless network) their social-emotional interactions, in a fun and engaging way [23]. Current studies offer usable technologies that favor communication and analysis of data of behaviors. There is a lack of specific studies for the systematic application of instructions regarding ABA, which prioritizes learning through the DTT system by increasing degrees of difficulty.

3 ABA Fundamentals

The first step for participatory design was to acquire knowledge of the ABA method. We interviewed the therapists, watched videos of therapy sessions, and we were also present at ABA sessions with different children and therapists. This was possible only with a few selected children who were able to work in spite of an external presence. The observer stands behind the child at a certain distance and does not interfere in any way with the teaching session. ABA therapy is conducted in a simple environment that offers no distractions for the child. A simple empty table is used for arranging the article on acquisition (to be learnt) and the additional materials (such as distracters) needed for the trial. The teacher sits facing the child, the table between them, and prepares the child’s work environment and then announces the command. The child executes the directive either independently or in response to the therapist’s prompt. A reinforcement may be given after a sequence of trials such as a game, or something else particularly pleasing.

Accessible Education for Autistic Children 515

The teacher decides on which program to work:

1. matching: image to image, image to word, word to image, word to word 2. receptive program (e.g. using the command: “Touch apple”) 3. expressive program (e.g. using the command: “What is this?”).

and consequently, (s)he presents the child with the material (usually hand-made colored square cards), organized in categories. In addition to cards, real elements such as plastic objects, dishes, fruit or other elements may be used. In ABA each program is implemented by using a sequence of trials of increasing degrees of difficulty:

4. Mass Trial: basic trials ensuring children’s success with the item on acquisition in a personalized program

5. Distracter phase: first a neutral distracter is added to the item on acquisition, and next two neutral ones. Then one non-neutral distracter is added, and next, two non-neutral ones.

6. Extended Trials: a choice involving 3 items, 2 must be mastered and the third is on acquisition

7. Random Rotations of learned items. If the program is just at the beginning and 3 items have not been mastered, it is possible to work on a rotation of the 2 items mastered in that program (Block Rotation).

8. Generalization: consists in proposing new ways of acquiring each mastered item. After acquisition, articles are subjected to an abstraction process.

Within this sequence of learning, different types of prompts may be provided, according to the learning needs of the subject, to help him complete the trial successfully.

4 The System

We believe that technology can enhance lives of children with autism, by creating more sophisticated eLearning tools. Our project mainly aims to define an educational ad hoc methodology for teaching autistic children and create didactic computer-based courses in order to render therapy more effective and efficient. The idea is to map ABA principles in creating a specific software program suitable for therapists and children. To simplify child-computer interaction and allow a modality “similar” to physical ABA therapy, we chose to use touch-screen devices and vocal synthesis to announce the commands of the learning modules (exercises).

The software must enable great flexibility and be adaptable to the learning abilities of any autistic child. For this reason we decided to offer an adaptation of the presentation modality of the activities to be carried out, first of all fitting into the topography of the prompt. For example: a child who needs more visual support can be facilitated by the introduction of the written discriminative stimulus (SD) and not only the vocal, pronounced by the speech synthesizer. Caregivers may also speak in order to integrate commands in the best way to stimulate the child. The language required is generally simple and minimal (short sentences without articles, e.g. touch apple,

516 S. Artoni et al.

match yellow). It may take place in more structured sentences depending on the level of difficulty reached by the child and its ability to generalize the concept expressed.

Figure 1 shows a logical scheme of the software architecture of the eLearning environment. The three main software components are:

• Didactic software, i.e. basic ABA programs, for learning categories of articles (such as colors, numbers, etc.). To this aim an intuitive User Interface (UI) was studied, in order to be as clear as possible, easy to use for the less ABA-experienced individuals.

• Monitoring software, as required by the ABA therapy, to control the child’s progress. Computer-based therapy is only one medium for promoting learning but classic face-to-face one-to-one ABA therapy will be alternated. The monitoring software will allow recording data from both face-to-face and computer-child therapy.

• Data analysis software allows conversion of raw data into easy-to-use graphics, showing the child’s learning progress. Web interfaces allow caregivers to easily and rapidly access child learning data, facilitating decisions on how to best direct the educational program and render the learning process more effective.

Fig. 1. Scheme of the ABA eLearning Environment for child with autism

Our project covers all these aspects as a whole and not as separate components. After developing an initial prototype, we will perform a pilot test with a few low-functioning autistic children to perform software tuning and customization.

A crucial aspect is to customize the UI. Each child has personal abilities that vary over a wide range, so the software must be easily configured to meet both specific (receptive communication, expressive communication, etc.) and temporary needs

Accessible Education for Autistic Children 517

(if the child fails a trial on a mastered item, the therapist would immediately jump to a lower level [see Section 3] to reiterate and consolidate the acquisition).

Data from each trial are stored in a database in order to create a path of acquisitions (mastered items), difficulties and prompt adaptation to reach the objective. The historical archive is useful for ensuring greater consistency and coherence between the caregivers in their daily work, and make the subject’s “learning history” available for the scientific analysis of gathered data.

5 The User Interfaces

Participatory design with therapists and caregivers is essential for reaching reliable results. In order to avoid influencing the imagination of therapists we asked them to picture how they image the user interface of the software, without showing them our personal ideas. Figure 2 shows three examples of organization.

Fig. 2. The UIs proposed by therapists: carousel (up), circular icons (bottom left), square icons (bottom right)

Starting from the top one, there is a carousel (for selecting the category on which work), a circle of icons, and icons (arranged in a square) with a search function.

As previously mentioned, the complexity of Discrete Trial Training and teaching by levels increases the complexity of the UI (Fig.3). Another crucial point regards the organization of data related to the teaching session. This information concerns the child, but it is also linked to therapists and programs.

Each performance of each elementary unit is recorded in a database. Data then needs to be aggregated and transformed in graphic form to highlight the progress or weakness of the child and allow directing the therapy as needed.

518 S. Artoni et al.

The generalization applicable to the software is obviously limited compared to what happens during a typical ABA session (without computer). Within the software modules the generalization is carried out changing:

• the Discriminative Stimulus (i.e. the therapist’s command) • the position of items on the screen • the stimulus, the visual features of items (photo, drawing, outline, sketch).

Fig. 3. Programs (matching, receptive or expressive) for the selected category (color)

6 Conclusion

In this paper we have described the design of didactic software dedicated to autistic children, involving the Applied Behavior Analysis. This method was selected since it has scientific validation, is based on structured teaching and is customized.

Numerous studies demonstrate the effectiveness of ABA. The teaching is structured by DTT, follows the “errorless learning” procedure, based on the use of prompts, and adapts the educational program to the child's abilities. The software design of the proposed eLearning environment closely follows the procedure for teaching ABA, suggesting Discrete Trials through levels of increasing difficulty.

The software ensures a high degree of flexibility thanks to its ability to customize the type of prompt to the learning needs of each student. It relies on a consolidated scientific approach since it is based on an effective and validated method for teaching children with disorders of the autistic spectrum. It also provides accurate data collection for monitoring the individual learning path, essential for focusing the educational effort where necessary for the child.

Currently, the system design has been completed, and the user interfaces were drawn with the valuable support of children therapists and parents. The next step will be implementing the didactic software component, which is the core of the eLearning system.

Accessible Education for Autistic Children 519

Acknowledgments. We thank the Regione Toscana that funded this project within the framework of the “FAS 2007 2013 Delibera CIPE 166/2007 PAR FAS Regione Toscana Action Line 1.1.a.3” (Gen 2011-Gen 2013). Furthermore, we thank all therapists who participated in this user interface design, in particular Simone Di Cesare and Francesca Lenci.

References

1. American Psychiatric Association. DSM-IV-TR. Diagnostic and Statistical Manual of Mental Disorders. 4 edn. Text Revision (2000)

2. World Health Organization, The ICD-10. Classification of Mental and Behavioural Disorders. World Health Organization, Geneva (1992)

3. Anderson, S.R., Romanczyk, R.G.: Early Intervention for Young Children with Autism: Continuum-Based Behavioral Models. The Journal of The Association for Persons with Severe Handicaps 24(3), 162–173 (1999)

4. Corsello, C.M.: Early intervention in autism. Infants & Young Children 18(2), 74–85 (2005)

5. Myers, S.M., Johnson, C.P., and the Council on Children With Disabilities: Management of Children With Autism Spectrum Disorders. Pediatrics 120, 1162–1182 (2007), doi:10.1542/peds.2007-2362

6. Maurice, C., Green, G., Luce, S.C.: Behavioral intervention for young children with autism: A manual for parents and professionals (1996)

7. Dawson, G., Zanolli, K.: Early intervention and brain plasticity in autism. In: Novartis Found Symp., vol. 251, pp. 266–74 (2003); discussion 274–80, 281–97

8. Baron-Cohen, S., Scott, F.J., Allison, C., Williams, J., Bolton, P., Matthews, F.E., Brayne, C.: Prevalence of autism-spectrum conditions: UK school-based population study. The British Journal of Psychiatry 194, 500–509 (2009)

9. Leaf, R., McEachin, J.: A work in progress: Behaviour Management Strategies and a Curriculum for Intensive Behavioral Treatment of Autism, pp. 1417–1418 (1999)

10. Lovaas, O.I.: Teaching developmentally disable children: The me book (1981) 11. Lovaas, O.I.: Behavioral treatment and normal education and intellectual functionating in

young autistic children. Journal of Consulting and Clinical Psychology 55, 3–9 (1987) 12. Rogers, S.J.: Brief Report: Early Intervention in Autism. Journal of Autism and

Developmental Disorders 26(2) (1996) 13. Fenske, E.C., Zalenski, S., Krantz, P.J., McClannahan, L.E.: Age at intervention and

treatment outcome for autistic children in a comprehensive intervention program. Analysis and Intervention in Developmental Disabilities 5, 49–58 (1985)

14. Rosenwasser, B., Axelrod, S.: The contribution of applied behavior analysis to the education of people with autism. Behavior Modification 25(5), 671–677 (2001)

520 S. Artoni et al.

15. Savner, J.L., Myles, B.S.: Making Visual Supports Work in the Home and Community: Strategies for Individuals with Autism and Asperger Syndrome. Autism and Asperger Publishing Co., Shawnee Mission (2000)

16. Eikeseth, S.: Outcome of comprehensive psycho-educational interventions for young children with autism (2008)

17. Sallow, G., Graupner, T.: Intensive behavioural Treatment for Children with Autism: Four year outcome and Predictors. American Journal on Mental Retardation 110(6), 417–438 (2005)

18. Maine Administrators of Services for Children with Disabilities, Autism Task Force Report (2000), http://www.madsec.org/

19. Weiss, M.J.: Expanding ABA intervention in intensive programs for children with autism: The inclusion of natural environment training and fluency based instruction. The Behavioural Analyst 2(3), 182–186 (2001)

20. Rosenwasser, B., Axelrod, S.: More contributions of applied behavior analysis to the education of people with autism. Behavior Modification 26(1), 3–8 (2002)

21. Artoni, S., Buzzi, M.C., Buzzi, M., Fenili, C.: Didactic software for autistic children. In: Cipolla Ficarra, F.V., de Castro Lozano, C., Pérez Jiménez, M., Nicol, E., Kratky, A., Cipolla-Ficarra, M. (eds.) ADNTIIC 2010. LNCS, vol. 6616, pp. 73–80. Springer, Heidelberg (2011)

22. Hayes, G.R., Hirano, S., Marcu, G., Monibi, M., Nguyen, D.H., Yeganyan, M.: Interactive visual supports for children with autism. Springer Personal and Ubiquitous Computing, 18 (2010), doi: 10.1007/s00779-010-0294-8

23. Kaliouby, R., Goodwin, M.S.: iSET: Interactive Social-Emotional Toolkit for Autism Spectrum Disorder. In: Proceedings of IDC 2008, pp. 77–80 (2008)

24. Hirano, S., Yeganyan, M.T., Marcu, G., Nguyen, D.H., Boyd, L.A., Hayes, G.R.: vSked: Evaluation of a System to Support Classroom Activities for Children with Autism. In: Proceedings of CHI, pp. 1633–1642 (2010)

25. Pino, A., Kouroupetroglou, G.: ITHACA: An Open Source Framework for Building Component-Based Augmentative and Alternative Communication Applications. ACM Transactions on Accessible Computing (TACCESS) 2(4), art. no.: 14, 30 pp. (2010)

26. Monibi, M., Hayes, G.R.: Mocotos: Mobile Communications Tools for Children with Special Needs. In: Proceedings of IDC 2008, pp. 121–124 (2008)

27. Sampath, H., Sivaswamy, J., Indurkhya, B.: Assistive systems for children with dyslexia and autism. ACM SIGACCESS Accessibility and Computing (96) (2010)

28. Putnam, C., Chong, L.: Software and technologies designed for people with autism: what do users want? In: Proceedings of the 10th ACM SIGACCESS Conference, pp. 3–10 (2008)

29. Wikipedia, Participatory Design, http://en.wikipedia.org/wiki/Participatory_design

30. De Leo, G., Leroy, G.: Smartphones to facilitate communication and improve social skills of children with severe autism spectrum disorder: special education teachers as proxies. In: Proceedings of the 7th Int. Conf. on Interaction Design and Children (IDC 2008), pp. 45–48 (2008)

31. Kientz, J.A., Hayes, G.R., Westeyn, T.L., Starner, T., Abowd, G.D.: Pervasive Computing and Autism: Assisting Caregivers of Children with Special Needs. IEEE Pervasive Computing 6(1), 28–35 (2007)

32. Hailpern, J., Karahalios, K., Halle, J., Dethorne, L., Coletto, M.: A3: HCI Coding Guideline for Research Using Video Annotation to Assess Behavior of Nonverbal Subjects with Computer-Based Intervention. ACM Trans. Access. Comput. 2(2), Article 8, 29 pages (2009)