Virtual environments supporting learning and communication in special needs education

LWW/TLD LWWJ350-02 July 24, 2007 6:14 Char Count= 0

Top Lang DisordersVol. 27, No. 3, pp. 211–225Copyright c© 2007 Wolters Kluwer Health | Lippincott Williams & Wilkins

Virtual EnvironmentsSupporting Learning andCommunication in SpecialNeeds Education

Sue V. G. Cobb, PhD

Virtual reality (VR) describes a set of technologies that allow users to explore and experience3-dimensional computer-generated “worlds” or “environments.” These virtual environments cancontain representations of real or imaginary objects on a small or large scale (from modeling ofmolecular structures to buildings, streets, and scenery of a virtual city). Potential use in educationwas first considered in the early 1990s, when it was suggested that VR technology could providepowerful learning environments not available through other means (M. Bricken, 1991). A grow-ing VR research community has since sought to examine the benefits of using this technology inmainstream and special needs education, as well as other learning environments. This article firstpresents a brief description of VR technology and its use in teaching and learning. VR can be usedas a tool for communication, as a medium through which individuals express ideas, and for learn-ing about communication, through guided training and rehearsal in simulated social scenarios.This article concentrates on the latter application and presents 4 research projects conducted bythe Virtual Reality Applications Research Team and associated colleagues in special needs educa-tion since 1991. These projects involved the development and evaluation of virtual environmentapplications intended to support training for individuals with learning needs and communicationdifficulties in preparation for more independence in their everyday activities and communicationsin the community. Key words: communication, social skills, special needs education, task-basedlearning, virtual environments, virtual reality

VIRTUAL ENVIRONMENTS

Virtual reality (VR) refers to a set of com-puting technologies used to create, and al-low users to experience, three-dimensional

From the University of Nottingham, Nottingham,United Kingdom.

This article is based on a series of research studies con-ducted between 1991 and 2003 by members of the Vir-tual Reality Applications Research Team (VIRART) andcollaborating partners.

The author thanks all past and current members ofVIRART, collaborating partners, professionals, parents,students, and end-users who contributed to the projectsdescribed in this article.

Corresponding author: Sue V. G. Cobb, PhD, VIRART, Hu-man Factors Research Group, School M3, University ofNottingham, Nottingham, NG7 2RD, United Kingdom(e-mail: [email protected]).

(3D) simulated digital or “virtual” environ-ments (VEs). A VE is initially empty 3D com-puter space that then is populated with 3Dobjects that are given behavioral characteris-tics replicating some aspect of real-world be-havior (e.g., a doorbell chimes, a door handleturns, a door opens). A 3D navigation con-troller, such as a joystick, is used to movearound the virtual world, and a cursor con-troller, such as a standard computer mouse, isused to interact with virtual objects. Interac-tion will cause a response in the VE in accor-dance with programmed object behaviors. Toexplore a virtual house, for example, a usermay move toward a door by pushing forwardon the joystick controller. Placing the mousecursor over the door handle, she can clickand drag to turn the handle (the door opens),and then use the joystick to move through thedoor into another room.

211


212 TOPICS IN LANGUAGE DISORDERS/JULY–SEPTEMBER 2007

These VEs are distinct from traditional com-puter programs in several ways; the digitalenvironment is 3D rather than 2D, users canmove around in the digital environment in anydirection they choose rather than followinga preset route, users may use a special dis-play to become “immersed” in the digital en-vironment “as if they were really inside it.”Features of VEs useful to education includethe following (Cobb, Neale, Crosier, & Wilson,2002):

• Representation of objects and environ-ments: These may be representations ofreal things that exist in the real worldor things that do not exist or cannot or-dinarily be seen (such as molecules orparticles).

• Different viewpoints: Users can take dif-ferent perspectives and travel around theVE appropriately (e.g., as a wheelchairuser, eye-height would be at a fixed seatedlevel and movement control restricted toforward/back with limited rotation. As ahelicopter pilot, the viewpoint would beable to “fly” around the VE using the full6 df of movement).

• Reality and super-reality: VEs allow be-haviors not normally possible in real time(e.g., to “fly”inside a machine to view theinner workings to gain an understandingof how it works).

VEs may be explored via a standard desk-top computer or projected screen (similar tostandard computer use), or via immersive dis-play devices. Using these latter devices, userswear a head-mounted display system so thatthey see only the VE. When they move theirhead from side to side, it is as if they are look-ing around inside the VE. VEs may be designedfor single-user interaction (SVEs), in which asingle-user controls navigation and interactionwith the objects in the VE. In collaborative vir-tual environments (CVEs), several users sharethe same VE and each user is represented byan avatar: a virtual character that can be seenby other users. Users may communicate witheach other via audio headphones with micro-phones and collaboratively interact with vir-tual objects.

VIRTUAL REALITY IN EDUCATION

VR was first discussed as a possible mediumfor mainstream teaching in the early 1990swhen it was suggested that characteristics ofVR learning environments, specifically the fa-cility to move around a simulated environ-ment and explore conceptual information inan intuitive manner, could offer powerfullearning experiences not available throughother means (Bricken, 1991). Numerous stud-ies throughout the 1990s considered the util-ity of VR technology in mainstream and spe-cial needs education and were reported in avariety of publications including a new jour-nal, VR in the Schools (Auld & Pantelidis,1995), the Virtual Reality in Education andTraining conference (VRET, 1997), a specialissue of the VR journal Presence: Teleoper-ators and Virtual Environments (1999), aswell as other education journals. Reportedprojects examined the use of VR for teach-ing mainstream curriculum subjects includingmathematics (Bricken & Winn, 1992), science(Byrne & Furness, 1994; Dede, Salzman, &Bowen Loftin, 1996; Salzman, Dede, BowenLoftin, & Chen, 1999), and engineering (Bell& Fogler, 1995).

The first substantial review of educationaluses of VR technology was reported in 1998.Citing more than 50 studies, Youngblut (1998)concluded that there was some evidence ofimprovements in learning from using VR. Inthe ScienceSpace project, for example, a se-ries of VEs was constructed for teaching sci-ence relating to physics (Newton World), elec-trical fields (Maxwell World), and chemistry(Pauling World). Subject knowledge tests ap-plied to 18 students studying high schoolphysics found that learning in 3D immer-sive Maxwell World was significantly betterthan with traditional teaching methods (Dedeet al., 1996; Salzman et al., 1999). A more re-cent review of VEs used in mainstream educa-tion (Moshell & Hughes, 2002) defined differ-ent ways in which VEs may support learning;constructivist learning may be supported viastudent self-directed exploration of a VE (e.g.,the Virtual RadLab radioactivity laboratory


Virtual Environments for Learning and Communication 213

for testing the shielding properties of dif-ferent materials with a range of radioactivesources; Crosier, Cobb, & Wilson, 2000), con-structionist learning may be supported whenstudents work together to create their ownvirtual models (e.g., construction of the SolarSystem to illustrate how eclipses occur; Barabet al., 2000), and situated learning may be sup-ported by encouraging interactive role-playin simulated scenarios (e.g., the ExploreNet2D collaborative improvisational drama simu-lation; Moshell & Hughes, 1996).

The first conference on virtual reality anddisability (Murphy, 1993) identified poten-tial application of the technology for improv-ing accessibility, mobility, learning, rehabili-tation, and assessment. Many of these havesuccessfully been demonstrated at the now es-tablished international conference series ondisability, virtual reality, and associated tech-nologies (ECDVRAT, 1996). An extensiverange of assistive devices and display systemshas been developed to allow access to mul-tisensory digital media used for a variety ofapplications including assessment and reha-bilitation, behavior therapy and phobia treat-ment, and training and education for individ-uals with disabilities, and there is growingevidence of successful application of VEsand VR technologies in these areas (Cobb &Sharkey, 2007). A review of VEs used in train-ing and education of people with intellectualdisabilities highlights specific advantages ofVEs in providing a safe place for people tolearn about and practice skills, behaviors andactivities that they may need to do in the realworld, such as preparation for a courtroom ap-pearance (Standen & Brown, 2005). Transferof training has been found following the useof VEs for vocational training of students withintellectual disabilities in kitchen skills (Rose,Brooks, & Attree, 2000) and workplace tasksin sheltered factories (Mendozzi et al., 2000).

VIRART PROJECTS

The Virtual Reality Applications ResearchTeam (VIRART) was established at the Univer-sity of Nottingham, UK, in 1991 to explore

and develop applications of VR technology. Aspecific area of research interest was the useof VR technology for education and training.Demonstrator projects have been developedfor industrial training, rehabilitation and main-stream, and special needs education (VIRART,2007). This article presents an overview offour projects aimed at supporting learningand communication skills for children andadults with special educational needs (SEN).1

One of the challenges in design of VEsfor SEN was that there were no guidelinesavailable to inform content and interface de-sign, and very little understanding of how in-dividuals would interpret and react to VEs.User-centered design methods were appliedand evolved to ensure that the VEs devel-oped were appropriate for the intended usersand teaching professionals. Throughout thisperiod of application development and ex-ploration, our research team has addresseda number of research aims including thefollowing:

-- Design and development of VEs to meetuser requirements for special populations

-- User understanding of the technologyand what the VE represents

-- Observation of user interaction with VE-- Evidence of learning gain or other benefit

and transfer of learningTable 1 presents a summary of the selected

projects, listing the VEs developed, main re-search questions, and outcomes. Detailed de-scriptions of these projects have been pub-lished previously, and readers are invited torefer to cited papers in Table 1 for furtherinformation. The first three projects wereconducted in partnership with the ShepherdSchool, a UK specialist school for childrenwith severe and profound learning disabili-ties. The last project involved the RosehillSchool and Sutherland House for children

1In the context of this article SEN includes severe intellec-

tual or cognitive disability. Readers should be aware that,

in this article and others describing this work, the term

severe learning difficulties (SLD) also refers to individu-

als with intellectual or cognitive disabilities.


214 TOPICS IN LANGUAGE DISORDERS/JULY–SEPTEMBER 2007T

ab

le1

.Su

mm

ary

of

VIR

AR

Tp

roje

cts

exp

lori

ng

vir

tual

en

vir

on

men

ts(V

Es)

for

lear

nin

gan

dco

mm

un

icat

ion

Pro

ject

an

dV

Es

Pu

bli

cati

on

sR

ese

arch

qu

est

ion

sO

utc

om

es

Ex

peri

en

tial

VEs

for

child

ren

wit

h

specia

ln

eed

s(1

99

3–9

5)

Vir

tual

car

dri

vin

g

Vir

tual

skisl

op

e

Vir

tual

ho

use

Vir

tual

sup

erm

arket

Bro

wn

,K

err

,an

dW

ilso

n(1

99

7)

Can

child

ren

wit

hle

arn

ing

and

mu

ltip

led

isab

ilit

ies

co

ntr

ol

inte

racti

on

wit

hin

avir

tual

en

vir

on

men

t?

Ch

ild

ren

co

uld

inte

ract

wit

hV

E

Ch

ild

ren

dis

pla

yed

en

joym

en

t

Teac

her

reac

tio

np

osi

tive

Usa

bilit

yis

sues

rais

ed

Vir

tual

en

vir

on

men

tsfo

rte

ach

ing

sign

lan

gu

age

and

Mak

ato

nsy

mb

ols

(19

93

)

Mak

ato

nth

em

ed

“war

eh

ou

ses”

Bro

wn

et

al.(1

99

7)

Bro

wn

,C

ob

b,an

dEas

tgat

e(1

99

5)

Can

we

success

fully

bri

ng

the

ou

tsid

e

wo

rld

into

the

sch

oo

lcla

ssro

om

?

Ho

ww

illth

issu

pp

ort

stu

den

t

lear

nin

g?

Som

ech

ild

ren

reco

gn

ized

ob

jects

,

wh

ere

aso

thers

had

dif

ficu

lty

Ch

ild

ren

co

pie

dvir

tual

sign

er

and

rep

eat

ed

wo

rds

Peer

tuto

rin

go

ccu

rred

Inte

rfac

ed

esi

gn

and

need

for

rew

ard

syst

em

Usa

bilit

yis

sues

rais

ed

Lif

esk

ills

ed

ucat

ion

(19

98

–9

9)

Vir

tual

ho

use

Vir

tual

tran

spo

rt

Vir

tual

caf

e

Vir

tual

sup

erm

arket

Meak

inet

al.(1

99

8)

Bro

wn

,K

err

,an

dB

ayo

n(1

99

8)

Co

bb

,N

eal

e,an

dR

eyn

old

s(1

99

8)

Neal

e,B

row

n,C

ob

b,an

dW

ilso

n(1

99

9)

Bro

wn

,N

eal

e,C

ob

b,an

dR

eyn

old

s

(19

99

)

Neal

e,C

ob

b,an

dW

ilso

n(2

00

0)

Wh

atlife

skills

do

stu

den

tsn

eed

to

lear

n?

Ho

weff

ecti

veis

the

VE

inte

rms

of

Usa

bilit

y

En

joym

en

t

Skillle

arn

ing

Skills

tran

sfer

Stu

den

tsfo

un

dit

eas

ier

tou

seV

Es

than

men

tors

believe

dth

ey

wo

uld

Co

ntr

olac

tio

nan

dd

ecis

ion

-mak

ing

shif

tfr

om

teac

her-

init

iate

dto

stu

den

t-in

itia

ted

An

ecd

ota

lre

po

rts

of

ben

efi

tto

perf

orm

ance

Stu

den

tfe

ed

bac

ko

nu

ser-

cen

tere

d

desi

gn

meth

od

po

siti

ve

Usa

bilit

yis

sues

rais

ed

Socia

lsk

ills

trai

nin

gfo

rad

ole

scen

ts

wit

hau

tist

icsp

ectr

um

dis

ord

er

(ASD

)(2

00

0–0

3)

Sin

gle

-use

rV

Es

Vir

tual

bu

s

Vir

tual

caf

e

Co

llab

ora

tive

VEs

Socia

lcaf

e

Inte

rvie

wro

om

Par

son

set

al.(2

00

0)

Co

bb

et

al.(2

00

2)

Neal

eet

al.(2

00

2)

Kerr

,N

eal

e,an

dC

ob

b(2

00

2)

Ru

tten

et

al.(2

00

3)

Par

son

san

dM

itch

ell

(20

02

)

Par

son

s,M

itch

ell,an

dLeo

nar

d(2

00

4)

Par

son

s,M

itch

ell,an

dLeo

nar

d(2

00

5)

Can

stu

den

tsw

ith

ASD

inte

ract

wit

h

VEs?

Do

stu

den

tsw

ith

ASD

un

ders

tan

dV

Es

asre

pre

sen

tati

on

so

fso

cia

l

scen

ario

s?

Isth

ere

any

evid

en

ce

of

socia

lsk

ills

lear

nin

g?

Isth

ere

any

evid

en

ce

of

skills

tran

sfer

betw

een

co

nte

xts

?

Stu

den

tsw

ith

ASD

lear

ned

tou

seV

Es

qu

ickly

Inte

rpre

ted

the

VE

no

nlite

rally

Reco

gn

ized

bas

icd

iffe

ren

ces

betw

een

VEs

and

vid

eo

ed

scen

es

Stu

den

tsle

arn

ed

app

rop

riat

ere

spo

nse

s

wit

hin

co

nte

xt

Tra

nsf

er

of

lear

nin

gd

em

on

stra

ted

betw

een

med

ia(V

Eto

vid

eo

)w

ith

in

co

nte

xt

No

tran

sfer

of

lear

nin

gb

etw

een

co

nte

xts



Figure 1. Virtual skiing.

with autistic spectrum disorder (ASD) andthe UK’s National Autistic Society (NAS). Sum-mary and outcomes of the projects are dis-cussed below.

Experiential VEs for children with

special needs

At the outset, teachers suggested that oneof the most appealing features of VR was thatit could be used to bring the outside worldinto the classroom. It was considered that aVE could model places and activities that stu-dents with severe learning difficulties (SLD)may not have access to perhaps in their life-time and so the VE could provide some ex-periences that would normally be beyond thereach of these students. A set of experientialVEs was developed that was intended to al-low the user to control movement within VE.These included a virtual ski slope (Figure 1)and a virtual city (Figure 2). The intended pur-pose of these VEs was to provide students

Figure 2. Virtual driving.

Figure 3. Student and facilitator.

with the opportunity to control the environ-ment and to explore features with which theycould interact.

One of the primary research aims was de-termining the suitability and ease of use of theinput devices. Observation of students usingalternative control devices concluded thatspecialist VR-control devices were not suit-able for students with SLD as they were toosensitive and difficult to control (Hall, 1993).A further study observed six students, us-ing standard ICT control devices (joystick,mouse, and keyboard), and found that mousecontrol of navigation was extremely frustrat-ing for students with SLD. Standard joystickcontrollers were found to be much more suit-able for controlling movement around the VE(Crosier, 1996).

Observation of how these early VEs wereused in the classroom revealed that studentsdid not use the VE on their own, but were sup-ported by a facilitator or teacher seated along-side them (Figure 3). Observation of student–teacher interactions, when using the experi-ential VE, found that individual characteristicshad more of an effect on user behavior thandid VE design, with individual students’ abili-ties influencing their awareness of position inthe VE, whether they needed instruction forthe next task, and their contribution to col-laboration (Neale, 1997). Conclusions drawnfrom these early studies were that studentswith SLD appeared to enjoy having controlover the VE and could successfully interactwith it but required support from a colocated



facilitator to guide them through interactivetasks (Neale et al., 1999).

VEs for teaching sign language and

Makaton symbols

Development of the experiential VEs haddemonstrated the feasibility of using VR to“bring the outside world into the classroom.”Teachers considered that a useful applicationof this finding would be to support teach-ing and learning of the Makaton languageof symbols and signs used for communica-tion throughout the school. Makaton com-prises a library of 350 picture-based icons(Grove & Walker, 1990) and teaching of Maka-ton is traditionally achieved by association—showing an object, stating the name of theobject, showing the Makaton symbol repre-senting the object, and then performing thesign. One of the problems highlighted washow to teach associations for objects that can-not be brought into the classroom (e.g., ve-hicles, buildings). Traditionally, teachers over-came this by using representations of realitems such as toys or photographs, but theyexpressed concern that students may not un-derstand the function of objects if they couldnot explore and interact with them. It seemedsensible to utilize VR to accomplish the task,allow students to explore a VE, and gain an un-derstanding of object function by interactingwith virtual objects.

The Makaton project was constructed us-ing a “split screen” interface design compris-ing on one side an explorable VE containinga variety of virtual objects and, on the other,a permanent representation of the Makatonsymbol for the object of interest and an ani-mated virtual signer. Figure 4 shows the “boatwarehouse”—one of the four components inthe transport VE. The student could movearound the VE to view the 3D objects fromdifferent angles and could also activate in-teractive features (e.g., the boat sails aroundthe VE). Placing the cursor in the box con-taining the Makaton symbol triggered an au-dio file stating the word represented (in thiscase “boat”). If the cursor was placed in thebox containing the manikin, then the virtual

Figure 4. VE, Makaton symbol, and signer repre-

senting “boat.”

signer would perform the appropriate signfor that word. Thus, the student could acti-vate the signer and listen to the word as manytimes as necessary.

Evaluation of the Makaton project took dif-ferent forms. Multiple activity analysis exam-ined interaction behavior between student–teacher pairs and found that, for this appli-cation, there was not much need for teach-ers to guide interaction (Neale et al., 1999).Although some students found it difficult touse the mouse to control the cursor, mostdemonstrated that they knew what to do.Spontaneous peer tutoring also was observed.A teacher commented that she had seen “anolder student guiding a younger and lessable student through a Makaton environment,helping her to form the hand sign and say theword associated with each symbol. The ad-vantages for both are manifold”(Brown, 1994,p. 7).

A later experimental study conducted in-dependently found that, with practice, stu-dents took more control over their interac-tion with the VE and that their Makaton vo-cabulary increased (Standen & Low, 1996).Although the interface itself was easier touse than in the previous project, possibly be-cause the activities in this VE were more struc-tured, usability issues were identified partic-ularly relating to the system feedback to stu-dents. This was resolved by introducing a pos-itive reward system for correct choices and



encouragement to try again when incorrectchoices were selected.

VEs for learning independent

living skills

The practice element of exploration and in-teraction in a virtual world was regarded asa potentially useful mechanism for studentsto learn how to engage in activities for them-selves. The Life Skills Education project wasa community-based research project intendedfor students with SLD aged 16+ to learn aboutand practice skills needed for independent liv-ing (Brown et al., 1999). A virtual city wasconstructed comprising four main elements:a virtual house, virtual supermarket, and vir-tual cafe linked together by a virtual transportsystem.

Students were given a task scenario that re-quired them to make decisions and performtasks in the virtual world, including makinga shopping list and planning a bus journeyto the supermarket to buy food. At the busstop, they needed to compare the name onthe front of the bus with that displayed in theinformation bar at the bottom of the screento check that they were getting on the cor-rect bus (Figure 5). In the virtual supermarket(Figure 6), the viewpoint was fixed so that thestudents could move around the supermar-ket pushing the shopping trolley in front ofthem. A split-screen interface was used fromwhich the shopping list could be selected tofull-screen view allowing the users to checkwhat items they needed to find (Figure 7).

Figure 5. At the bus stop.

Figure 6. Virtual Supermarket.

A user-centered design methodology in-volved users directly in design decisions con-cerning content and the VEs (e.g., the choiceof learning scenarios) and evaluation of the VEinterface (Meakin et al., 1998). Teachers andtraining professionals informed the structureand presentation of tasks embedded withinthe learning scenarios.

Evaluation studies examined usability, en-joyment, demonstration of skill learning, andtransfer of learning to real-world activities.Observational analysis, questionnaires, and in-terviews were used to assess usability andenjoyment. An experimental study was con-ducted in which 14 student–teacher pairswere observed in task performance in fiveweekly sessions using the VE compared withpre- and post-VE task performance in the real

Figure 7. Virtual shopping list.



world (Cobb, Neale, & Reynolds, 1998). Foreach task component the level of supportprovided by the teacher was recorded. Thisranged from “no support given,”through “ver-bal,” “visual,” and “physical prompts,” to “theteacher does the task.”The importance of thismeasure was that it allowed a comparison be-tween the support given in the VE and sup-port given to carry out the same task in thereal world. It was important to ensure that thetraining tool used was not more difficult to usethan to carry out the task in the real world. Ifthis were the case, it would invalidate the useof the VE.

The results showed that the students foundthe programs accessible and enjoyable. Oneinteresting result was that students with SLDperformed much better at VE interaction thantheir support workers had predicted (Cobbet al., 1998). This result might reflect a genera-tion gap between students and their teacherswith regard to acceptance of new technolo-gies, highlighting the importance of user in-volvement in design and evaluation. Positivefeedback was received with support work-ers commenting upon improved confidenceand enthusiasm for real-world activities afterthey had used the VE for training. Students re-ported that the VE had helped them make de-cisions and know what to do in different situa-tions. These anecdotal reports suggest poten-tial for VE training to transfer into real-worldbehavior.

Observation of student behavior, while us-ing the VE, led to the identification of threetypes of usability problems: understandingtask instructions, moving around the VE, andinteracting with virtual objects (Neale et al.,2000). For example, one of the tasks in thevirtual cafe was to find a table to sit at. To suc-cessfully achieve this task, students needed tounderstand that they had to look for an emptyseat at one of the tables, use the joystick tomove their viewpoint to the table, and thenuse the mouse to select the seat that theywanted to sit on. Classification of problemsexperienced helped identify issues with thedesign of the VE, and solutions to usabilityproblems were implemented to ensure wider

accessibility. For example, many users experi-enced problems reading texts when creatingtheir shopping list, reading menus, and whentext overlays were presented to them. Thiswas resolved by replacing text menus withpictures of the items on the shopping list (asseen in Figure 7), making the search task eas-ier for students as they could directly matchthe picture with objects on the shelves.

VEs for social skills training

The Life Skills Education project haddemonstrated that VEs could successfully beused for learning of procedural tasks and thatstudents with SLD could transfer learning toreal-world behavior (Cobb et al., 1998). TheAsperger’s Syndrome (AS) Interactive projectexamined the suitability of VR technology andVEs to support learning of social communi-cation skills for teenagers and adults withASD (Cobb et al., 2002). This was anothercommunity-based project with partners fromthe UK National Autistic Society (NAS), autismtraining professionals, an autism specialist,and other special needs schools in Notting-hamshire, UK.

The primary characteristics of ASD includeinappropriate use of language, overly literal in-terpretation of words and phrases, and lim-ited understanding of social norms and ex-pectations leading to inappropriate behaviorin social contexts (Wing, 1996). It has beensuggested that VEs might be an ideal mediumthrough which individuals with ASD can learnbehavioral skills (Clancy, 1996) and early stud-ies indicate successful interpretation of VEsimulated scenarios (Strickland, 1996). The ra-tionale behind the AS Interactive project wasthat, if social scenarios could successfully bereplicated within VEs, the limited personalinteraction afforded by the computer inter-face would be inherently more attractive tochildren with ASD and therefore provide asafe and supportive environment for learning(Parsons et al., 2000). In addition to the useof VEs for individual student training usingsingle-user VEs (SVEs), as had been used in theprevious projects, it was considered that thisproject would benefit from the use of CVEs,



Figure 8. Virtual cafe.

allowing several users to share the same VE.One of the objectives of the AS Interactiveproject was to examine the feasibility of usingCVE technology for social skills training andinteraction in ASD schools.

Two scenarios were chosen representingtypical social situations that would be famil-iar to most users (a cafe and a bus). The ob-jective was to support social interaction be-havior specific to two tasks: queuing (i.e.,taking one’s place in line) and finding some-where to sit. This required students to controlmovement of their virtual character (“avatar”)through the VE, to respond appropriately toother avatars, to make decisions about whenthey should communicate with others in theVE, and decide what they should say. Again, asplit screen interface was used, this time withthe VE in the central part of the screen. Thedifficulty level for the VE was selected fromtabs at the top of the screen, and instructions,prompts, and feedback presented on the in-formation bar at the bottom of the screen.Figure 8 shows the virtual cafe. Dependingupon the VE level selected, there were emptytables (providing the option to sit down ata free table) or empty seats only (so that itwould be necessary to sit down at a table withother people). To sit down at an empty table,the student had to navigate to the table andclick a chair to sit on it. However, it was notpermissible to sit at an occupied table with-out asking a question to the avatars seated atthe table. A choice of questions was presented(Figure 9), and if the student selected an ap-

Figure 9. Choosing what to say.

propriate question, then he or she could com-plete the task.

The SVE was designed to support learn-ing with regard to asking appropriate ques-tions and interpretation of responses. For ex-ample, a user might approach a table withthree avatars seated and one empty seat andask whether she could sit down. Sometimesthe response would be “yes”but at other timesshe would be told “No, that seat is taken.”Classroom observation found that teacherswere using this program for group teachingsessions, using a projection screen so thatup to six students could view the SVE whileone student took control over navigation andinteraction with the VE (Neale, Cobb, Kerr,& Leonard, 2002). Teachers commented thatthe VE provided a visual tool that could beused to support teacher-led discussion withthe students about what to do in social situ-ations. In the example above, an empty seatwas described as “taken.”Using a “pause”but-ton to “freeze”the program, leaving the visualscene on the screen, the teacher could discusswith students what this response might mean.At other points of the program, this facilitywas used to stop the program and ask stu-dents what they thought that characters in theVE might be thinking or what they thoughtwould happen next.

It was expected that CVEs may offer a dif-ferent type of learning support than the SVEdue to the facility for interaction with otherpeople participating in the shared VE. Thus,this type of interaction would be closer to



Figure 10. CVE social cafe.

social interaction but, instead of direct face-to-face contact with another person, the stu-dent would communicate with them indi-rectly through the medium of the VE inter-face. The social cafe was constructed as aCVE replication of the single-user virtual cafe(Figure 10). Students were asked to find theirfriend and then buy a drink. One of the ad-vantages of the CVE was that a teacher couldfacilitate student learning in a more naturalway, themselves acting as an avatar in the VEand leading conversations, rather than sittingalongside users offering them prompts for in-teraction choices in the SVE. In the case ofCVE, a facilitator played the role of a “friend”in the VE. In this situation, the teacher andthe student colocated the VE; however, inthe real environment, they were seated in dif-ferent rooms. User testing of the CVEs con-ducted with adults with Asperger’s syndromeobserved that they successfully navigated theCVE and appeared to interpret the VE asrepresentative of the real world. However, itwas not possible to examine social interactionskills as the participants avoided each other inthe CVE (Rutten et al., 2003). Attempts to con-duct CVE trials in schools found that the tech-nology was not sufficiently robust to worksuccessfully in situ. It is hoped that as thesetechnologies mature, they will become suffi-ciently stable and portable to be tested in ateaching environment.

The SVE tasks of queuing (joining a line)and finding somewhere to sit also were pre-sented in a different context—traveling on abus (Figure 11). In this example, two avatarsare already seated on the bus, but there arelots of empty seats. Figure 12 shows the view-

Figure 11. Virtual bus.

point from the user’s seat once he or she hassat down. As can be seen, it is apparent thatthe user chose to sit next to one of the avatars.This was not prevented (as it would be in thevirtual cafe) as it is more permissible to sitwhere you like on public transport. However,although the avatar in this scenario makes nocomment to the user, in this situation the per-son would most likely be wondering why theuser chose to sit next to him when there wereother seats available. The information bar dis-plays the thoughts of the avatar.

The AS Interactive project used a com-bination of experimental and qualitative ap-proaches to investigate use and understandingof VEs and the potential they offer for learningof specific social skills. At the outset it wasnot clear whether ASD students would rec-ognize the VE as a representation of a real-world scenario and whether their behavior

Figure 12. What are they thinking?



would reflect their understanding and behav-ior in the real world. An experimental studycompared responses from ASD students withmatched students with and without learningdifficulties when asked to describe objects,characters and activities in the VE and videoimages of real-world equivalents (Parsonset al., 2004). This study found that most ofthe students with ASD did interpret the VEin a nonliteral way and were able to verbal-ize differences between the real world and theVE representation. However, in a follow-upstudy, students with ASD, low verbal IQ, andweak executive function skills demonstratedlimited understanding of the VE and requiredadditional support from a facilitator to com-plete tasks (Parsons et al., 2005). Although stu-dents could not generalize their learning toa different social context (e.g., cafe to bus),successful skill learning and transfer was evi-dent from VE to real behavior within the samesocial context (Mitchell, Parsons, & Leonard,2007). It was concluded that VEs could beused for education and training but not asa stand-alone teaching method. Teaching ex-pertise is required to ensure appropriate usefor individual learners (Neale, Cobb, & Wil-son, 2001). Following this project, anecdo-tal reports also indicated that use of the VEcould improve student confidence and learn-ing (Wiederhold, 2004).

DISCUSSION

This article has presented a summary ofprogressive projects aimed at exploring theapplication of VR technology to develop VEssupporting learning and communication inspecial needs education. As VR is an emergingtechnology, initial research questions wereconcerned with utility and accessibility; couldthis medium be used as a teaching resource inschools, and would students with SEN be ableto use it successfully? The studies presented inthis article provide evidence that VE technol-ogy is accessible to (and acceptable by) stu-dents with special education needs, includingSLD (intellectual/cognitive impairment) andASD.

It is important to ensure that learningwithin a VE is not more difficult than it wouldbe through other means (i.e., that control ofthe VE itself does not present a barrier tolearning). Contrary to expectations of learn-ing facilitators, the initial studies found thatstudents with SLD could use 3D control de-vices to move around the VE. More recent re-search has found that dual control devices areconfusing for users with learning difficultiesand that it is better to separate control-actionby using a joystick for navigation and mousefor interaction selection (Standen, Brown, An-derton, & Battersby, 2004).

Positive outcomes were reported in all ofthe projects, and anecdotal evidence indicatessome degree of learning following use of VEs.However, it was not possible within thesestudies to comment with any degree of au-thority concerning what type of learning issupported by VEs for special education needs.One reason for this was the methodologicalapproach taken—it was unrealistic to expectthat use of any training medium over a shortperiod would generate demonstrable learningacquisition, particularly for students with SLD.Subsequent studies conducted by other re-search teams have found evidence of learninggain resulting from the use of these and similarVEs. Standen and Low (1996) observed an in-crease in Makaton vocabulary in students withSLD, and Standen, Cromby, and Brown (1998)found that the use of a VE for rehearsal of asimilar supermarket shopping task resulted insignificantly faster and more correct selectionof items in performance of a real-world shop-ping compared to students who had not re-ceived VE training.

Despite these and other examples of suc-cessful learning and training transfer (e.g.,Mendozzi et al., 2000; Rose et al., 2000), stillnot enough is known about the pedagogicalbasis through which VEs support learning inspecial needs education. Domain knowledgeacquisition, supported by constructivist andconstructionist principles, has been demon-strated in the use of VEs in mainstream educa-tion. Of course, it is probable that these meth-ods of learning are not appropriate for the



learning of skills and behaviors. Examinationof evidence of constructivist principles (as de-fined by Jonassen, 1994) in the use of theMakaton and Life Skills projects found that thedifferent VEs met these principles in differ-ent ways; the virtual supermarket supportedreflection on the real world and unaided ex-ploration whereas the Makaton environmentsupported less independent exploration butmore collaborative instruction (Neale et al.,1999). It was difficult to determine knowl-edge construction through observation of stu-dent performance in use of the VEs largelydue to difficulties in control of the interac-tion devices. However, there was an observedtrend toward student initiation of decisionsand actions throughout the VE training periodand from pre–post real-world observation oftasks in the Life Skills project (Cobb et al.,1998). Constructivist learning is more likely tobe supported by VEs that replicate real-worldscenarios or behaviors since students will beable to draw upon their understanding of thewider context. Abstract VEs, such as the Maka-ton environment, have no real-world counter-part for the students to draw upon. However,there is a danger that the more closely a VEdoes replicate the real world, features of theVE that do not behave as expected may serveto confuse or irritate students and this may de-tract from the learning experience.

CONCLUSIONS

Application of VR technology in specialneeds education has not yet realized thedream of the “Virtual Visualization Room”in which skills learned in virtual situationstransfer to real situations (Brown, Stewart, &Mallett, 1997). Virtual reality and its asso-ciated technologies have developed rapidlysince 1991; still little is known about how toapply these technologies for learning. Educa-tion theorists can indicate what is required ofthe learning experience, but few studies havebeen conducted to measure learning gain.

The experience of VIRART’s research in thisfield over the last 15 years has been one ofrealization of the importance of human fac-tors and principles of human–computer inter-

action in designing applications of new tech-nology. In all of the studies reported in this ar-ticle, usability and interface issues were iden-tified and, in many cases, resolved by iterativedesign improvement. However, there are stillno comprehensive guidelines for the design ofVEs for these user populations, and it is rec-ommended that researchers still need to in-clude a variety of stakeholders in the designof new VEs for learning (Neale, Cobb, & Kerr,2003).

Long-term integration within the teachingcurriculum requires successful implementa-tion within the teaching environment and up-take from teaching staff. One of the barriersencountered when developing a new tech-nology is that they may be unstable and notcompatible with the existing technology in-frastructure. To overcome this, it is essen-tial that researchers work closely with teach-ing professionals and focus on the contextof use, experience, and expectations of tech-nology among the teaching staff. Successfulimplementation requires teachers to takeownership of the teaching program. More suc-cessful outcomes will be achieved if teachersand training professionals take ownership ofthe research from the outset (Cobb, Neale,& Stewart, 2001). Direct input from teach-ing professionals is essential to ensure thatVE development leads to a useful and usableteaching aid. The AS Interactive project em-ployed a teacher to develop a teaching re-source pack to accompany the VE, guidingteachers how to use the system (downloadavailable from www.virart.nott.ac.uk/asi/). Ofcourse, the technology must meet the needsof the teaching requirements if it is to beadopted as a teaching aid and, unfortunately,this required iterative development and evalu-ation extending beyond the timescale of theseearly projects.

VR technology has a lot to offer. There ap-pears to be potential for VEs to benefit learn-ing but further work is needed to direct VE de-velopment and methods for evaluation (Winn,2002). It is hoped that these early studies willprovide encouragement and insight to othersseeking to utilize VEs for children with specialeducation needs.



REFERENCES

Auld, L. W. S., & Pantelidis, V. S. (Eds). (1995, June). VR inthe Schools, 1(1).

Barab, S. A., Hay, K. E., Squire, K., Barnett, M., Schmidt,

R., Karrigan, K., et al. (2000). The virtual solar system:

Learning through a technology-rich, inquiry-based,

participatory learning environment. Journal of Sci-ence Education and Technology, 9(1), 7–25.

Bell, J. T., & Fogler, H. S. (1995, June). The investigationand application of virtual reality as an educationaltool. Paper presented at the annual conference of the

American society for Engineering Education, Session

No. 2513, Anaheim, CA.

Bricken, M. (1991). Virtual reality learning environments:

Potentials and challenges. Computer Graphics, 25(1),

178–184.

Bricken, W., & Winn, W. (1992). Designing virtual worlds

for use in mathematics education: The example of ex-

periential algebra. Educational Technology, 32, 12–

19.

Brown, D. J. (1994, September). Special computers for

special children. Open Learning Systems News, pp.

7–8.

Brown, D. J., Cobb, S. V., & Eastgate, R. M. (1995). Learn-

ing in virtual environments (LIVE). In R. A. Earnshaw,

J. A. Vince, & H. Jones (Eds.), Virtual reality appli-cations (chap. 14, pp. 245–252). San Diego, CA: Aca-

demic Press.

Brown, D. J., Kerr, S. J., & Bayon, V. (1998, September 10-

11). The development of the virtual city: A user cen-

tred approach. In Proceedings of 2nd European con-ference on disability, virtual reality and associatedtechnologies (pp. 11–16). Skovde, Sweden: Mount

Billingen. .

Brown, D. J., Kerr, S. J., & Wilson, J. R. (1997). Virtual

environments in special-needs education. Communi-cations of the ACM, 40(8), 72–75.

Brown, D. J., Neale, H. R., Cobb, S. V., & Reynolds, H.

(1999). Development and evaluation of the virtual city.

International Journal of Virtual Reality, 4(1), 28–41.

Brown, D. J., Stewart, D., & Mallett, A. (1997, Spring). Vir-

tual rooms. The SLD Experience, 17, 15–16.

Byrne, C., & Furness, T. (1994). Virtual reality and educa-

tion. IFIP Transactions, Computer Science and Tech-nology, 58, 181–189.

Clancy, H. (1996). Medical field prescribes virtual reality

for rehabilitation therapy. Computer Reseller News,698, 76.

Cobb, S., Beardon, L., Eastgate, R., Glover, T., Kerr, S.,

Neale, H., et al. (2002). Applied virtual environments

to support learning of social interaction skills in users

with Asperger’s syndrome. Digital Creativity, 13(1),

11–22.

Cobb, S. V. G., & Sharkey, P. M. (2007). A decade of

research and development in disability, virtual real-

ity and associated technologies: Review of ICDVRAT

1996–2006. International Journal of Virtual Reality,

6(2), 51–68.

Cobb, S. V. G., Neale, H. R., Crosier, J. K., & Wilson, J.

R. (2002). Development and evaluation of virtual en-

vironments for education. In K. Stanney (Ed.), Vir-tual environment handbook (chap. 47, pp. 911–

936). Hillsdale, NJ: Erlbaum.

Cobb, S. V. G., Neale, H. R., & Reynolds, H. (1998, Septem-

ber 10–11). Evaluation of virtual learning environ-

ment. In: The 2nd European conference on disabil-ity, virtual reality and associated technologies (pp.

17–23). Skovde, Sweden: University of Reading.

Cobb, S. V. G., Neale, H. R., & Stewart, D. (2001, August 8–

10). Virtual environments—Improving accessibility to

learning? In Proceedings of 1st international confer-ence on universal access and human computer in-teraction (pp. 783–787). New Orleans, LA: Erlbaum.

Crosier, J. (1996). Experimental comparison of differ-ent input devices into virtual reality systems usedby children with severe learning disabilities. Unpub-

lished dissertation, University of Nottingham.

Crosier, J. K., Cobb, S. V. G., & Wilson, J. R. (2000).

Experimental comparison of virtual reality with tra-

ditional teaching methods for teaching radioactivity.

Education and Information Technologies, 5(4), 329–

343.

Dede, C., Salzman, M., & Bowen Loftin, R. (1996). Sci-enceSpace: Virtual realities for learning complexand abstract scientific concepts. Paper presented at

the Proceedings of Virtual Reality Annual International

Symposium (VRAIS) 1996. pp. 246–253.

ECDVRAT. (1996, July 8–10). Paper presented at the 1st

European conference on Disability, Virtual Reality and

Associated Technologies Maidenhead, UK. Retrieved

from www.icdvrat.rdg.ac.uk/archive.htm

Grove, N., & Walker, M. (1990). The Makaton vocabu-

lary: Using manual signs and graphic symbols to de-

velop inter-personal communication. AAC Augmenta-tive and Alternative Communication, 15–28.

Hall, J. (1993). Explorations of population expectationsand stereotypes with relevance to design. Unpub-

lished dissertation, University of Nottingham.

Jonassen, D. (1994). Thinking technology? Towards a con-

structivist design model. Educational Technology,

34(4), 34–37.

Kerr, S. J., Neale, H. R., & Cobb, S. V. G. (2002). Virtual en-

vironments for social skills training: The importance

of scaffolding in practice. In: Julie Jacko (Ed.), The 5thinternational ACM conference on Assistive Technolo-gies (ASSETS 2002) (pp. 104–110). ACM Press.

Meakin, L., Wilkins, L., Gent, C., Brown, S., Moreledge,

D., Gretton, C., et al. (1998, September 10–11). User

group involvement in the development of a virtual

city. In P. Sharkey, D. Rose, & J. I. Lindstrom (Eds.), The2nd European conference on Disability, Virtual Re-ality and Associated Technologies (pp. 1–9). Skovde,

Sweden: University of Reading.

Mendozzi, L., Pugnetti, L., Barneiri, E., Attree, E. A.,

Rose, F. D., Moro, W., et al. (2000, September 23–25).



A generic productive process to train persons with

disabilities. In Proceedings of the 3rd internationalconference on Disability, Virtual Reality and As-sociated Technologies (ICDVRAT) (pp. 115–122).

Sardinia, NY.

Mitchell, P., Parsons, S., & Leonard, A. (2007). Using vir-

tual environments for teaching social understanding

to adolescents with autistic spectrum disorders. Jour-nal of Autism and Developmental Disorders, 37,

589–600.

Moshell, J. M., & Huges, C. E. (1996). The virtual academy:

A simulated environment for constructionist learning.

International Journal of Human–Computer Interac-tion, 8(1), 95–110.

Moshell, J. M., & Hughes, C. E. (2002). Virtual environ-

ments as a tool for academic learning. In K. Stan-

ney (Ed.), Handbook of virtual environments, de-sign, implementation and applications (pp. 893–

910). Hillsdale, NJ: Erlbaum.

Murphy, H. J. (1993, June). Virtual reality and persons

with disabilities. Proceedings of the first annual con-ference: Virtual reality and persons with disabili-ties. Northridge: California State University Center on

Disabilities.

Neale, H. (1997). A structured evaluation of virtual en-vironments in special needs education. Unpublished

dissertation, University of Nottingham.

Neale, H. R., Brown, D. J., Cobb, S. V. G., & Wilson, J.

R. (1999). Structured evaluation of virtual environ-

ments for special needs education. Presence: Tele-operators and Virtual Environments, 8(3), 264–

282.

Neale, H., Cobb, S., & Kerr, S. (2003, March 25–28). An in-

clusive design toolbox for development of educational

virtual environments. In Include2003 (pp. 230–235).

London: Royal College of Art.

Neale, H. R., Cobb, S. V., Kerr, S., & Leonard, A. (2002,

September). Exploring the role of virtual environ-

ments in the special needs classroom. In Proceed-ings of the 4th international conference on disabil-ity, virtual reality and associated technologies (ICD-VRAT) (pp. 259–266). Veszprem, Hungary: University

of Reading.

Neale, H. R., Cobb, S. V. G., & Wilson, J. R. (2000,

September 23–25). Designing virtual learning environ-

ments for people with learning disabilities: Usability

issues. In Proceedings of the 3rd international con-ference on disability, virtual reality and associatedtechnologies (ICDVRAT) (pp. 265–272). Sardinia,

NY: University of Reading.

Neale, H. R., Cobb, S. V. G., & Wilson, J. R. (2001). Involv-

ing users with learning disabilities in virtual environ-

ment design. In Proceedings of universal access inHCI (pp. 506–510). New Orleans, LA: Erlbaum.

Parsons, S., Beardon, L., Neale, H. R., Reynard, G., East-

gate, R., Wilson, J. R., et al. (2000). Developmentof social skills amongst adults with Asperger’s syn-drome using virtual environments: The AS interac-

tive project. Paper presented at the 3rd international

conference on Disability, Virtual Reality and Associ-

ated Technologies (ICDVRAT), Sardinia, NY, Septem-

ber 23–25, 2000, pp. 163–170.

Parsons, S., & Mitchell, P. (2002). The potential of virtual

reality in social skills training for people with autistic

spectrum disorders. Journal of Intellectual DisabilityResearch, 46(Pt. 5), 430–443.

Parsons, S., Mitchell, P., & Leonard, A. (2004). The use

and understanding of virtual environments by ado-

lescents with autistic spectrum disorders. Journal ofAutism and Developmental Disorders, 34(4), 449–

466.

Parsons, S., Mitchell, P., & Leonard, A. (2005). Do ado-

lescents with autistic spectrum disorders adhere to

social conventions in virtual environments? Autism,

9(1), 95–117.

Rose, F. D., Brooks, B. M., & Attree, E. A. (2000). Vir-tual reality in vocational training of people withlearning disabilities. Paper presented at the 3rd in-

ternational conference on Disability, Virtual Reality

and Associated Technologies (ICDVRAT), Sardinia, NY,

September 23–25, pp. 129–135.

Rutten, A., Cobb, S., Neale, H., Kerr, S., Leonard, A.,

Parsons, S., et al. (2003). The AS interactive project:

Single-user and collaborative virtual environments for

people with high-functioning autistic spectrum disor-

ders. Journal of Visualization and Computer Anima-tion, 14, 233–241.

Salzman, M. C., Dede, C., Bowen Loftin, R., & Chen, J.

(1999). A model for how VR aids complex conceptual

learning. Presence, 8(3), 293–316.

Standen, P. J., & Brown, D. J. (2005). Virtual reality in the

rehabilitation of people with intellectual disabilities:

Review. Cyberpsychology & Behaviour, 8(3), 272–

282.

Standen, P. J., Cromby, J. J., & Brown, D. J. (1998). Playing

for real. Mental Healthcare, 1, 412–415.

Standen, P. J., & Low, H. L. (1996). Do virtual envi-ronments promote self-directed activity? A study ofstudents with severe learning difficulties learningMakaton sign language. Paper presented at the 1st

European conference on Disability, Virtual Reality and

Associated Technologies, Maidenhead, UK, July 8–10,

pp. 123–127.

Standen, P. J., Brown, D. J., Anderton, N., & Battersby, S.

(2004). Problems with control devices experiencedby people with intellectual disabilities using virtualenvironments: A systematic evaluation, 2004. Pa-

per presented at the 5th international conference on

Disability, Virtual Reality and Associated Technologies

(ICDVRAT), Oxford, UK, September 20–22, 2004, pp.

299–304.

Strickland, D. (1996). A virtual reality application with

autistic children. Presence, 5, 319–329.

VIRART (2007). Available from Virtual Reality Applica-

tions Research Team, University of Nottingham, UK,

Web site: www.virart.nottingham.ac.uk



Virtual reality and education. (1999, June). Presence: Tele-operators and Virtual Environments, 8(3, Special

issue).

VRET (1997). Proceedings of Virtual Reality Educationand Training, University of Loughborough, U.K. Re-

trieved from http://www.vrnews.com/vret97.html.

Wiederhold, B. K. (2004). Virtual healing. San Diego, CA:

Interactive Media Institute.

Wing, L. (1996). Autistic spectrum disorders. British Med-ical Journal, 312, 327–328.

Winn, W. (2002). Current trends in educational technol-

ogy research: The study of learning environments. Ed-ucational Psychology Review, 14(3), 331–351.

Youngblut, C. (1998). Educational uses of virtual realitytechnology (Tech. Rep. No. D-2128). Alexandria, VA:

Institute for Defense Analysis.

Virtual environments supporting learning and communication in special needs education

Documents

Transcript of Virtual environments supporting learning and communication in special needs education