Digitally augmented reality characters in live theatre performances

International Journal of Performance Arts and Digital Media, Volume 5 Number 1.

© Intellect Ltd 2009. Article. English language. doi: 10.1386/padm.5.1.35/1

Digitally augmented reality characters in live theatre performances

Daniel Jernigan Nanyang Technological University

Stephen Fernandez Nanyang Technological University

Russell Pensyl Nanyang Technological University

Lee Shangping Nanyang Technological University

AbstractThis article investigates what we believe to be the first use of Augmented Reality(AR) in the theatre: NTU-IERC’s production of The Ultimate Commodity(based on the Gopal Baratham short story) in Singapore and Toronto (where itwas re-titled Everyman: The Ultimate Commodity). Augmented Reality is dif-ferent from Virtual Reality in that AR ‘supplements, rather than supplants, thereal world’ (Azuma, 1). This essay describes the technological innovations employedin bringing AR to the theatre, discusses how implementing this technology enablestheatrical innovation, and then uses our experience to theorize the implicationsof blending the virtual and the theatrical.

IntroductionAugmented Reality (AR) evolved out of the more popular Virtual Reality inthe early 1990’s (Azuma 2004: 4), with its most identifiable form compris-ing the table-top AR configuration of a participant wearing a mobile virtualreality headset which, when trained on a black and white marker, enablesthe user to witness a virtual object superimposed in the place of thatmarker (see Figure 1).

As Ronald Azuma and G. Bishop (1994: 1) explain it, the more familiarVirtual Reality technology – which creates a virtual world that ‘completelyreplaces the real world outside’ – is different from AR, in that, AR ‘supple-ments, rather than supplants, the real world’.

For our project, we began with the premise that any use of AR technologyin theatrical performance would require that ‘the real and the virtual environ-ment[s]’ be combined so as to become ‘interactive in real time’ and in sucha way that the virtual image of the ‘live’ performance would be ‘registered inthree dimensions’ (Azuma 1997: 356). With this in mind, our own AR play,Everyman: The Ultimate Commodity, sought to accomplish this by beginningwith the idea that if the interaction between marker and human participantwere inverted (so that the marker became mobile and the participant static)AR technology would become theatrically relevant. Our solution was that the

35PADM 5 (1) pp. 35–49. © Intellect Ltd 2009.

Keywordsaugmented realityvirtual realitytheatre masksvirtual masksGopal Baratham

PADM_5_1_03-Jernigan_090003color 6/23/09 7:13 PM Page 35

actors would set the mobile markersinto motion while the audiencebecame the static participants asthey sat in their seats to witness theproduction (according to this sce-nario, in the place of a virtual realityheadset the audience would see thevirtual-object superimposition via anonstage screen). What we discov-ered through the staging of our playis that employing such innovationsin the theatre created two ontologi-cally distinct if sometimes overlap-ping spaces on the stage – one ‘real’and the other ‘virtual’ (or, torephrase Azuma, we found thatAR technology in the theatre func-tions most effectively when it sup-plements, rather than supplants,the theatrical world). In turn, wefound new and innovative ways toincorporate computer-generatedimagery (CGI) into what in manyrespects remained a traditional the-atrical environment.

Take one: The Ultimate CommodityAs the content for our production, we chose Singaporean writer GopalBaratham’s short story ‘The Ultimate Commodity’, which we recognizedas a uniquely suitable vehicle to investigate AR’s theatrical possibilities.Baratham’s short story imagines a future in which a Singaporean scientisthas created a formula (Substance X) which causes all those who ingest it tobe transformed into universal organ donors (whose organs can be har-vested and transplanted into any other person’s body without fear of rejec-tion). In this imagined future, the Singaporean Government has taken theliberty of adding Substance X to the city’s drinking water, with the objectivethat Singapore might finally fulfil its destiny as a country where the govern-ment’s claim that ‘Our people have become our only resource [. . .] hasbecome literally true’ (Jernigan 2007).1 However, a significant side-effect ofthe formula is that it also causes the distinguishing characteristics ofSingaporeans to disappear, so that every Singaporean becomes morpholog-ically identical to every other Singaporean. Against this backdrop, ourtheatre project focuses on a small part of the larger story; on the identitycrisis which occurs when a father (Binny) confuses his daughter (Leeni)with his wife (Ri).

For the first generation of our project (staged in Singapore in 2006), weexperimented with AR’s computerized morphing potential by having the

1 A copy of the scriptand video footage of the performanceare available at theInteractive andEntertainmentResearch Center(IERC) website,http://www3.ntu.edu.sg/ierc/uc_ismar.html

36 Daniel Jernigan, Stephen Fernandez, Russell Pensyl and Lee Shangping

Figure 1: Table-top Augmented Realityset-up – Using see-through aug-mented reality glasses to view thestationary markers on the table, themobile participant is able to witnessthe superimposition of virtual models over the fixed markers.


actors in our play hold marker enabled ‘theatre masks’ over their faces. Atracking camera at the front of the stage was utilized to capture the action,and this footage was then enhanced via AR technology, which morphed theactors’ faces from one to another according to the needs of the perfor-mance (see Figures 2 and 3).

37Digitally augmented reality characters in live theatre performances

Figure 3: Actors in front & behind the projection screen. Note that the actorsbehind the screen have their faces morphed from their own to others’.

Figure 2: Actors holding theatre masks – a black and white marker for thepurpose of augmented reality tracking – over their faces.

PADM_5_1_03-Jernigan_090003color 6/23/09 7:13 PM 7

The dramatic action in the first generation of the project took place bothbehind and in front of the projection screen (see Figure 4). When the per-formance required that the actors morph from one face to another, theywould be behind the screen (in the virtual space), only visible to the audi-ence via projection on screen. At all other times, they were in front of thescreen in the performance space (see Figure 4).

The idea was that the actors in front of the screen were part of the samephysical space as the actors onscreen. This conceptualization allowed forthe possibility that while a scene might begin with Binny talking to twocharacters (his wife and his daughter) it could end with him talking to asingle character (a morphed version of his mother and daughter) or, even,with him talking to himself, as his features morphed with those of theremaining characters. In relation to the dramatic content of Everyman, thisinnovation not only helped the audience to visualize the elimination of dis-tinguishing characteristics in real time, but also served to re-enforce themore general theme of the work – that is, that perhaps contemporarySingaporeans had already begun the process of losing their various distin-guishing characteristics. Furthermore, the simultaneous staging of ‘real’characters with ‘simulated’ characters also resonated with the theme ofidentity construction, itself a prevalent theme in Baratham’s original text.


Figure 4: Setup of the first generation AR system for theatre. Actors are in frontand behind the projection screen. The actor behind the screen holds a fiducialmarker over her face.


There were, however, a number of problems with this model, both tech-nological and performative. On the technical side, the AR system facedgreat difficulty in the dim light of the theatrical environment, which all toooften cause the tracking system to cut out so that instead of viewing onesuperimposed face after another, all the audience would see on the screenwas an actor holding a marker over their face. Moreover, in order to func-tion properly, the mask-marker always had to be oriented directly towardsthe camera (and the audience), or the AR system would fail, with the sameeffect. What is more, even when functioning properly this system onlyallowed for 2D superimposition of the actors’ faces. In terms of the aes-thetics of the performance, it is also fair to say that the decision to place theactors with marker/masks behind the screen was a mistake, as many audi-ence members did not understand that what was appearing on screen washappening in real time during the performance (in retrospect it is obviousthat everything on screen might just as easily been filmed before hand, aresult which betrays the original intention of using AR technology in thefirst place; that is, to show the simultaneous superimposition of virtualmasks on the faces of ‘live’ actors in real time).

Take two: Everyman: The Ultimate CommodityIn order to circumvent the problems encountered in the first generation ofthe project and, in the process, to more fully realize the technological poten-tial of morphing one character’s face into another in real-time, light-emittingdiode (LED) headset displays (see Figure 5) became a necessary replace-ment for the standard black and white marker found in table-top AR projects,


Figure 5: Hybrid LED array and motion sensor headset mounted on the actors’heads during the production of our original play.


as the light emitted from such devices could be tracked much more easily ina darkened theatre then could a simple black and white marker.2 In fact, forour project we employed different coloured LED displays, allowing us theoption of superimposing different faces over various actors according to theneeds of the performance. These LED displays could then be trackedthrough the three-dimensional space of the theatre via the tracking cameraswhich were hung from the ceiling. Furthermore, to acquire the orientation ofthe actors’ heads (where and how the head is pointing and tilting), we devel-oped an improvized tracking method by using wireless motion sensors(comprising Bluetooth, accelerometers, gyroscopes and digital compasses)that were embedded underneath the LEDs. The hybrid LEDs and sensortechnique allowed for the full six degrees of freedom (6-DOF) information ofthe head to be known (see Figure 6). In this way, the actors had completemobility, as their entire head could be replaced with a three-dimensionalvirtual face mask of the director’s choosing.

In our effort to create realistically functional virtual masks, 3D virtualfaces were rendered in real time onto the actors on screen. The 3D modelsof the actors’ heads were constructed from a 3D model of a generic head.Six different views of the actor’s face with 30 small markers were capturedby six pre-calibrated cameras. The 2D points on the captured images and3D points on the generic model corresponding with those markers on theactor’s face were selected manually. After that, the 3D generic model wasmorphed iteratively until the distance between the selected 2D points oncaptured images and 2D projections of selected 3D points on the genericmodels was minimized on all 6 different views (see Figure 7).

2 It is notable that even as employingAugmented Realityforced changes in the script, employingAugmented Reality in the theatre forcedinnovation of thetechnology.


Figure 6: The actors’ heads could be replaced by 3D head models according tothe colour (red, yellow or blue) of the LED headset the actor was wearing.


In addition, pre-recorded voice tracks were embedded in the ‘live’ videostream (synchronized with the script and dramatic action) and a lip-synctechnique was used to animate the virtual face’s lip movements. Phonemeswere then extracted from the pre-recorded voice tracks, time-stamped andsaved in text files. We then utilized game engine technology to access thetext files, read the timestamps associated with the phonemes and renderthe corresponding motion for the lips motions (visemes). Each virtual facewas constructed with a sequence of ‘visemes’ that were interpolated(morphed) to synchronize with the voice track. This allowed the virtualfaces to ‘talk’ in real time and in sync with the actual actor from the ‘live’performance space. In order to make the embedded voice tracks match pre-cisely with the live video streams, the actors and backstage production per-formed numerous rehearsals. As actors performed live on stage ourbackstage crew synchronized the video streams and voice tracks accordingto the needs of the script. The positioning of the virtual faces over theactor’s faces depended on the tracked location of the LEDs affixed to head-sets that were mounted on the heads of the ‘live’ actors.


Figure 7: The 3D model of the actors’ heads were constructed from a 3D modelof a generic head. Six different views of the actor’s face with 30 small markerswere captured by six pre-calibrated cameras. The 2D points on the capturedimages and 3D points on the generic model corresponding with those markerson the actor’s face were selected manually.


As the project progressed into the second stage, AR finally began toprove itself as uniquely suited to the telling of this story, especially for howit allowed Binny to be played by a ‘live’ actor even as the other two charac-ters’ features could be ‘augmented’ with virtual masks superimposed intothe scene with Binny by means of screen projection. In place of the singlestage of the first production, what we had, in essence, were two stages: a‘virtual stage’ and a ‘real stage’ (see Figure 8).

In the real stage, the audience would always and only see the real actors,albeit outfitted with the necessary technological devices used to deliver therequired projections to the ‘virtual stage’. In the ‘virtual stage’ (which was sit-uated side by side with the ‘real stage’ – so that each shared half the stage),the audience would see these same actors from the real stage projected ontoa large screen, albeit augmented with their respective ‘virtual masks’. Thisallowed real actors the ability to interact with ‘augmented’ actors in thevirtual stage (it is this ‘augmentation’ – and the way that it supplementslive actors [working in real time] rather than simply supplanting them –which differentiates the use of AR technology from other theatrical produc-tions wherein actors have interacted with images projected on screen, butmore on that below).

Innovations and traditionsEmploying such technological innovations in the theatre, however, raisesimportant questions about the role of technology in theatre more generally.For instance, does importing new technology truly revolutionize what ispossible vis-à-vis dramatic performance, or, rather, does it simply repro-duce what was always and already possible in the theatre? This is especially


Figure 8: A sketch depicting the stage setup for the augmented reality systemused in the production of Everyman at the 2007 Toronto Fringe Festival. Theaudience sees the juxtaposition of real and augmented spaces.


relevant question in our case, as Everyman: The Ultimate Commodity couldindeed be staged using more traditional masking techniques. We contend,however, that even while the integration of virtual elements into ‘live’ per-formances of contemporary mixed media drama often functions simply asan extension of theatrical approaches and technologies that pre-existedthem (for instance the overhead projection of various types of images madefamous by Brecht would have seen its early form in shadow puppetry),there are moments when a technology affords such innovative possibilitiesthat creating something entirely new becomes possible, and it is our con-tention that Everyman: The Ultimate Commodity successfully pushes theenvelope of what is theatrically possible vis-à-vis identity construction andconflation in ways that quite simply would never have been consideredwhen employing more conventional theatrical techniques.

We understand, moreover, that other theatre troupes have blended thereal and the virtual in similar ways. For instance, the British performancegroup, Station House Opera, explores the relationship between humans andthe environment in which they live by employing ‘live’ video in the construc-tion of multiple spaces weaved together in real-time to form a single perfor-mance. One of the group’s well-known mixed media production, Mare’s Nest(2001), features four people and their life-sized video doubles who occupy a‘half-physical and half-virtual’ stage that is real and imaginary at the sametime, thereby facilitating the interaction between ‘live’ action and filmed per-formance in the ‘complex, augmented space where architecture [of thetheatre] and video meet’ (Mare’s Nest 2001: 1). What this means is that ‘live’performers can simultaneously inhabit the reality of the stage as well as thevirtuality of the ‘video world’ in which their images are reproduced, in orderto create an intriguing concoction of filmed and ‘live’ action.

Such an intimate relationship between ‘live’ performers and their videocounterparts is echoed in The Wooster Group’s Brace Up! a mixed mediaadaptation of Anton Chekhov’s Three Sisters. Like Mare’s Nest, The WoosterGroup’s play also employs ‘live’ video technology as part of its theatricality.Two monitors (each mounted on a stand) move upstage and downstage ontracks which divide the stage into three equal sections. There is also a thirdmonitor which the ‘live’ performers can freely move around the play area.With this set-up, The Wooster Group re-configures the inherent power poli-tics in Chekhov’s original narrative by having ‘live’ actors play the charac-ters that are able to reproduce biologically and culturally through marriage,while the characters that cannot reproduce on their own are technologicallyreproduced as human images on the video monitors. In this way, theemployment of ‘live’ video in Brace Up! allows both the ‘live’ and filmedcharacters to communicate with each other in very much the same way thatthey do in Mare’s Nest. However, while we recognize that the innovationsemployed by Station House Opera and The Wooster Group are importantin their own right, it is also important to recognize that video technologyemployed in each case achieves a fundamentally different effect than whatwe achieve via the AR technology in Everyman, which allows us to morphthe individual faces of our characters from one face to another in real-time.



Indeed, as facial morphing is an essential feature of our play, the analysis ofthe face-changing technique employed in such traditional masked perfor-mances as the Sichuan Bian Lian opera are perhaps more relevant to ourdiscussion of the effectiveness of AR technology in expressing the identityconflation that is central to the dramatic narrative.

There are two principal masking methods in Asian theatre that supportidentity construction, the face-painting of the Beijing Opera and the mask-switching of the Sichuan Bian Lian. As Chinese theatre scholar Yi Bian(2005) explains, in traditional Beijing Opera painted faces for ‘Jing’ roles(good or bad men) ‘express the character’s personalities and traits as wellas the role type to which the characters belong’ (p. 15). The painting ofcolours on the face to represent the different character traits and the role-specific drawing of lines and patterns over the colours allows the audienceto recognize the various characters more clearly. As with the painted facesof the Beijing Opera, the faces of the actors in Everyman (itself a predomi-nately Asian production) are themselves ‘painted on’, albeit virtually, bymeans of the superimposition of virtual face masks.3 However, in contrastto the virtual face masks which are designed to facilitate the easy morphingfrom one facial appearance to another, masks in the Beijing Opera remainas static displays of facial expression.

It is for this reason that the Sichuan ‘Bian Lian’ (face changing) maskeddrama is, perhaps, an even better example of a theatrical precursor toEveryman for how it similarly functions to undermine the very ontologicalintegrity of identity construction in performance. Actors in Bian Lian wearvividly coloured face masks which change in quick succession. And whilethe colours of these masks reflect the character’s mood in the samemanner as that of the Beijing Opera, the act of switching masks is itselfsymbolic of the multiple personalities of human beings, elevating theordinary face mask beyond the level of theatrical function, in order to‘express those profound hidden conflicts of the mind which the probings ofpsychology continue to disclose to us’, especially since the opaqueness ofmasks prevents the audience from seeing what is hidden underneath(O’Neill 1988: 406). As a result, the Sichuan Bian Lian masked drama pre-figures how AR Technology in our own project also raises questions con-cerning identity conflation in contemporary society, as all the characters inthe play morph into a single character with similar facial characteristics. Inturn, the act of morphing itself – together with the technology whichenables the morphing – becomes symbolic of contemporary concernsabout the loss of identity in an increasingly digitized age. That is, whilethe Bian Lian suggests the wide diversity of attitudes which exists in theindividual, our play, by contrast, uses its technological effects to suggesta flattening of diversity into sameness. In turn, it successfully reproducesthe concerns of the original story, where the doctor in charge of the experi-ment – when faced with the consequences of his actions – suggests that itmight be worth it to ‘[s]ink your differences and form a nation’, a phrasemeant to invoke the way in which Singaporean policies promote homo-geneity (Baratham 2001: 67).

3 What weaccomplished inEveryman can bedescribed this way perhaps onlyreinforces thepossibility thattechnology fails to fully revolutionizethe theatre.



Theorizing the innovationsGiven that what we see in Everyman could just as easily be accomplishedthrough conventional masking methods, it is tempting to consider that ARdid not revolutionize theatre so much as simply enable innovation, and that,at most, AR simply pointed us in a direction that we might not otherwisehave considered. This line of thinking would argue that perhaps everythingtruly innovative in the performance could just as easily have been accom-plished by employing two sets of actors, such that everything seen on screenis reproduced by the second set of actors instead of by a complicated andcumbersome virtual reality set up. In such a scenario, even the horror whichcomes with seeing the very same actor who plays Binny being seduced bythe very actress who has been playing his daughter, would still be possible.

It is our contention, however, that by witnessing both stages simultane-ously, the audience is better situated to sympathize with the identity confu-sion which perplexes the characters themselves. For instance, when Leenivery nearly seduces her own father, the audience’s investment in this iden-tity confusion is compounded by the fact that it sees both scenes simulta-neously (i.e. what looks like the mother seducing her husband from oneperspective, is very clearly the daughter seducing her own father fromanother – see Figure 9). Consequently, while a more traditional masked per-formance only allows audiences the ability to see the role-specific facemask, ours simultaneously allows the audience the opportunity to piercethrough the materiality of the mask to the actual human face of the


Figure 9: This picture shows the actual performance on a stage. On the rightare the actors who are physically seated on a sofa in the ‘real’ performancespace; while on the left is a large projection screen with the ‘morphed’ actorsprojected on it.


performer playing the character and encounter what we might well be calledthe ‘double vision’ of witnessing in real time, the identity confusion unfold-ing in the virtual space projected on the screen as well as in the ‘live’ action.

The identity confusion on the virtual stage in Everyman, where each char-acter morphs into the face of another character by means of virtual masks,is, then, pitched against that which is occurring on the ‘live’ stage, whereactors in their unmasked state physically perform using their own faces. As aresult, the existence of the virtual and the ‘live’ performances on the samestage and the conflation of Ri’s identity with that of Leeni’s on the virtualstage, create potentially horrific realities that raise questions concerning theperformative nature of seduction both on stage and off, as well as forcingreflection about the social and moral values that categorically defines incestas a cultural taboo. In essence, we are asked to consider what it means thatthe more explicitly alienating of the two scenes (the virtual) is also the moresanitized version, while the more traditionally ‘realistic’ version is morealienating (at least in its content). What does this mean, then, except thattechnology itself is sanitizing – and that it is capable of making what hadbeen unthinkable all the more palatable. The nightmarish possibility that atotalitarian government might interfere with the very genetic structure of itsconstituents – and do so in a way that conflates identity – is, then, reflectedin the way in which technology itself is fused with performance.

In interpreting the ‘Real’, Slavoj Zizek purports that misrecognition isthe way to know the ‘truth’, whereby the only thing that exists behind theveil of appearance is the illusion of a ‘mysterious truth’ that is waiting to beuncovered. For Everyman, it can be argued that misrecognition does nottake place at all – at least not in the ‘real’ performance space. The illusionthat there is something to discover when the virtual face mask is removedis ‘a reflexive one’, in the sense that what is hidden behind the face mask isthe ‘possibility of this very illusion’, whereby both Binny and the audienceare subject to the assumption that there is a certain truth (i.e. the revelationthat Ri is in fact Leeni) to be unveiled in the dramatic action that occurs inthe virtual performance space (Zizek 1989: 193). However, the credibility ofthis assumption is challenged by the fact that Binny’s misrecognition doesnot occur in the ‘real’ performance space. And, moreover, because the audi-ence is privy to the dramatic actions of both the ‘live’ actors and theirvirtual images, they can clearly see that Binny and Leeni are the only char-acters in the scene, meaning that Binny’s misrecognition becomes an illu-sion that is explicitly played out on the screen, with the audience being fullyaware that in reality, there is no misrecognition. Hence, even though theinherent textual irony of the misrecognition scene in Everyman is under-mined by the set design, it is the ironic implications of the AR-enabled‘double vision’ technique used in the play that revolutionizes traditionalmirroring techniques employed in theatre performance.

Final considerationsLooking back at our experience preparing for Everyman: The UltimateCommodity’s eventual premier at the 2007 Toronto Fringe Festival, we now



find that while AR Technology did prove to be an effective means of express-ing and reinforcing the play’s themes, it also modified our artistic and dra-matic considerations in unexpected and, sometimes, discomforting ways,so that even while we tried to hold true to the original narrative as the mainthrust of the work, the play was significantly impacted by both the technol-ogy and the juxtaposition of media and ‘live’ action within a single space.What this means is that even while technology might be useful in enhanc-ing the visual (and sometimes audio) aesthetics of a play and in underscor-ing its intended message, it can also be a hindrance to the performance byforcing too many compromises on authors, directors, and even actors. Oneextreme example of such a compromise occurred in the first generation ofthe project, at which point the fiducial mask technology required that theactors always be facing directly at the audience, which severely limited theactors’ movements.

In the final analysis, however, we believe that our project serves as a rea-sonable example of how any attempt to integrate technology into theatreperformance can be a double-edged sword, as technology can simultane-ously be both supportive of – and disruptive to – the themes and aestheticsof a particular production. On the one hand, some of the technologicalrestrictions brought about by the incorporation of AR Technology inEveryman ultimately proved beneficial to the larger narrative, as the variousthemes of the play itself (specifically, identity confusion among Singaporeansin an increasingly technological environment set in the near future) wereadvanced by the various compromises reached. This was especially true inhow the split stage – which became one of the more important theatricalcharacteristic of the performance – itself began as a compromise thatbecame necessary when other stage configurations lacked theatrical reso-nance. Conversely, it is equally true that even as the technology forced inno-vation in the performance, the demands of the theatre lead to technologicalinnovations as well, some of which are currently being put to use in IERC’slatest Project, ‘The Long Bar: A Writer’s Notebook’.4

On the other hand, some of the technological features – such as themounting of small arrays of brightly coloured LEDs on headset devicesworn by the actors during the play – proved so cumbersome in perfor-mance that they were finally written explicitly into the script. Together, thesetechnical and dramatic compromises illustrate the ongoing difficulties pur-veyors of mixed media and digital drama can expect to confront, only rein-forcing the difficulty any performance will face when combining ‘live’ actorswith virtual ones in a single playing space.

As technology continues to evolve AR will no doubt become less invasiveto theatrical production, making it less necessary to accommodate the scriptto the technology, such that theatre will be able to seamlessly integrate suchmixed media technologies into theatrical performance. However, as technol-ogy continues to advance theatre practitioners will, no doubt, continue to seekout the latest technologies. In such instances, these new technologies are likelyto be as cumbersome as AR is today. As such, it is all the more essential thatthe accommodation of performance to technology be fully theorized.

4 For more information,see our website,http://www3.ntu.edu.sg/ierc/longbar_wnbook.html



In our case, even the more cumbersome accommodations rather neatlyserved the thematic meaning of the script – that is, the technology employedin performance mirrored the very technological achievements which gaverise to the dramatic tension of the play. Perhaps what this means is that themore cumbersome and revolutionary the technology, the more necessary itbecomes that it first be put to use in support of scripts which are themselvesinvested in technology, so that the technology can reinforce the thematicmeanings of the work even as it serves its other aesthetic purposes. Perhapsonly as technology becomes so refined that it no longer compromises theaesthetics of the play with the extraneous inclusion of bulky devices, can itfinally be appropriated to wider theatrical use.

ReferencesAzuma, R.T. and G. Bishop (1994), ‘Improving Static and Dynamic Registration in

an Optical See-Through HMD’, International Conference on Computer Graphicsand Interactive Techniques, New York: ACM, pp. 197–204.

Azuma, R.T. (1997), ‘A Survey of Augmented Reality’, Presence: Teleoperators andVirtual Environments, 6: 4, pp. 355–385.

——— (2004), ‘Overview of Augmented Reality’, International Conference onComputer Graphics and Interactive Techniques: ACM SIGGRAPH 2004 CourseNotes No. 26, New York: ACM, pp. 1–165.

Baratham, G. (2001), ‘Ultimate Commodity’, in Shah Alam (ed.), The City ofForgetting: The Collected Stories of Gopal Baratham, Malaysia: Times, pp. 60–67.

Beckett, S. (1994), Waiting for Godot: A Tragicomedy in Two Acts, New York: Grove Press.First published 1953.

Bian, Y. (2005), The Cream of Chinese Culture: Peking Opera, Beijing, China: ForeignLanguages Press.

Jernigan, D.K. (2007), Everyman, The Ultimate Commodity, Singapore: NanyangUniversity Press.

O’Neill, E. (1988), ‘Memoranda on masks’, in Travis Bogard (ed.), The UnknownO’Neill: Unpublished or Unfamiliar Writings of Eugene O’Neill, New Haven: YaleUniversity Press. First published 1932, pp. 406–411.

Smith, J.M. (2001), ‘Mare’s Nest’, Geneva: La Batie Festival and Station House Opera,http://www.stationhouseopera.com/PastProjects/maresnest.html#. Accessed25 March 2009.

Zizek, S. (1989), ‘Which Subject of the Real?’ The Sublime Object of Ideology, Londonand New York: Verso, pp. 153–200.

Suggested citationJernigan, D., Fernandez, S., Pensyl, R., & Shangping, L. (2009), ‘Digitally aug-

mented reality characters in live theatre performances’, International Journal ofPerformance Arts and Digital Media 5: 1, pp. 35–49, doi: 10.1386/padm.5.1.35/1

Contributor detailsDaniel Jernigan is Assistant Professor of English at Nanyang Technological University,Singapore. His interests include drama and theatre studies, postmodernism and cre-ative writing. Dr Jernigan’s critical work on Caryl Churchill and Tom Stoppard has beenpublished in Modern Drama and Comparative Drama. His edited collection, Dramaand the Postmodern: Assessing the Limits of Metatheatre (2008) is available through



Cambria Press. Contact: Assistant Professor of English, Department of English, S3.2B4 27, Nanyang Technological University, Singapore, 639798.E-mail: [email protected]

Stephen Fernandez received his BA from the English Division at NanyangTechnological University. His main research interest is in the theatrical relationshipbetween technology and drama. Under the guidance of Dr Daniel Jernigan, Stephenhas recently completed his honours thesis, ‘Performing “Technodrama”: Towards aTechnocultural Aesthetic in the Age of Postmodern Anxiety’. Contact: Division ofEnglish, S3.2-B4-28, Nanyang Technological University, Singapore, 639798.E-mail: [email protected]

Russell Pensyl is currently an Associate Professor and Director of the Interactionand Entertainment Research Centre at NTU. He received his MFA from WesternMichigan University in 1988. Pensyl has more than 23 years of experience in design-ing and creating interactive digital media content for corporations such as IBM,Apple, Motorola, Disney, and many others. Pensyl’s work has been exhibited inter-nationally with installations in SIGGRAPH 2002, the 2004 Shanghai Biennial andthe 2006 Second Art and Science Exhibition in Beijing. Contact: Interaction &Entertainment Research Centre, 81 Nanyang Drive, ART-B1-16 School of Art, Designand Media, Nanyang Technological University, Singapore, 637458.E-mail: [email protected]

Shang Ping Lee obtained his B.Eng (Mechanical and Production Engineering) andM.Sc (Mechatronics) at the Nanyang Technological University in 1997, and theNational University of Singapore in 2001, respectively. He worked at IBM SingaporePte. Ltd from 1997–1999, and then at SGP Global Technologies in 2000 before start-ing his research career at the National University of Singapore as a research engi-neer in 2001. He then joined the Interaction and Entertainment Research Centre atNanyang Technological University in 2005 where he has worked on several interac-tive digital media projects. Contact: Interaction & Entertainment Research Centre,81 Nanyang Drive, ART-B1-16 School of Art, Design & Media, Nanyang TechnologicalUniversity, Singapore, 637458.E-mail: [email protected]



Call for Papers

Alternative Materialities: Scenography in Digital Performance

The International Journal of Performance Arts and Digital Media (IJPADM) is seeking contribu-tions to a special issue that will engage specifically scenographic thinking about technologiesin performance.

Taking scenography to mean the materiality of performance – the sum total of the perfor-mance space, scenery, costume, lighting, sound, video, and so on – we want to addressscenography and its associated practices in relation to digital media. The widespread adop-tion of digital media across the field of performance arts has proposed alternative materiali-ties that have sometimes radically disrupted existing scenographic practices, andsometimes perpetuated established practices through new means. New possibilities havebrought new ways of thinking and doing, and new goals.

Previous scholarly accounts of these developments have only occasionally acknowledged sceno-graphic practices as such, so we want now to engage specifically with digital media from ascenographic perspective, including their use as scenographic materials and the challenge thatthey have presented to the working practices of (amongst others) set, costume, lighting, soundand video designers. The Scenography special edition aims to generate ‘scenographic thinking’by taking the physical materials of performance, and the performance-makers who manipulatethem, as a starting point in approaching the use of digital media in live performance.

IJPADM is an interdisciplinary, peer-reviewed journal drawing contributions from a wide range ofresearchers and practitioners who are placed at the rapidly developing interface of new tech-nologies with performance arts. Within the journal’s overall remit and the above agenda, themesfor contributions to the Scenography special edition might include, but are not limited to:

• Specific uses of digital media in performance scenography• The impact of new media on traditional scenographic practices• Changing ways of working and collaborating• The intersection between ‘mainstream’ and ‘avant-garde’ scenographic practices• The relationship between the academy and professional scenographic practice• The scenographic as a critical ‘point of view’ from which to observe digital performance• Digital media and the scenographic as a mode of enquiry into performance• Digital tools in the scenographer’s process.

All papers must be submitted by 1st November 2009Intended publication date: April/May 2010.

Details for submissions and notes for contributors can be found at:http://www.intellectbooks.co.uk/journals.php?issn=14794713

All papers are to be submitted to Nick Hunt, Guest Editor.General enquiries can be addressed to Dave Collins, Editor.

Nick HuntHead of School: Design, Management and Technical ArtsRose Bruford CollegeLamorbey ParkBurnt Oak LaneSidcupKentDA15 9DF020 8308 [email protected]


Digitally augmented reality characters in live theatre performances

Documents

Transcript of Digitally augmented reality characters in live theatre performances