159

Performative and Auditory Characteristics of theAudiovisual CD Books Midnight play and Alphabetbased on Kvita Pacovsk6's Picture Books

KuRrranl Utako

From the mid-80s onward, the form of "new medium"' or multimediabecame widely acknowledged as a vehicle of artistic expression, andcomputer-assisted publishing developed by leaps and bounds as a medium ofcommunication. The development of the expressiveness in this field seemsto be fundamentally related to its level of interaction. However, to date, thestyle or structure of multimedia expression has not been extensivelyanalyzed, in contrast to the large number of reviews and socialcommentaries written about the medium. Moreover, despite the fact that thecoexistence of sight and hearing are, of the five senses, conspicuous in thisartistic field,'? consideration of the auditory and temporal rather than thevisual and spatial dimensions of multimedia appears to be particularlylacking. In order to refine this field, it is necessary to investigate theperformative and auditory characteristics of multimedia expressions thataccompany the work as an aesthetic "object" and transform it into anaesthetic "event." The aesthetic "event" in multimedia expressions, or thephenomenological form of interactive multimedia works, has not beenadequately considered as a primary subject in itself. In my opinion, thisstagnation is partially the reason that the identity of multimedia work can beestablished only after a person operates it in a manner similar to that of amusical composition, whose identity can only be established with a player.3

As a case study, this paper focuses on the audiovisual CD booksMidnight Play ("Mayonaka no Oshibai," 1998) and Alphabet (..Arufabeno,"1999), widely known under the name of Kvdta Pacovskii, a Czech painterand picture book author.n rhe former is an audiovisual cD book acclaimedby a Japanese mediology magazine as a "rare affirmative example in onlineexpressions"s at a time when the medium of electronic text was still regarded

Kuntunne Utako is research associate in the Department of Acoustic Design at KyushuUniversity. Her research interests include nineteenth and twentieth century: French music, styleanalysis of Claude Debussy and Gabriel Faur6, and a historical survey of the pnr de Rome. She iscurrently researching music in the animation of Norman Mclaren.

' ICONICS Volume 8 (2006). Published by the Japan Society of Image Arts and Sciences

160 KURIHAU Utako

as a threat to the book market6 and was unappreciated by conservativecritics. According to the article, "Midnight ptay by pacovsM,'? ushers in afundamental progression from conventional books to audiovisual cD books.In contrast to this masterpiece, for example, Just Grandma and Me (.,Oba-chan to Boku to," 1998)8 a representative work in the field of electronicpicture book incorporated sounds and motions into Mercer Mayer'seponymous picture book and remained, experientially, simply an ordinarypicture book electronically reprocessed. The latter audiovisual cD book,Alphabet, won fifteen international prizese and was praised particularly forits "haptic sens"r' and "artistically and musically pureness in operationalfeeling"rr that is "free from any pedagogic direction."r2 In other words, thisaudiovisual cD book is esteemed for its aesthetic autonomy, which differsfrom the appraisal of Pacovsk6's picture books for "unexpectedly offeringthe features of a picture book for intellectual tr&ining.',r:

Both Midnight Play and Alphabet are co-produced not by pacovskd butby many people, including interactive attraction planners Murielle LeGbvreand Nara Kazue. Lefbbvre is the incorporator of DADA Media, and Nara isa contracted employee at NHK Educational Corporation. That bothaudiovisual cD books are generally recognized under the name of theoriginal painter Pacovsk6 alone instead of the real producers is considered asa result of the visual appearance of a work tending to specify the ontologicaldimension of the work as a concrete object. Here, we detect less of anopportunity to appreciate audiovisual CD books as examples of ..new

medium" than picture books as constituting the "old medium" more toreflect the present lack of stylistic analyses on multimedia expression. In thispaper, the schematic parallel between forms of expression in old and newmedia is similar to the contrast between the linear temporality in books andcinema and the nonlinear one in audiovisual cD books with substantialauditory stimulation. Therefore, a "handler-spectator" who operates anaudiovisual cD book must be distinguished from the traditional "reader-spectator" of a picture book. In this paper, this "handler-spectator,' isrefened to as an "interactor."

The sounds and music used in Midnight PIay and Alphabet are alsoexamined, since the phenomenon of sound not only represents the multiplenature of new medium but is also nearest to the temporal Form desAngeschauen'o (form of intuition), which belongs not to the object shortlyafter it is generated but to the performative experience of the event. Whenexamining the relation between visual and auditory elements, it becomesrelevant to consider the trichotomy of in, hors-champ, and off as proposedby Michel Chinon.'5 This is an effective taxonomy from the viewpoint of amovie in which sequences proceed in a fixed order. However, it isinapplicable to the present research in its current form, because the

趾 I,漁蹴瘍臨 ド歯減f阻飢I『露話留澱F 161

experiential order differs from chance to chance; moreover, the sound eventsdirect the interactor to follow the work experience in the audiovisual CDbooks Midnight play and Arphabet In the following analysis, we use not thetrichotomy of in, hors-champ, and offbut the dicf,otomy oi..uu"tgrouno,'and "occurrence," although these *oid, ,nuy sound rather conventionar andamateurish. "Background" refers to the elements that appear on the scene,independent of the interactor's operations. ..occurrence,,

refers to theelements that appear based on the inieractor,s operation.In order to clarify the nature of the periormative experience of aninteractor compared with that of a reader-spectator and to compare thestructural components of the audiovisual cb books Midnignt iloy unaAlphabet, we observe pacovsk6's sryle in prastic works and in;ic;;;ooks.we will then discuss the differences in the experiential order of eachaudiovisual CD book.

1. The Style ofpacovskd1-1. The Free Graphic Artsr6 of pacovsk6

Kvdta Pacovsk6's father was an opera singer and her mother was aforeign language teacher. She is said to have grown up in the midst of manybooks and pictures and was surrounded by mJsic. uniortunatety, rrl. ]r*irr,father was carted off to be killed by the Nazis and she was unable to attendschool for four years, from the age of thirteen.r? At this time, she becameinfluenced by the junk art of Kurt schwirters (lgg7-194g). er,L, gruduu,ingfrom the studio of prof. Emile Filla at the Academy of Applied Arts prague,she operated in the domain between free graphic urt, *i-illurtrationsrs andbegan drawing illustrations for fairy tur"r uv the Grimm Brothers or Hanschristian Andersen' She also began a piciure book of her own, aroundlg52.te

It is interesting that the editors of pacovski{'s work catalogue take note ofher originality in an expression of "interactivity" as early as i995, ..In ,"""ntyears, she began to develop the medium of the picturebook into an objectwhich children can interact with, for their own aesthetic exploration.,,20 If so,what are the elements for the dynamic plasticity observed in her ..freegraphic arts and illustration" without any audiovisualization of multimedia?one notices in her tableaux many retters and numbers, and in her soridworks the familiar tools such as a clock and a pencil. They are at once signsraising symbolic backdrops and beautiful patierns whictr provide rhythmicexcitation to her work in the form of static objects. She says:

I feel the beauty of a letter or a number, which has beenrefined by its cultural history over thousands of years. I stickto used paper and an old score on my own work because I like

162

the feeling of having to struggle on the cultural history that thematerials have.'z'

The letters from A to Z and the numbers from zero to nine are sometimesconstituted as animals or personalities, as in the tableau The Hedgehog'sStory (1987). In this work, the body of a hedgehog is developed with tenconvoluted alphabetic letters. Other times, animals or personalities aretransformed into letters or numbers, as in Mr. Four in One, Fige, Many(1990), a bird with a human hand in a dressed sleeve grooming his back witha long beak and balancing on one leg to become the figure four. Everypersonality dwells in each letter of the alphabet and the numbers are likely tobegin dancing at any moment under Pacovsk6's preference of the ponderouspresenca of a large brush suoke to a thin light line.

Furthermore, Pacovsk6's surrealistic style heightens the reader-spectator's sense of play. Every figure in her tableaux is boldly disto(ed andcontains vividly contrasting colors, especially bright vermilion and shinyveridian. The motif of a clown, generally with a big round tummy,frequently appears in Pacovskd's drawings exuberantly colored and withdecorative accessories. The yellow clothing of a fat clown in Rund undeckig: Ein Formenspielbuch (1994; "Maru, Sankaku, Shikaku," 1995) isfrilled with argent leaves to hansform him into a pineapple. He says, "I am achattering pineapple!" The deformation of a clown continu€s to advance inthe lithograph, A Clown with Knobs (1979-1985). Here, the clown has ahuge cubic body colored vermilion and green, with a big smiling face andwith both sides of the body containing a window with a knob, from whichclowns laugh.

1-2. The Picture Books of PacovskiMost of the illustrations appearing in Pacovsk6's picture books are based

on her own free graphic arts in the past. Therefore, rich color conftast,surrealistic deformation of personalities, and tactile or quasi-tactilevariegation are also inhoduced into her picture books trough the use ofeveryday objects. In one such book, A Srory of the Paper-Made Town("Kami-no-Machi no O-Hanashi," 2000),r'?Mr. Pyramid Pierrot is a

photograph of a rnaquette for the installation Pyramid-Clown in 1994: Mr.Perforated Chocolate is a photograph of a solid work entided DelimitateSpace for a Small Marionette (1994);'?r Mr. Gargle Drum is a photograph ofCircle-Dialogue (1994).! The dear Line, dear Triangle, and dear Square inanotler picture book, Rund und eckig, are ftom the paper sculptures in tlrc198Os: Red-green Sculpture (1982 and 1986) and. The Colar Space Figure(1988). Similarly, several pieces in the lithographs it A Big Clown seies(1979-1985)'?5 have been adapted for the picture book Midnl eht Play (1994),

協羅″湯″船路腸∫蹴よ∬Il薔糠鍛『」留:瀾f謄 163

whose rearrangement into the form of an audiovisual CD book will bediscussed in this study.

Also, the textures of the materials are so varied in her picture book thatnot only the reader-spectator's vision but his tactile r"nt" i, uiro rti-urut"oby the use of materials such as cloth, board, aluminum pupo _J'"u*n,space. Pacovskd thus leads her reader-spectator on u tou, of tf," *o.t,O inriO"her picture book through the use of various tactful .u,J"ir-*iii'i ritingcolors and a screen composition that appears to U" in fiu"fy rnotion. fn"ymay serve as corridors for the reader- spectator,s imagi;ation to incitereciprocal experiences, or the inieractivity, ofpicture booki

In addition, the absence of narrative progress is obseryed as an importantstylistic feature of pacovsk6,s picrure books. For example, I Stori of the,:!:::r:l:

-r:., i5 a rale.of ..a girl with pink hair..who tiu., in i pup.,

made whlte town and is on her way to meet Mr. Sleepy yellow penci'I. Onecannot find any clues about meeting Mr. Sleepy V"U"* p"""if,'V"i *i,f,e1+ tuT ol a page, the reader-spectator encounre$ King Cat, a blue birdwith striped socks, Mr. perfomted Chocolate, Mr. Gargl"e O."_, "i".

f, Ltrue there is scarce exception of Alphabet 11996), a collcdon oi'uioluU.r"le19rs fiom A to Z by the hand of pacovskd, ana fne titii" it"r;rKirg(1992), a.fanciful story of a green rhumb king in search of u prin""rr.However in Pacovskd's picture books. the narraiue Oir..tiuif i, !.neruffyminimiged or eliminated through the use of the compilation t""t niqu"; i.e.,creating an atmosphere of irrational fantasy by accumulating iniiuiAuufworks of her own.

.. Pacovskd's picture books approach opera aperta (work in open fom)dismissing the firm linearity of general picture books. That openiess givesthe reader-spectator a transnormally playlul spirit, as one need' not esiaUtishcontinuity- with the preceding portion of the book. The ."ud"._rp""tuto,

"unimmerse himself/herself in pacovskd's imaginative *o.fO ,.gu.Ji"r. ofwhich page he/she opens first or where he/she closes the book. F-urthermore

the opemess is in a sense a process to forfeit a self-contained entity.

Pacovski4 thus innovates something interactive in her picture books andattracts hemendous attention worldwide_from both children and adults.,u

2. The Productive Framework of Mi.dn ight play and AlphabetSince the audiovisual CD books Midnight play and Alphabet are not

based on narratiye continuities, we are considerably attached not to verifythe authority of their originals but to appreciate them within the frameworkof their polysynthetic production to realize auditory stimulant andperformative motion into the existant visual objects.

The promotional catchphrase fot Mid.night play by DADA Media was"Liberty of action and the wobble (Libeft6 d'action et de d6ambulation).',?7

t64

The other catchphrase by NHK Educational Corporationr was as follows:"It's amazing in this CD ROM that colors, forms, and sounds form acommon language without a language and button operation."d It is not easyfor us to describe the new world constituted by this "common language"since it is a concept without precedent. However, describing its experientialstructure from the viewpoint of performative and auditory characteristicsmay be possible. It is probably safe enough to consider that the samephilosophy holds true in Alphabet as well, because Nara Kazue who led upthe production crew for Midnight Play took over the head of Alphabet justone year after the former.

However, a few differences exist between the two audiovisual CD books,according to the credits and the product information (Fig.l). Several of thesedifferences reflect the rapid development of technology from 1998 to 1999,such as required memory and input methods. Other differences reflect theperformative amenity as for the lineup of collaborative crews. The fivecrews lumped together as "arrangemenf' are detailed on DADA Media'sbwebsite and in the source related to a Japanese TV program by NHK.3'Leldbvre and Eliard who are active in the field of video art created using acomputer program, took charge of coding for both Midnight Play andAlphabet, progranroing artists Peugeot' and Durieus3 took charge of coding,and sound designer Birge took charge of sound just for AlpicDer.

uidniCht PIdr (t998) {lphabet (1999)

:lick, DIas llick, Dl!s, Microphonc, T'"irg,

MA, I2 MB 16 MB.20MB.30 MB

C

)risinal tuib Prcowkn (Cachoslovstia) (var! Pa.!!Rk{ (C&clo,lovstia)

ilun€lle L.tatvrc (rl8ncc)

caFPhilippe P€llscol (Franc!)

)cnis Eliard (Francc)

!,{uriellc t fabvr. (Fmncc)

ircdericDuricu (BelstuIn)

leaFhcqucs Birs€ G6ncc)

)€nis Eliad (Francc)

lara rlaarc (Japan)

14usic lshina Michiru (Jape)

Fig.1: Outline ol the audiovisual CD books Midnight Play and Alphabet

The latter draws our attention to the lack of a music composer, whereas

the former used 6shima, a well-known composer in the field of popular filmmusic.! We can suppose significant differences in the auditory stimulantsbetween two works in the change of the sound artist's position: dshima as a

musician in Midnight Play ndBrg€ as an arratget h Alphabet.

w#i"#*f;fl,gxffi ,T".:*JJ#{.#i,:'""#f 165

3.'The Structural Component ofMrdn ight play anil Alphabet3-1._Th: llrujturat Component ot Minisht itar

,Midnight-Play is comprised of three parts: the pretude, the main content,and the postlude. In the prelude and postlude, u" in,".u"* i"Ji"". ri*"danimation accompanied by music for forty_five seconds, duri,ig *ii"iii."

"Pierot moves about in an old clock, a luad"a u _"rry_go_-o-ni,;;;; .".In the main content, every characrer in the picture 6ook Mr.dz *r-prq Xreconstructed in the audiovisual CD book Midnight p/ay in tlre?m of asubject-specific solutional arraction to enable the interactor to exDeriencethe entire piece of_work as a composirion with five

"t arr.,*fr,i oiir. ff,"attractions in the five sections are summarized based on the ,ypi

"f ,f,"i.description, operation, and the number ofpictures used.

Section A: puzzle game-Drag to link the appropriate parts withthe coresponding sound effect fo.

"very cu.so, po;i,io; u;

build a character-Five picturesSection B: Conduct-Click to determine the time at which the

^ sound_of a drum accompanies a character_Seven picturesSection C: Acrobat dire ctions_Click to

"nyoy u rouna

"if""t,zooming into a character on the screen_Two picturesSection D: Spare clothiug_Click to enjoy u rounO

"ff""t,selecting a suitable hat and clothes for the lower fraf oi tfrebody-Eight pictures

Section E: Hide and seek_Drag to bring to the front a hiddencharacter guided by the sound_Six pictures

The five sections are clearly delineated and are not intermingled. Eachsection appears in random order (Fig.2). An interactor.uy rnouJ frorn on"section,to another only after conquering the subject in the pie.;;.;, ;;;ti""

Alrnough the interactor inMidnight Play bears someresemblance to a computer gameplayer comprising subject-specific solutional attractions, thetwo roles are not very similar.Playen often compete with eachother in terms of thei progress inthe game, while the interactorwill never compete with others interms of his/her progress inplaying the audiovisual CD book.Moreover, while a player

,S∝ tl。輸

166 KUHHARA Utako

experiences each attraction in a predefined order, an interactor in MidnightPlay does so in an indefinite order. This non-linearity of production andreception guarantees that the CD book serves as an expressive and aestheticexperience.

3-2. Sound/Music Elements of Midnight PIayThe primary auditory stimulant in the prelude and the postlude of

Midnight Play is "background sound," a melody of eight bars in four beatsby Oshima (Fig.3). From the viewpoint of its impressiveness, this melody isconsidered the musical "theme" of this work. It begins with a minor, withthe first motive (to turn the keynote around) and second motive (chromaticscale-descent) played twice. The music then continues and fades out withoutany impression of tonal determination. One might suppose "the dolls ofmidnight theater" as the conceptual motif of this melody, since the wonderof the midnight theater is expressed through the indefinite tonality and theuse of the augmented fourth, which deviates from the norms of Westernmusical tradition. Mechanical dolls are expressed by multiple use of thetechnique of frequent repetition.

While the background of the prelude and the postlude consistsexclusively of this music, many types of "occurrence sounds" are observedin the main content. Instrumental music using the tambourine and marimbaplays in the scene in which the characters play the violin and the drums(Section B), as well as in the clothing montage and hide-and-seek scenes(Sections D and E). The puzzle game and the acrobat directions make ampleuse of sounds created with the help of naturally occurring materials, e.g.,rolling gravel and splashing water (Sections A and C). Though the creator ofthe sound collection is regrettably not identified in the production crodits theadoption of individual sounds at each moment depends on the position of theclick on the screen. Thus, the sounds enable the interactor to work out thesubject in each section and, as we have seen, helps him/her to step forwardfrom one section to another according to subject conquest.

3-3. Structural Component of Alphabet

趾 :器鵬 ∫蹴 朧 臨盤i群

燎柵

mヽll番聯RS

A.3.GDュ ニGH.:.J.11,vヾ

ff夕RS.

II!TYP----- r

Fig.4: Diagram ot Atphabet

Here, the Menu variety displayed at random is interesting. Each of thethree different Menu screens consists of a visual and auditoi backgroundset:

(a) Red screen-Asian or African percussion instrument music(b) White screen-Modem jazz music(c) Green screen with bluish green spots-Asian string music

when an interactor inputs a letter through the keyboard, the letter outputted

167

Alphabet, like Midnight play, consists of three parts: the prelude, themain content, and the postlude. While the prelude "o,nprir", a fixed

animation of approximately forty seconds, it differs ,tu"tu.ully rrom thos"of the previous works: many types of music are played at random during thepostlude, with an assured display of credits. tn aoa-ition to the music, soundeffects are heard when an interactor moves the mouse over the characters inthe postlude.

Three sections are included in the main content: the Menu, Section A,and Section B (Fig.a). To some extent, this resembles a layered strucrure,i'e', the progressive graduation to "another stage" in a computer game.However, the interactor in Alphabet differs from the player of a computergame in that he/she has the option of moving to the other iuy"r, ut any pointin time. Movement from the Menu into section A or from Section A intoSection B is possible using the retum key, and movement from Section Binto Section A is possible by clicking on th" upp", right comer of the screen.In any section, an interactor may retum to the Menu by clicking on the upperleft corner ofthe screen.

168

by Pacovsk6's brush on the screen is the visual occunence. A child's voiceenunciating the letter is the auditory occunence.

Additionally, when the interactor wishes to proceed without inputtingcharacters, the program responds by continuously producing randomalphabets by sounds and brushstrokes every four to six seconds. Thercfore,this audiovisual CD book, maintained in this state of automatic performance,

acts something like an installation, delighting those around the computer.We now assume that the intelactor carries out a keystroke and progresses

to Section A and Section B. With regard to the auditory aspects of the twosections, the sound occurrences are produced by means of certain noises orinstrumental sounds, depending on the interactor's keystroke. With regard tothe visual occurences, a great variety of screen compositions are displayedfor each of the twenty-six letters, reflecting the variegation of the originalpictures by Pacovsk6. Although a synthetic description of the visualcharacteristics of each section is a difficult task, it can be stated that, overall,Section A is based on the concept of "appearance" and Section B on"transformation."

For example, when "F' is inputted in Section A, a red sEing in the shape

of F appears oa a white screen. When "W" is inputted, the white plane state

of W in the mechanism of Pacovskf's solid work is displayed and colorfulfragments app€ar one by one according to artitrary keystrokes inputted bythe interactor; this results in the completion of Pacovsk6's W. In Section B,when an interactor presses a key, any of the following three processes is

shown: the parts of a letter break and scatter, the letters transform intoanother letter, and a mobile doll rotates to form various letters.

By the way, the mode of operation in Alphabet should be highlighted as

another remarkable performative characteristic. The productive chief Nara

and the coder Durieu refer to the imagery of "flying a kite" behind the

coding:

I adhered to the mode of operation while ptodncing Alphabet.

[...] In order to realize comforts like flying a kite, physical

elements such as gravity and elasticity were incorporated intothe programrning.r5

I have always been a lover of kites, astronomy and the beauty

of the natural world. Funhermore, thanks to my raining as an

engineer, I like to use math and the laws of physics [...].Gravity, the forces of friction, accelerated motions, atomicforces, elasticity, Lissajou curves, derivatives, complexnumbers, fractals..... The Alphabet overflows with them.x

7躍″褥″齢剛謝湖謝∬I譜:槻盤ぎξf淵≧撫 169

Needless to say, the technical developments that occuned one year after theproduction of the previous work contributed to the realization of the"comforts like flying a kite', in Alphabet. Such developments included asignificant increase in computer memory and the familiarization of vanousinput tools.

- Generally, a reader-spectator directly participates in the physical worldof a picture book using his/her ..hand." An interactor may be concemed thatan audiovisual CD book would cause him/her to lose touch with the texrureof paper and the experience of a conventional book. Therefore, to guaranteehis/h€r satisfaction when using an audiovisual CD book, it is neciessary tofulfill this need through the real-time narure of the work, by wtrlcfr tleinteractor will realize the relevance of his4rer inputs to the outputs from thecomputer, i.e., he/she will recognize the interaction. This need is furtherhighlighted when the main outputs constitute a continuous transformation ofcharacrers in Section B of Alphabet.

3-4. Sound./Music Elements of AlphabetThe musical and sound sources are presumably developed by the hand of

Birg€, as a "music composer, cine artist, sonority designer, and multimediaauthor."rT

The background music in both sections A and B consist of twenty_sixtypes of content, depending on the number of letters of the alphabet that arerandomly chosen at each trial from fourteen types" of recorded traditionaland popular music from different parts of the world. Moreover, a largenumber of occurrence sounds correspond to an operation. These sounds canbe classified into the following six types:

(1)The enunciation of the alphabet (from A to Z) ir the voices ofJapanese and French children

(2) Laughter and the mimicry of animals by the same children(3) The vowel sounds in the chest voice of an adult(4) Vehicle sounds(5) Sounds of wind and percussion instruments(6) Amplified clicking sounds

The interactor's operations in certain scenes do not always cause thesame occurrence sound but instead cause two or three different sounds.Since the combination differs for every session, the occunence sounds inAlphabet do not provide any hints to lead the interactor into the nexrsections. Although they retain the obvious relationship with an interactor,sacdons by appeadng on the basis of an interaction and indicate the executionof an operation, they are not signals as they do not entirely co[espond to the

170

operation and to visible objects. This ambiguous nature of variousoccurrence sounds in the audiovisual CD book Alphabet even occurs inletters and numbers created by the free graphic art of Pacovskd, whichetfects the deconstruction of linearity that makes the picture bookinteractive.

4. The Experiencial Order of an lnterac tnr In MidniSht Ptty and AlphabaWhen the diagrams of the two works' main contents are compared, at

first glance, the diagram of Midnight PlLy appears lo be much better devised

Ihan that of Alphobet, except that Alphabet is obviously devised for the

prelude and the postlude. However, we must notice tlat the something in our

experiential reality generally drops out of its spatial grasp in diagrams,r'particularty in its ordinal or temporal aspect.

In fact, the random nature of the two works' struchnres is not considered

enough in the above mentioned diagram, because these two audiovisual cDbooks are works that "are formed gradually" by the operation of an

interactor. In order to examine the Possibilities of experiential Processes, Iwill now compare some examples of ordinal interaction with both works,

assuming that a certain interactor experiences each work twice (Fig.5).

In the first session of Midnight Pla), an interactor may begin by

comprehending the subject in Section B (musical instrument performance

directions) and concludes hiVher work experience after proceeding to A' D,

C, and E (Fig.sa). In the second experience, he/she begins bycomprehending the subject in Section C (acrobat directions) and proceeds in

the order of C, A, E, B, and D (Fig.5b). Although the interactor may then

attempt one or two of the sections from A to E, this may soon lead to

boredom because he/she has already experienced these sections. On the

other hand, an interactot of Alphabet etcounters, in his/her first session, nine

types of characters in the menu, eleven types in Section A' and six types in

Section B (Fig.5c). In anothet session, after the interactor encounters three

characters in the menu and eight in Section A, he/she retums to the menu

and enjoys frve types of audiovisual presentations. Then, via one cha.racter in

Section A, he/she encounters five characters in Section B. Afterencountering three characters in Section A as well as three in Section B,h€/she finally concludes the session (Fig.5d).

The following points emerge from this comparison of the two works inthe experiential processes.

ln Midnight Play, as a subject achievement attraction, one tends to avoid

tackling the same subject because a feeling of sufficiency is prepared foreach of the five types of attractions. Therefore, the plural trials in MidniShtPlay are always described as singleJayered, like a star drawn without liftingthe brush from the paper. On the other harLd, Alphabet contains only about

熙 常∬論鵬 ∫糧讐∬鷺』:糠飢1昴r習譜盤お 171

ｕ　　　Ａ

　

　

　

Ｂ

Ｍｅｎ血血

Fjg.5a (top tett):Ordinal experience of the main content ol Midnight ptay__axample 1

Fi9.5b (top right): Ordinat experience of lhe main content of Md;igh t itay_exa,mpte zFig.sc (bottom left): Ordjnal experience of the main content ot 4/p;abet___exampte tFig.5d (bottom right): Ordinat experience of the main content of A/ph abet_example 2

three types of simple concepts: the .,presentation" of a character symbol anda character sound in the menu, the ..appearance,,

of each individuai charactersymbol. in Section A, and a ,,change" in the character symbol in Section B.In reality, this abstract setting is experienced as fifty_two or possiblyseventy-eight different individual components, dissipated into the twenty-sixletters of the alphabet. Since no specific achievemeni subject is prepared, theinteractor goes back and forth freely and unrestrictedly between multipleattractions. Therefore, the plural trials found in Alphabet can no longeranymore be described as a linear process.

5. ConclusionIn order to determine one of the stylistic possibilities in the multimedia

attraction, we have examined the performative process and the nature ofauditory stimulation in two audiovisual CD bo;ks_Midn ight play andAlphabet-starting with the examination of the .,interactivity,, in pacovskri,sgraphic arts and picture books. We can srate that these new media productsacquire their noticeable playful spirit, by increasingly splitting apart thelinearity of the old medium. We are now prehensile to the incimation ofnonJinearity that steps on each work. lrr Midnight play, where the relationbetween input and output is clear, the interactor is supposed to solve thesubject in each section according to sound clues. He/Jhe steps forward toanother section automatically just after the completion of each section. Thejourney through each of the five sections is randomly caried in line. The

172 KUroHARA Utako

playful spirit is more enhanced here than in the original picture book,implying higher development of temporal mobility. On the other hand, inAlphabet, where no stable particular relation exists between the interactor's

operation and sound selection, the interactor is not urged into any directionby hearing stimulus. Moreover, a greater variety in the order of twenty-sixdifferent scenes is built into the two sections. It is up to the interactor toprogress into the following section, to return to the former section, or stay inthe same section by going into another scene. Here, the highest degree ofnon-linearity is attained in this series.

In Pacovsk6's picture books, both the high involvement with tactile and

symbolic materials and temporal mobility, or the fragmentation of narrativelinearity, seems to trigger the concept of such playful attraction with much

room for choice in the audiovisual CD books. However, it is important to

remember that, conceivably, both audiovisual CD books aim not just todemolish the linear process of the conventional picture book but to compose

a non-linear process based on the nature of their medium. This creates that

same flexible relationship between visual and auditory stimulus that may

serve the aesthetic quality of the new medium.

Notes

I am gnteful to Nakamura Shigenobu for his indication of these audiovisual CD books

as well as for his warm encouragement. My equal gratitude goes to the juries ofICONICS's Editorial Committee for their critical opinions. I would like to thank its

chairperson, Kaneko Masakatsu, for his kindness in patiently offering his scholarlyinsights, which have been exremely helpful to me in enriching my views.

l. The term "new medium" gained popularity in the mid-1980s. "New medium"refers to media characteri-zed by iriteractivity and content diversity, such'asCATV, satellite broadcasting, and the Internet, as opposed to existing "oldmedium" such as newspapers, radio, and television. Within this context, themedium of a CD-RONlas a package requiring a comPuter drive is generallyclassified as "new medium".

2. As for the other fields, for example, spatial representation is reflected in theform of pictures and temporal representation in the form of music'

3. With regard to the "identity of a musical composition," the writings ofRoman Ingarden (1893-1970) merit attention. Ingarden investigated thismatter from the phenomenological perspective, comparing the ontologicalstatus of a score, a performance, and the act of hearing a piece of work.

4. Each of her picture books has been translated into German, English'Japanese, French, Italian, Portuguese, Dutch, Chinese, and other languages.

The original title is not always available because the original language ofpublication and the number of words have not been detailed'

S. -Baitorlat

Stafl '"The novelty of the electronic picture book Midnight Play,"Book and Computer, February 1999, originally in Japanese and cited with a

Perfomativc ild Auditory Chmcteristics of the Audiovisual CD BoksMidnight Ploy ud Arprrab"r bed on Kvata pacovskd's picturc Boks 173

translation by the author. The english summary reads online, ..Midnight play,a New Kind of Electronii picture Book" .t ttp,/**rltron.o.net/archive/990204.htm1>, (22 December 2005.)

6' Today,it is recognized that electronic publishing is not a real threat to bookpublishing.

7. Editorial Staff, "The novelty of the electronic picture book Midnight play,,op. cit.

8. The picture book (English edition) was published in 19g3; the audiovisual9P Poof. (Japanese Mac version), by Interprog in 1992; und th. auJiouisuarCp boo,k (English Mac version), by-Random i{ouse in iSS:. fn *,i, nu.r"rypicture book, a summ€r day at the seaside of a small rat boy ctitter ano trisgrandmother is described.

9. ft: prizes_won by Midnight play are not detailed. Asfor Alphaber, see thewebpage of F. Durieu, <http://www.lecielestbleu.comfrtrnl/alp'habet.h to,,, (zzDecember 2005.)

10. "Rezensionen" in Die Zeit, July 27, 2C/Jf|,.I l. Ibid.I 2. "Rezensionen" in Siiddeutsche Zeitung,December 22, 2005.13. Sonoda Toki, Hundred picture Boolis we wanna Read with our children,

(Tokyo: soudo Bunka_Sha, 1997), r04; originalry in Japanese ano citJ-wittra translation by the author.

14. Edmund Husserl, zur phiinomenologie des Inneren zeitbewusstseizs, ed.$gd9lf Biihm, (Haag: Martinus Nijhoif, 1966), I l.

15. Michel chion, te son au cin1mi (paris: cairiers du cin6ma, r9g5),32-39.The "in" is defined as a sound whose source can be seen on'ttre sciein anothe "off is defined as a sound emitted a space-time away from irr"-r.r".n.The- "hors champ" is a sound assumed to exist if it is in th; sp;;;

"ontil"ou,to the space that is conterminous to the screen being viewed, dtrrffi tnesound source does not appear on the screen.

16. The term. "fr99 graphic arts" is used to indicate a convergence of painting,lithograph, silkscreen, sculpture, etc., in her ..BiographllA Not"r;, fuaro

. _ Pacovslui, (Tokyo: Chihiro Art Museum, l9g5), 1,37:17. Matsumoto Takeru, 'sensitivity polished in prague," Kvdta pacovskd,

(Tokyo: Chihiro Art Museum, 1995t, l2S.18. "Biographical Notes", Kvdta pacovski, (Tokyo: chihiro Art Museum, 1995),

t37.'lKvdta Pacovsk6", Writer's Who,s Who, Nichigai Associates, Inc., 2003.,<http://www.nichigai.cojp/web-service/sakka.trtrit',

1to Decem6er 2001. '"Biographical Notes", op. cit., 137.originally in Japanese and cited with a translation by the author. MatsumotoTakeru, op.cit., 127; Kawai retsuo, "An Artist on the Borderline: Kvdta119.o.".t1, a lr4agician o-f. C_9l9rs and Space" Kvdta pacovslca, (Tokyo:Chihiro Art Museum, 1995), l3Z.

{ -story of the Paper-Made Town is a commissioned work by the chihiro Art

Museum.-In this work, she drew and wrote some Japanese characters at therequest of the Japanese Organization of picture Book Culture.Kvdta Pacovskri, (Tokyo: Chihiro Art Museum, 2000), 103.Ibid., 104.Ibid., 34.Her picture books are translated in Chinese, Danish, English, French,

９

　

０

１

１

　

２

２

22.

２３

２４

２５

％

174 KURMM Utako

German, Italian, Japanese, Netherlandish, and Portuguese. She has wonGrand Prix (1965) and the Golden Apple (1983) at the Biennale ofIllustrations Bratislava, the Grand Prix (1988) at Catalonia, the HansChristian Andersen Award (1992), and so on. In Japan as well, her work has

been showcased at exhibitions organized by "Bologna Fiera del Libro perRagazzi" (Itabashi Ward Art Museum, 1994) and at her private exhibition"KvEta Pacovsk6" (Chihiro Art Museum, 1995). She received an honourarydoctorate of Design at Kingston University.

27. The author is unsure if this is conectly regarded as a "catchphrase" since thiswebpage source was found at least after March, 2004. <http://www.dadamedia.com/artCat.php3?id-article=1 55&id-rubrique=46&init=141 ), (9January 2006).

28. NHK Educational Corporation Co. is the company that designed andproducted Midnight Play and Alphabet.

29. Originally in Japanese and cited with a translation by the author.30. DADA Media, <http://www.dadamedia.com/>, (9 January 2006).31. In TV hogram Digital Stadiwn, supposedly broadcasted on June 10, 2000.

<hup://www.nhk.orjp/digista/onair/2000/1 l-06l0.html>, (22 December20os).

32. Peugeot is an independent programming artist, and expresses his "eagemessto the macintosh technology,",http://www.dadamedia.com/artCat.php3?id-article=21&id-rubrique=53&init=141), (9 January 2006).

33. Durieu has a website LeCielEstBleu, <http://www.lecielestbleu.coml>, (22December 2005).

34. Oshima is skilled at combining the words of Westem traditional music and

the tone of folk instruments with the help of a synthesizer. Wicked Wives("Gokud6 no Tsumatachi," 1996-2001), the Diary of a Fishing Nut XII("Tsuri Baka Nisshi XII," 2001), and Godzilla vs. Megaguiras ("Gojira taiMegagirasu," 2000) are representative of her work.

35. <http://www.nhk.or jp/digista/onair/2000/l l-06l0.html>, (22 December2005). Originally in Japanese and cited with a translation by the author.

36. "Alphabet" in the webpage of F. Durieu, <http://www.lecielestbleu.com,/trtml/alphabet.htm>, (22 December 2005).

37. A webpage of Jean Jacques Birg6, <http://www.drame.or$/BirgeBio/birgeaccueil.html>, (22 December 2005).

38. In the author's own interaction, lasting for approximately eighteen hours, thetrial interaction allotted for counting the types of background music wasapproximately seventy minutes for twelve repetitions.

39. Nicholas Cook, Music, Imagination and Culture, (Oxford: Oxford UniversityPress, 1990), passim; especially Chap. 3 "The perception of form: sometests" in "Musical form and the listener. "