1

Eyes to hear.

A study about an accessible way to translate music in film.

Relatore: Prof.ssa Dina Riccò

Alicia Gonzalez Fernandez

2

3

Eyes to hear. A study about an accessible way to translate music in film.

Politecnico di MilanoScuola del Design

Laurea Magistrale Design della Comunicazione

A.A. 2018-2019

Alicia Paule Gonzalez Fernandez858510

RelatoreProf.ssa Dina Riccò

4

__________

ABSTRACT__________

THE AUDIO IN THE VISUAL Study about the accessibility of the sound in films

1. The importance of the invisible 1.1. The marriage between visual and sound

1.2. Musical Communication Process 1.2.1. Models of Film Music Communication 1.2.2. Functions of the music 2.2.1. The illustrative functions 2.2.2. The implicative functions

1.3. The induction of emotions 1.3.1. Laws of emotions 1.3.2. Understandemotionsinfilmscore

2. Audio Translation and its accessibility

2.1. The evolution of Audio-Visual Translation 2.1.1. AVT modes and apparition of new media 1.1.1. The Dominant Types 1.1.2. The Challenging Types

2.2. The spread of AVT 2.2.1. Collaborative subtitling 2.2.2. Issue mapping on the web

3. The challenge of music accessibility

Eyes to hear.A study about an accessible way to translate music in film.

7

9

11

11

1414171721

242426

30

323333

3736

38

39

34

5

__________

COMMUNICATE THROUGH VISUAL STIMULI Visual exploration to hear music

4. Interaction between senses

4.1. One sense triggers another

4.2. Synesthesia and cross-sensory implication 4.2.1. Sensoriality and accessibility 4.2.2. Visual Representation of the sound

5. Eyes to hear

5.1. Typography and visual sensation 5.1.1. The typographic experiments 5.1.2. Personality of the typography 5.1.3. Sense of voice and musicality 5.1.4. Kinetic typography: (e)motion 5.2. Motion, emotion and shapes 5.2.1. The shape of sound 5.2.2. The emotional value of line 5.2.3. Case studies Hugo Meier Thur narrated by M. Haverkamp Oskar Fischinger’ Experiments ICE Cinema and the peripheral vision

43

45

45

474849

53

5353555760

63636467676972

6

__________

KINETIC SUBTITLING PROJECT Anaccessiblealternativetotranslatemusicinfilms 6. Kinetic subtitles’ production

7. Guidelines to alternative subtitles 7.1. Starting point 7.2. Creative process 8.2.1. Hatred Scene 8.2.2. Bliss Scene

8. Evaluation form 8.1. Method (sampling and research) 8.2. Data visualisation 8.3. Discussion 8.4. Limitations

9. Conclusion and possible future applications

__________

SOURCES Bibliography Sitography Filmography

__________

APPENDIX Digital methods research 1 Digital methods research 2 Italian form French form

75

77

7979818187

949495

101101

103

106106110110

114120122128

7

ABSTRACT__________

This thesis studies the music translation in film in order to understand and promote its accessibility for deaf and hard-of-hearing. Music plays an important role in landmarking significant experiences and spaces. Classics such as 2001: A Space Odyssey (1968), Once Upon a Time in the West (1968), or even more recently Dolan’s film such as Inception (2010), only to mention a few, have lived in time in great part due to their ear catching musical scores. In the audio-visual, as the name indicates, there is as much valour in the audio as in the visual. It’s then unthinkable to study one part without the other one. The importance can be found in what they communicate together to the spectators; the global perception of the situation presented on the screen. Although music often arrives in the background of the stage, JR Julien insists already in the 1980’s on its essential contribution in movies. He observed that music delivers specific emotions that figure in the narration.

Considering film score as a fundamental, narrative and emotive element, this study aims to propose an alternative to the traditional system of translation. The thesis explores the synaesthetic relationships between vision and hearing through topics such as sound, writing, forms and their meaning.

Typographers and designers have played for long with shapes and letters to enhance textual meaning and to deliver the actual emotion of the sentences. The thesis hypothesized that kinetic text may be used to communicate a broader range of paralinguistic sensation and thrill that music delivers than it is possible with the static text. Already in film title sequences, kinetic typography has been widely and successfully used because of its ability to express the mood of the movie. Nevertheless, its

8

potential has rarely been used outside of these contexts. In this research, animated typography is brought directly at the centre of the movie in order to design subtitles able to deliver the pace of the score and its emotive content.

The thesis is divided in three parts. The first one aims to understand the actual situation of music translation, emerging technologies and the accessibility of those translation methods for deaf and hard of hearing. The second part dig for visual alternative to the deliver music by the use of forms and by playing with the typefaces. Finally, in the third part, I will propose an alternative translation with the use of kinetic subtitles for two extracts of the film Birdman.

With this pilot study, I hope to arouse interest for further researches on kinetic typography in order to enrich cinematic experience not only for the hearing-impaired persons but for the whole audience.

9

PART 1

THE AUDIO IN THE VISUAL Study about the accessibility of the sound in films

10Cover retrieved in the Making of ‘Jaws’ (1975)

11

1. THE IMPORTANCE OF THE INVISIBLE__________

1. The marriage between visual and sound

Black screen, dark room, the movie begins. During the summer of 1975, on the island of Amity, a group of students celebrates on the beach. A young woman moves away from the group for a midnight swim while her flirt falls asleep on the beach, drunk. After a few breaststrokes, the young woman is caught and disappears in the water, screaming with terror. A few seconds later, the sea regains its nocturnal calm. A monster just carried her away. This is the dark atmosphere in which Steven Spielberg took us for one of his greatest successes in horror cinema: Jaws. __________

Figure 1. Retrieved in Jaws, Stephen Spielberg (1975)

12

What about removing the soundtrack of the movie? The scene becomes abstract, the fear set up much later. Deprived of sound, the breaststroke scene seems gentle and peaceful; a swim under the moonlight. Remember that this scene was almost unbearable with the music? The questioning arrives when she struggles in the water, but for now no reason to panic, no scream. Is she under attack? Carried away by the current? Finally, calm comes back, nothing very terrifying but a lot of questioning. What was clear to us thanks to the input of the music now seems blurry.

The soundtrack of John Williams is very functional, meaning it express itself through a crescendo of two notes played in repetition only when the shark arrives. It’s simple and effective. Music instead of highlighting the image from a distant point of view, runs after it in order to accentuate it. The Jaws Theme belongs to the predator and only to him, which is a bit at its height since the shark actually appears on the screen few times.So why continue to use the verb watch a movie when, in the end, the audio-watch it? Paradoxically, Quebecers use the verb ‘listen to’ a film. By watching a film, we almost forget the importance that sound could bring to our ears. The intention is not to say that the sound outtakes the image, but that one perception influences the other and transforms it. The combination of sound with the visual creates what Michel Chion would name an «illusion audio-visuelle» (Chion, 2014). A curious relationship between the sound and the visual that is not obvious at first, where the sound enriches an image until giving the impression that information is released naturally and already contained in the image itself.

«Films are 50 percent visual and 50 percent sound. Sometimes sound even overplays the visual» (Lynch, 1998)

Who more than David Lynch itself to illustrate this philosophy? His world is filled with disturbing supporting role, troubling places and here music plays an essential role. Either off-the-wall, intense or unsettling, it never leaves us indifferent. When we talk about Lynch’s music, we also have to talk about the Italian-American

13

compositor Angelo Badalamenti, with whom the film-maker worked regularly. We owed it the slick of troubling synthesizer that is so characteristic from Lynch works.

As early as Eraserhead (1977), its first feature film, music explodes the audience’ senses and bring a dramatic contrast to the visual, it even reinforces the unreal and fantasy aspect of the movie. In that respect, the strange scene where the Lady in The Radiator sings «In heaven, everything is fine», a song written by the cineaste itself. Thorough his career, he went along with the crowd by creating his own music supported by his new acolyte Badalamenti, their collaboration started for the movie Blue Velvet but went along for years for other famous works such as Twin Peaks or even Mulholland Drive (Valenzuela, 2008). Lynch also picked his song from famous artist such as Elvis Presley with Love Me and Love Me Tender that occupies a central role in the movie Sailor and Lula. For the movie, Nicolas Cage performed the two songs with a certain talent.

In short, Lynch’s movies are not restricted to frightening synthesizer but dig in all kind of music style, the goal is to explore dark nooks of the American collective imaginary. We couldn’t say that his playlist is homogeneous but that’s a good way to remember mythical scene of the director, one of the most unique and oddest of his time.

14

2. Musical Communication Process

2.1. Models of Film Music Communication

Many models have been proposed to express the aspects of the film music communication. A tripartite model was first proposed by Campell and Heller in 1980, consisting of a composer, a performer and a listener. Kendall and Carterette expanded this schema into an elaborating model (Figure 2) expressing the inter-relationships between those three angles. The schema involves various phases and states of coding, decoding and recoding.

«In the context of music communication, the degree to which a composer or director succeed in communicating a musical message is in direct proportion to the level of agreement between the emotional and/or expressive intent of the message and that perceived by the listener» (Lipscomb and Tolchinsky, 2004).

The process starts with an intended musical message that is recoded from ideation to notation by the composer, then recoded from notation to acoustical signal by a performer, and finally recoded from acoustical signal to ideation by the listener.

__________

Figure 2. Retrieved in Kendall and Carterette’s Music Communication Model (1990)

ComposerUnshared

Implicit & Explicit knowledge

PerformerUnshared

Implicit & Explicit knowledge

ListenerUnshared

Implicit & Explicit knowledge

SharedImplicit & Explicit knowledge

NotationalSignal

AcousticalSignal

ContextEnvironment

ContextEnvironment

ContextEnvironment

15

In 1988, Marshall and Cohen proposed a first Congruence Associationist Model that suggests that the music altered the meaning of the film depending on the degree of congruence between the audio and the visual components.

Cohen, in 2001, reworked one of her model (Figure 3) and proposed the Congruence-Associationist Framework for Understanding Film-Music. The complex model suggests that the score directly affect the subject passing by the sensory surfaces of the spoken narrative (speech), the visual images and the musical sounds. The overall structure of the model extends to a top-down process based on experience and LTM (long-term memory). Level A represents bottom-up processing based on physical features. Level B represents the congruence based on semantic and syntactic attributes. Level D represents the top-down processing, it’s determined by an individual’s past experience and the retention of that experience in a long-term memory. Level C is where the observer’s conscious mind is, an in-between level B and D, where information is transfer to STM (short term memory). Lipscomb and Tolchinsky precise that this model is based on a visual primacy and they express reservation about this assumption and suggest additional researches.

Authors maintains that on the one hand, music, in a cinematic context, may move above classical boundaries. To their point of view, Lipbscomb and Tolchinsky in The role of music communication in cinema (2004)

«Film music is one component of a spectrum of sound that includes the musical score, ambient sound, dialogue, sound effects, and silence. The functions of these constituent elements often overlap or interact with one another» (Lipscomb and Tolchinsky, 2004).

Components became even more blurred as the film evolved in the 21st century. It’s now almost impossible to consider separately the components. As an example, they often take the orchestral hits of Star Wars saga, where the music merge with the sound effects and the dialogues, all support and augment the emotional impact of the visual image.

16

On the other hand, they maintain the complexity of the relationship between sound, image and audience, each constituent having its own memory, presence and role. The role of the composer is dramatically influenced by the wishes of the director, the performers that record the score cannot be separated by the sound editor, their work will directly influence the visual product and finally the listener-viewer will get this work on his way, depending on his mood and state of attention. Authors quote the Gesamtunstwerk, the total work of art that the cinema has become. Richard Wagner used the exact term for the first time in 1849, in his essays Art and Revolution1, where he explained is idea of unifying all works of art via the theatre.

1 Die Kunst und die Revolution is a long essay by the composer Richard Wagner

published in 1849. The essay talked about the role of art and opera in the society.

__________

Figure 3. Retrieved in Cohen’s Congruence-Associationist Framework for Understating Film-Music (2001)

LTM

A

B

C

D

STMSpeech

Speechsurface

Visualsurface

Musicsurface

STMMusic

STMVisual

Narrative

Speech

meaning structure

Speech Music

meaning structure

MusicVisual

meaning structure

Visual

17

story. In this category the functions are born directly with the content of the movie and the iconic continuity. From this category, Julien digs three functions.

• The decorative function

This feature refers to the presence of music that is fully integrated into a specific location where it would normally be broadcast. According to the author, it gives an immediate recognition by the spectator of the social space where the action takes place. The music is inherent to the place and the action. The source of the sound is being filmed or not (examples: concert, popular festivals, nightclubs, bars, cafes). Julien explains that diegetic music, but also extra-diegetic music can perform the same function.

Lipscomb (Lipscomb, Kendall, 1994) makes the same suggestion but starting from a broader framework. The use of music to represent and characterize the place of action could be part of its function; music conveys the general mood of the movie and its sub-functions; sphere of action, quality, size of space and placement in time. Those features proposed by Lipscomb is very ample and will be resumed later in other functions as well. The characterization of the places of action by music, conceptualized in several ways, can be observed as a common practice. The immediate recognition of the place of action and the non-ambiguity of the representations are

2.2. Functions of the music

In 1949, Arthur Hoeree, a Belgian movie compositor, talked about three particular function of the music in movie, first to cover the sound of the projector; then to screen some disruptive sounds; and finally, to enrich the movie and the rhythm of the images (Hoeree, 1950). Music film helps understanding the movie, it stimulates and leads the feelings; it creates an atmosphere and prepare the audience to receive auditive emotions. Nowadays we can find an uncountable number of functions and adjectives that describe the music, to go towards an alternative translation, it seems important to me to study how music operates on the image. In this chapter I’ve based my categorization on Jean-Remy Julien’s researches (Julien, 1987), professor of musicology at the University of Lyon II and director of research at the University of Paris Sorbonne and Scott D. Lipscomb (Lipscomb, 1994 and 2004), Director of the School of Music from the University of Minnesota.

J.R. Julien distinguishes two categories of functions; the illustrative functions and the implicative ones. Those two functions can be developed in different subcategories in order to reinforce the concepts.

2.2.1. The illustrative functions

This function is born to accompany the image and to complete it. This function illuminated the film and underline the

18

very important in the Hollywood cinema. Music has been part of this feature of Hollywood cinema for a long time. This immediate recognition has a strong connection to the spectator’s schematic expectations. The faster it recognizes the place of action and maybe the status of a character, the more accurate are his assumptions and predictions of a following sequences or the whole movie.

In summary, the film uses the musical and, more general, the cultural, knowledge of the spectator, who can therefore foresee or establish a context so that their predictions can be more precise in relation to a specific place such as the characters, their culture or even the era.

• The symbolic functionOne of the most effective ways in which a musical score can augment the narrative is to express the unseen implications by underlining the narrative. Music can convey character; a director can choose to define a character by sound or by the music. In this case the music follows the heroes, it becomes its sound memory. What about Amélie without the music of Yann Tiersen? The film owes its success so far to the original soundtrack. The accordion is at the centre of the movie as much as the principal heroine, it follows its path and construct the retro atmosphere of Montmartre. As cited before, in another example of this symbolic music are the strident notes that goes with the shark in Jaws.

This function quoted by Julien can join the idea of a leitmotif:

« […] a theme, or other coherent musical idea, clearly defined so as to retain its identify it modified on subsequent appearances, whose purpose is to represent or symbolize a person, object, place, idea, state of mind, supernatural force or any other ingredient in a dramatic work» (Whittall, 2003).

The film-music scholar Frank Lehman, from the Tufts University, works on an up-to-date Complete Catalogue of the Musical Themes of Star Wars2 which can be found online . The catalogue

2 Lehman F. (2019), Lehman Star Wars Leitmotif Catalog. Retrieved from https://

drive.google.com/file/d/1xJ0Jj-mLfOPUCtcAm_HDGIkFwvHL5gbX/view

19

now includes fifty-five distinct leitmotifs and forty-three Incidental Motifs, which, Lehman says are, «musical ideas that occur in multiple cues but lack substantial development or symbolism» (Lehman, 2019) in other words, they do not meet criteria for proper leitmotifs but nonetheless possess dramatic significance. We could take the famous example of the Force Theme, the musical theme repeated across the musical score of the saga. The theme is a signature line of music and one of the most recognizable of John Williams’ Star Wars motifs. The music is also considered Luke’s Theme, due to the music first being used with Luke looking out at the sunset, and its connection with not only the Force, but Luke as well.

• The conjunctive functionLipscomb talks about a priming function where the score introduces the atmosphere for a new scene or even an opening where music announces de beginning of something new (related to the film, the sequence or the narrative). Those functions prepare the audience to anticipate the developments of the movie. They could be analysed on a general point of view; directly link with the story, or on a precise point, taking part in a specific scene.

The author also noted that the music could give the general mood of the movie. The film score helps the audience to understand if the narration would be romantic, scary or funny. And this function would be emphasized by the level of ambiguity of the visual part. The more ambiguous the scene is, the more important the music would be for the interpretation.

The opening credits are one of the clearest case to illustrate the priming function, we could also quote thriller scenes such as pursuit or hunting scenes. When the music starts at the beginning of the scene, the spectator can anticipate more precisely what will happen even if he doesn’t know when and how it will happen for sure. This contrast between the intention of the music and the unknown of the future are essential points to augment the perception of the tension.

20

Julien, on a more technical point of view, will see the musical intervention as a help to join different scenes and try to keep a continuity between the images. Filmmakers have to make choice and to choose is to forsake. A movie can obviously not last days and keep all its natural sounds. The audience is not used to raw movie, the reality is one thing and its transposition in audio-visual, a sensorial reduction, is another thing. Directors aim to go towards an «unit illusion» (Chion, 2014), that is to say that they aim to standardize the work in the most natural way possible.

«Le trompe-l’oeil est, de même que le “trompe-l’oreille”, un art noble» 3 (Chion, 2014)

The ear analyses and synthesizes faster than the eye because it can afford to follow a unique sound line, on the contrary of the eye that works as well in space as in time. The sound (or in other cases, the music) can thereby mislead the eye by playing on the temporality and the speed of the images. In that way, for the hearing persons, a sudden slamming of the door, will be understood faster by the sound than by the image itself.

I find interesting to underline the information gathered in Mr. Chion’s book about the deaf perception of that kind of sound special effects (Chion, 2014); Deaf persons trained in sign language develop a proper ability to read and structure the fast-visual phenomenon4. Thereby, we might think that they mobilize the same brain areas that the hearing people use for sound.

3 Translation: Optical illusion are, in the same way as earing illusion, a noble art.

4 M. Chion illustrates this concept with the sound «pschhhhttt» used by the

filmmaker Irving Kershner in The Empire Strikes Back. This sound was created to

represent the opening of the spatial door. Actually, I. Kershner used a succession

of two images: one image with the closed door and the other one with the open

door. The opening of the door is not really filmed, but we don’t see it because the

sound makes the union between the two scenes.

21

2.2.2. The implicative functions

If Julien categorizes his functions in two heaps (unlike Lipscomb) it is because the first refers to the different functions of music that are born from the iconic content and development of the film. On the other hand, the following implicative functions, the second heap, result above all from a sensitivity between the spectator, the composer and the others in the theatre.

Composers are looking for an emotional univocity, a kind of collective music unconscious that dictates the emotions of the audience; a melody that increases the laugh during a Warner Bros’ cartoon, a strident sound to create fear in Hitchcock’s movies or a powerful song before a DC Comics battle.

As previously written, Lipscomb tends to mix those two categories and see in the emotional implication a functional aspect. Music has the potential to evoke emotion in a scene that would in its absence, be perceived as neutral (Lipscomb, Tochinsky, 2004). As a well-known example to demonstrate this fact his Alfred Hitchock’s Psycho (1960). In the rainstorm sequence, Herrmann’s musical score creates a dissonant tension that would not exist if the film is viewed without music.

• The kinetic function

In this case music tunes up with the action of the characters. The orchestration choice will depend on the intention of the scene. Imagine a tracking shot from behind a man walking slowly on the street, his displacement would be defined by semi-breves and minims, but the tone and the harmonization will give a dramatic or on the contrary, a happy tint to its walk. Music make the audience live at the characters’ tempo and follow its evolution.

• The melodramatic function

For this function we can talk about a boost of emotions. Composers prepare narratively the audience to the right state of mind, they highlight the psychological and behavioural state

22

of the characters. The music is thus also used to underline the emotions felt by the character. For Julien, this includes musical interventions that accompany or reinforce or the stereotypical expression of human sentiments. It is interesting to observe that Julien provides a ranking of feelings expressed by the support of the music. It is for him emotional states strongly musicalisable, from the French “music” and “usable”. Musicalisable because they most often lead to musical reinforcement. That is to say that the scenes that follow this factor are those with more often musical accompaniment. The Musicalisable and emotions table of Julien will be explained in the next chapter.

• The phatic function

Last but not least, the phatic function. The concept was first defined by Roman Jakobson as one of the six great functions of language. This function is used primarily to establish, extend, or interrupt communication. For the music it is about the same principle, it allows the spectator to delve directly into the universe of the film or to go back to it during projection when something is going to happen or even when the action reaches its climax. It is often the time when we stand on our armchair in the cinema.

It is worth noting that these functions do not play especially solo, they mix up and allow then the music to move from one status to another in a short time.

With Baby Driver, Edgar Wright takes his method to a whole new level, writing the 35 songs into the script and letting each track dictates the narrative of the scene. Wright completely did it on its own method since, normally, producer prefer to work on the music afterwards, after the editing of the scene and with a composer. But Wright sent the script to the actors along with the thumb drive containing the soundtracks and decided to have the music directly on set and had actors listen to synchronized iPod. The producer is normally famous for his colourful image and his dynamic realization. With Baby Driver he added a great focus on the music. On the one hand, the movie is full of exuberant

23

personages that rob banks and fire gun and on the other hand, as a contrast, the protagonist Baby, the driver, is a really ordinary personage, quiet and reserve, wearing normal jeans and t-shirt. This contrast is compensated by the use of music, Baby has indeed some tinnitus problem and use his earphones to stay focus and let the music dictates his movements. The function of the film is situated here between a playful, phatic and melodramatic function. It let us enter in the emotion of this cold personage, it put us inside Baby’s head but also the music determines the rhythm of the scene. For instance, during the running scene of Baby, Wright chose the song Focus. This song written by the Hocus Pocus has the particularity to change tempo frequently. It was a good way to let the song dictate the rhythm of is run, fast guitars when he was sprinting, yodeling section for when he was hiding behind a tree, an accordion when he breaks into a car and so on.

24

3. The induction of emotions

3.1. Laws of emotions

«In the era of silent films, the director first specifies the emotion of a scene, and then the organist chooses musical examples labelled with that emotion to providing the background ‘mood music’ during its screening.» (Cohen 2001)

M. Chion was seeing two major ways of creating music emotions. On one hand the Empatethic Music that follows the emotion of the scene, it communicates its rhythm and its tone, in other words it gives the general mood of the images. In my point of view, this type could be linked to the illustrative functions of J-R Julien in the way that the music has a role of highlighter for the image but also for the emotion of the viewers. On the other hand, there is the Anempathetic Music, in this case the music is not in line with the scene, but it enhances a sense of tragic apathy. For instance, a happy music when a character dies tragically. This indifference intensifies the individual emotions.

Successful directors and film composers try to provide right cues to guide the emotions. Cognitive studies have been led to show how efficient could be the ruse. In 1994 Bullerjahn and Guldenting (Cohen, 2011), two film composers, led an experience on a short segment of film; they created five different soundtracks and use it on the same segment to study the reaction of the audience. Thanks to the quantitative and qualitative methods used for the analysis, they discovered strong, visible and quantifiable trends in the perception of the scene (e.g. sad, thrilling, sentimental, vivid).

Emotions are really individual human phenomena. They express what the surround means to the person and they compose his/her personal reaction to the world at a specific time of his/her

25

life. In Film as an emotion machine, Tan (Tan, 1996) outlined six laws of emotions generated by music of film.

• Control precedenceControl precedence means that the feeling takes precedence over other planned or executed thoughts. It can manifest itself by sudden interruption of behaviour. Music in movies controls emotion responses, the background music exerts control over the audience.

• Law of concernMusic guide the attention to a specific object. That’s what often happened with advertisement. Therefore, the evocation of the music helps the object to induce the audience some emotions. • Law of situational meaningEmotions arise in response to the situations. The audience will deduce the character’s emotions, concerns, and situational meaning from the behaviour exhibited and from information given about the context.

• Law of apparent reality«Emotions are elicited by events with meanings appraised as real, and their intensity corresponds to the degree to which this is the case.» (Tan, 1996)

Music contributes to the emotional understanding of the diegesis.

• Law of changeAs explained with the phatic function of the music, a soundtrack could help to understand a change in the narrative story. This change is a fertile source of emotion.

• Law of closureLaw of closure is part of the Gestalt theory; brain tends to perceive forms and figures in their complete appearance despite the absence of one or more of their parts. Music commands interest and the emotion generated by the music is outside of conscious control.

26

Love is one of the first emotion that doesn’t need words to be understood. Often mocked or seen as cliché, love scenes are supported by a specific type of orchestration on which actors lead. Violins go out and exalt the passion and amorous rapture. Here, examples overflow, let’s stay in cliché and take Romeo+Juliet (1996, Baz Luhrmann). The Love at First Sight scene takes place without dialogs, the events unfold in terms of glances and the diegetic music Kissing you sung by Der’ree herself emphasizes the emotion already transmitted by the actors.

Secondly, Julien talked about a set of strong emotions on different levels that go from anxiety to sadness to suffering. He called them the psychic temporarily disruptions. In this case, the clips are not described by themselves, it needs music to underline the emotion. Quiet often, music is also present to fight boredom of the scene even if this one is useful to the narration, without added score it could appear slow or monotone.

Then we find the music of danger, that has to be differentiated from the previous emotions to the extent that they are more suspense states. The music of danger is present in well-known American thriller such as Psycho, already quoted, or The Exorcist from William Friedkin. Julien talked about a strange paradox between music of danger and those scenes because they normally require silence. But if we turn off the sound, the actions

3.2. Understand emotions in film score

In the audio-visual, as the name indicates, there is as much valour in the audio as in the visual. It’s then unthinkable to study one part without the other one. The importance can be found in what they communicate together to the spectators; the global perception of the situation presented on the screen. Although music often arrives in the background of the stage, that means that it’s often worked on post-production and it’s sometimes described as redundant, JR Julien insist already in the 1980’s on its essential contribution in movies. He observed that the feelings expressed by the image and for whom music is created for, can be divided in two types. Firstly, the emotional state strongly ‘musicalisable’ (Julien, 1980) word invented by Julien in order to avoid repetition such as «feeling that composer easily set to music». And then, secondly, the emotional stage lowly ‘musicalisable’ because the production and the pictures doesn’t let the music support them.

It’s clear that specific emotion has to be represented in the cinema, as an evidence that has to figure in the narration. It is legitimate that in every great style, and moreover in the one accessible to the majority of the audience, emotions such as love and fear that are directly linked to the human’s existence, has to be present. Following examples are defined by Julien as strongly musicalisable.

27

chose a square ratio image to tighten the attention to the characters and their confinement. This format is nevertheless changing few times during the film to a larger ratio, also with a synchronization of the music, in order to accentuate the few moments of happiness and release. As example, the Faux Ending, Experience, perform by Ludovico Einaudi where the sounds of anger are mixed with the moment of pure happiness. Music thrill these two opposite emotions that Julien saw has lowly musicalisable. The scene ends with the return to the isolation of the frame at the rhythm of the piano.

Following the literature of Julien, Marshall and Cohen and finally Lipscomb and Kendall, the University of Southern California Viterbi School of Engineering6 investigates the effect of music on viewer’s emotion response to film. The quantitative and visual analysis was done by comparing emotion ratings for film without music, music alone, and film with music. They selected film clips7 where the ambiguousness was obvious, in order to have a greatest emotional impact on the visual events. On one side,

6 Parke, R., Chew, E., and Kyriakakis, C. 2007.

Quantitative and Visual Analysis of the Impact of

Music on Perceived Emotion of Film. ACM Comput.

Entertaint..

7 24 clips were excerpted from five movies, five clips

from Amélie, one from Little Man Tate, eleven from

Maria Full of Grace, four from Memento and three

from Three Kings. They tried to select lesser known

films in order to avoid prior emotional associations

from the audience.

lost great part of their efficiency.

Tenderness is the last emotion classified as strongly musicalisable, we can find in tenderness love that parents demonstrate to their child, the musical evocation of friendship, affection or even the renouncement to love.

«Couper le son, c’est abolir cette fonction magique de la musique, celle d’être une parole idéale, profondément ancrée dans l’inconscient collectif.»5 (Julien, 1980)

In the section lowly musicalisable, Julien holds the hatred, the fury and their opposite: happiness and bliss. They are, according to him, hard to be accompanied with music, otherwise it could create a rivalry between music and the real sound and noise considered as key role to the narration (such as door slam, reverse furniture, verbal abuse, shocks, etc.).

It’s important to note that this ranking dates from the 80’s, since then music and cinematographic genres have evolved. Happiness and hatred are themes often faced by Xavier Dolan, for instance, in Mommy (2014). Dolan succeeds in transmitting emotional value throughout his film as with his unique style of geometry in the image as with his score choices that fit perfectly with the visual. Indeed, Dolan

5 Translation: Turn the sound off is like abolishing

this magic function of the music, the one that is an

ideal speech, profoundly anchored in the collective

unconscious.

28

they select classical genre music, mainly known by the audience and from disparate emotion: content, depressed, exuberant and anxious. One piece from each category was paired with each film. For instance, Clair de Lune of Debussy, categorized as a ‘content mood’ music, was paired with Amélie and then test on the audience to understand in which way this affect the visualization and the emotion of the clip. On the other side, they created new score that could match the original emotion of the clip to see if the original score could be replaced by another score that follow the same mood. The results were displayed on Osgood’s three-dimensional space, consisting of axes representing stress, activity and dominance where the results were displayed.

The experience is a great conclusion to this chapter. In addition to demonstrate a way to qualitatively asses the degree of the emotional impact, they brought results that support the original premise on the strong effect of music on film. They have demonstrated that the audio and the visual could give a good prediction, in term of emotion, if taken separately but that the two components result stronger together than taken apart. The mood of the music influences sufficiently the emotion of the audience and finally they have noticed than the emotion was strengthen if the music was made to measure to the type of emotion rather than if it was picked from the popular culture.

In succession, an exhaustive list to illustrate Julien’s classification of the music in film with specific scene and its music composer.

29

LoveCasablanca, “As Time Goes By”, Dooley WilsonBreakfast at Tiffany’s, “Moon River”, Henry Mancini Dr. Jivago, “Lara’s Theme”, David Lean Cinema Paradiso’s final scene, Ennio MorriconePearl Harbour, Sunset scene, Hans Zimmer

Music of danger (fear)Psycho,” The rainstorm”, Bernard Herrmann Jaws, “Main theme”, John WilliamsDunkirk, “The Mole”, Hans ZimmerInterstellar, Miller’s Planet, Hans Zimmer

SorrowfulOnce Upon a Time in America “Debora’s theme”, Ennio Morricone Once Upon a Time in the West “Harmonica”, Ennio Morricone Eternal Sunshine of the Spotless Mind “Everybody’s Gotta Learn Some-times”, BeckInception “Time”, Hans ZimmerInterstellar, Cooper leaves Murph, Hans ZimmerLion King, Mufasa’s death, Hans Zimmer

TendernessE.T., “Ride in the Sky” , “I’ll be right here”, John WilliamsLa vita è bella “Abbiamo Vinto”, Nicola Piovani The Little Prince, The Rose, Hans Zimmer

Furious / Hatred8 Mile, “Lose Yourself”, Eminem The departed “I’m shipping up to Boston”, Dropkick MurphysThe Dark Knight “Joker Theme”, Hans Zimmer Inception “Mind Heist”, Zack Hemsey Happiness / Bliss Amelie, Blind man scene, Yann TiersenThe Green Mile, “I’m in heaven” scene, Fred AstaireDead Poets society, “O captain my captain” scene, Maurice Jarre and Peter Weir

Stro

ngly

mus

ical

isab

le

Low

ly m

usic

alis

able

30

2. AUDIO TRANSLATION AND ITS ACCESSIBILITY__________

The World Health Organization estimate than over 5% of the world’s population has disabling hearing loss8 and estimates that by 2050’s over 900 million people - one in every ten people- will have disabling hearing loss.

We could talk about a great challenge of accessibility in the world of cinema for Hard-of-hearing persons since the issue on the matter arrive on various level. First of all, without entering in the theme of music, the basic accessibility of the subtitles are really poor in the theatres9 and if they are by chance subtitled screenings it would be shown during unsociable times, that means during working hours or late at night, Even though United Nations Convention on the Rights of Persons with Disabilities (CRPD), firmed by all the European states, stipulates clearly that

8 Hard of hearing refers to people with hearing loss ranging from mild to severe.

https://www.who.int/en/news-room/fact-sheets/detail/deafness-and-hearing-loss. Deaf people have profound hearing loss, which implies very little or no hearing.

But it seems difficult to understand a precise number of deafness in the world.

9 Statistics from https://www.actiononhearingloss.org.uk/live-well/our-

community/our-blog/our-fight-for-cinema-subtitles/ shows that at the release of

the blockbuster The Avengers, in the opening week, only two film screenings out

of 179 were subtitled in the UK.

31

though the different legislations often underline that music and every sound should be subtitled, we rarely find those translation on the screen. When it comes to music, the rules are quite simple, it should describe either the type of music (e.g. classical music), either the type of sensation (e.g. sad music) or the exact name of the music (e.g. Requiem form Mozart). The BBC subtitle guidelines11, for instance, insist of having the source and the name of the music if there is.

eg. : “She whistles a Jolly tune” or “MUSIC: “God Save the Queen”

Those guidelines could sound equivocal since the name of a music, that has never been listen formerly doesn’t communicate any useful information to the DH audience. This lack seems irrational since, as demonstrated in the former chapter, music plays a great part in the understanding of the narration in film.

In this part I will introduce the Audio-visual Translation (AVT), with a special focus on the Audio Translation since the main goal of my thesis is studying the audio accessibility for Deaf and Hard of Hearing (DH). Some AVT mode will be described, focusing on subtitling. Then I will introduce two important concepts; the accessibility of the translation and their reception.

11 BBC Subtitle Guidelines 2018, Music and

songs retrieved from http://bbc.github.io/subtitle-

guidelines/ consulted on December 2018.

“States Parties recognize the right of persons with disabilities to take part on an equal basis with others in cultural life, and shall take all appropriate measures to ensure that persons with disabilities: a) Enjoy access to cultural materials in accessible formats; b) Enjoy access to television programs, films, theatre and other cultural activities, in accessible formats.”10

Subtitles should be then an evidence in every theatre.

Coming back to the focus of the world of TAV, confronting to the distribution, accessible translation for deaf doesn’t seems to be perfect neither. The distribution of AV media as growth and the accessibility doesn’t necessarily follow the trend, even though we all agree that the importance should go to offer equivalent information to different typo of audience. It’s to say that the work of translators is not so easy because it requires multi-tasks skills and they often focus on the linguistic importance of the translation instead of the effects produce on the audiences. Indeed, nowadays, translator should be well informed about the viewers’ needs, the reading habits and the reception capacity.

When sound has to enter on stage, other problems are pulled out. Even

10 Retrieved from the CRPD of the United Nations,

Article 30 – Participation in cultural life, recreation,

leisure and sport.

32

1. The evolution of AVT (Audio-Visual Translation)

Audio-visual translation (AVT) is the term used to refer to the transfer from one language to another of the verbal components contained in audio-visual works and products. Feature films, television programs, theatrical plays, musicals, opera, web pages, and video games are just some examples of the vast array of audio-visual products available and that require translation. As the word suggests, audio-visuals are made to be both heard (audio) and seen (visual) simultaneously but they are primarily meant to be seen. (Chiaro, 2008)

The apparition of the AVT is closely linked with the birth and the evolution of the cinema. In the transition from the silent to the speaking film we find the origin of the audio-visual translation such as subtitling, dubbing, etc.

During the Silent Film Era (1890’s-1930’s), the image and the writhing text were the only source of information for the audience. It was a succession of visual and graphical board called title cards to explain the story of the movie. The mimic of the actors and the music used were often really explicit to help the lack of sound and dialogs. Those titles were the first apparition of AVT. Progressively the silent film gets longer, and the onscreen inter-titles were used more often to narrate story points, dialogues and to comment the action of the actors. They also became more graphical, with illustrations and more decorative to state the details of the action. Title writer became a real profession and it was even seen as an independent job, separate from the scenario writer.

Silent film was easy to export since only the titles had to be translate. In 1927, the Jazz Singer was released in the theatres, it was the beginning of the talking movies. But the enthusiasm of the Talkie Era was quickly followed by the reality of the translation. The cinema, seen as a universal communication media, thanks to the power of the images, was now freeze by the linguistic barrier. The problem was first resolved thanks to the multiple-language version film in the 30’s, the idea was to produce the same film

33

1.1. AVT modes and apparition of new AV media

For most people, screen translation is limited to dubbing and subtitling. Nevertheless, various types of screen translation exist today. They can be categorized into two main groups: dominant and challenging. (Gambier, 2003) Out of interest for my paper, I will try to focus on the AVT modes that are linked with deaf and hard-of-hearing persons.

1.1.1. The Dominant Types

In the dominant types we can find the inter-lingual subtitling, dubbing, consecutive or simultaneous interpreting,voice-over, free commentary, simultaneous translation and multilingual production.

Inter-lingual Subtitling are the written lines of the dialogues, they can be translated from one language (the spoken one) to another (the written one) or in some case to bilingual subtitling like in Belgium were they often put both official language of the country (Dutch and French). Inter-lingual subtitling can be offered to any audience, including the deaf and hard of hearing (DH). But they are not as complete as the SDH (Subtitle for Deaf and Hard-of-hearing) that describes all the sound of the media. This type is the only dominant type, as conceived by Y. Gambier, that concerns the DH. It is also the most common one,

several times in different languages for international market. For instance, Paramount on Parade, a film released in 1930 by the studio Paramount, was translate in twelve different languages. Unfortunately, this technique had its limits: the studios were running nights and days and they were invaded by foreigner actors. Moreover, the processes were long, tiring and expensive, that’s why, gradually, they opted for subtitling and dubbing. One of the first technique was called ‘International Sound Version’, the methods had to replace every dialogue with music or foreign inter-titles. Warner Bros was the first one who begin issuing this technique, it was cheaper and easier to spread but obviously not the best technic to respect the original version. After all those trials, they finally implement the subtitles as we know them nowadays.

During the 1970s and 1980s SDH (Subtitles for Deaf and Hard-of-hearing) was introduced in the US and Europe, but accessibility services were considered costly and adequate only to the needs of a reduced population. Since then, audio-visual accessibility has become an afterthought in the film industry even though the 2000’s brought new researches and development on the subjects, thanks to new technologies and many AVT scholars such as Gambier, Chiaro, Fryer and Freeman or Chaume.

34

effects and the music are translated. The colour’s codification for the subtitles vary from one client to another or from one channel to another, yet there are standard norms14. At the same times, the subtitles have to melt into the film without creating any discomfort for the viewing of the audience.

Special Subtitle

Unlike the classics subtitles, special subtitles do not follow a system of norms, they transgress the latter and question the traditional practice. Very often they are designed by animation software and offer an abundance of plastic possibilities, that is to say of shapes and colours, but also of mode of appearance and disappearance. Unlike the classical subtitles, they are much more integrated in the image and the narration. Especially since they are often followed by the filmmakers of the film. In her article Formes, sens et pratiques du sous-titrage spécial, Sabrina Baldo de Brébisson creates an empirical inventory of 28 different forms of special subtitles. For example, the term pop-on is used to describe subtitles placed at a strategic location on the screen without distracting viewers ‘ attention from the images of the film. Another term found in

14 The subtitles written in white correspond to

the person who speaks on the screen, the yellow

is used for people who speak out of the field, red

corresponds to indications of noises, magenta

corresponds to music, cyan to internal voice or

voice-over comments and green for indications or

transcripts in foreign language.

that we can find almost on all the media such as TV, films, cinema, online series and YouTube videos. It is to say that they are often criticized by translation specialists and semiotics that see them as a compression or a «necessary evil». (Baldo De Brébisson, 2016)

1.1.2. The Challenging Types

Challenging because the rules are not always completely defined, and the reception of those translations are not enough controlled. In this section, we find the scenario or script translation12, intralingual subtitle for DH and I will also add the Special Subtitle, the Surtitling13 for the stage and the live translation.

Subtitling for Deaf and Hard-of-hearing

Subtitling for DH also called Closed Caption, are done to strengthen the comprehension of whom who are Deaf or Hard-of-hearing. Those captions faithfully transpose what it’s heard on the video with variation of colours, special positioning and written details about the sound event. That means that the dialogues but also the sound-

12 Scenario/ script translation is needed in order to

get subsidies, grants and other financial support for

co-production. These translations are usually not

edited. (Gambier, 2003)

13 Surtitles for the stage: From the French language

“sur” that means “over” and the English word “title”.

They are subtitles displayed on a screen, commonly

used in opera or musical performances.

35

the inventory is mobile, used for subtitles that mimic the action of the image, these bring a hint of humour or fantasy.

One of the first critics of the so-called classic subtitles is that one should not read a movie but watch it. Thanks to this type of subtitles, this criticism is no longer necessary; it saves reading time, it has a better readability, a better viewing and thus a better semiotics of the film. Moreover, the originality of the special subtitles ensures a function that we could call phatic, because they attract the attention of the spectator and they ensure that transmission of the communication. It also provides an emphatic function, for instance, they highlight or accentuate the soundtrack and/or the visual part. Another function is to play on the transmission of emotions, the searched effect is naturally in accordance with the genre of the film; For example, subtitles floating, or gushing would be used for a 3D movie. The best example that probably exists, described by Baldo de Brébisson (2016), is from the film NightWatch, made in 2004 by director Timur Berkmambetov. In the international version of the film, the Russian director wanted special subtitles written for American English to be created as elements of mise-en-scene.

__________

Figure 4,5,6 Retrieved from Night watch, Timur Berkmambetov (2004)

36

Live translating

The technique consists on listening the original soundtrack of a live programme and reformulates it to a speech recognition soft-ware, which turns the recognised utterances into subtitles displayed on the screen. It can also refer to translate a live program or even a concert by using the sign language. We often see that on television, for instance for the news bulletin but more and more we can find that kind of translation in music festival or concerts. At the Lasemo festival, accessibility for all became one of the most important point of the festival. For some concert the translator Cindy Barate, come on stage and become part of the concert, like a new instrument would do. She learned the lyrics and translate them during the show in a poetic way to help DH spectators to enjoy it more.

2. The spread of AVT

The spread of audio-visual translation (AVT) happened in the late 20th century, with the help of the Internet, the expansion of the audio-visual¬ distribution and the advent of technology such as DVD or better; mobile phones, tablets and other electronic devices. In this part the interest is focused on studying where AVT is going and understand if its growth includes the music translation and its accessibility. For that, before entering in the investigation of music translation, it seems important to me to contextualise the expansion of AVT nowadays and the rise of its collective

Berkmambetov is fortunately not the only one to be concerned about the audio-visual translation of his works, Stanley Kubrick for example was used to go to the post-production meetings in order to give his opinion on the content and form of the subtitles. Likewise, Roman Polanski, had imposed for his film Tess, that the subtitles appear on two short lines instead of one long, to avoid the spectator to do some ping-pong with their eyes.

Of course, the number of specialists involved depends on the budget allocated to the audio-visual translation section. The practice of special subtitling represents a long and relatively expensive job compared to the cost of conventional subtitling as it is done plan by plan. They are also often practiced in movies that have received an important budget such as Night Watch ($ 4 million15), Slumdog Millionaire ($ 14 million16) or even Avatar ($ 425 million17).

15 The numbers, http://www.the-numbers.

com/movie/Nochnoy-dozor#tab=summary ,

consulted the 15th of August 2019.

16 Sulmdog Millionaire , https://

www.the-numbers.com/movie/Slumdog-

Millionaire#tab=summary , consulted the 15th of

August 2019.

17 Avatar, https://www.the-numbers.com/

movie/Avatar#tab=summary , consulted the 15th of

August 2019.

37

interest thanks to one phenomenon in particular the fansubbing.

2.1. Collaborative subtitling

The phenomenon of the fansubbing refers to a practice adopted by the fanbase, the fans of a series or even films, that translate without any money contribution. It’ was born in the 80’s with the fans of Japanese Anime. The goal was to make it accessible to the rest of the world. In Europe it became really famous in the 2000’s with Lost18 and the practice of the streaming. Fansubs are nowadays one of the most important manifestation of fan translation, having turned into a mass social phenomenon on Internet, as proved by the vast virtual community surrounding them such as websites, chat rooms, and forums.

The subbers are often amateur translators, some of them are students, other come from the professional translation world, or some are just passionate by the idea of a sharing world. There are not specific rules, but we can

18 Lost is an American television series

created in 2004 by Jeffrey Lieber, J.J. Abrams and

Damon Lindelof. It has been consistently ranked by

critics as one of the greatest television series of all

time. The first season had reached almost 16 million

viewers per episode on American Broadcasting

Company. ABC Television Network. SEASON

PROGRAM RANKINGS, 21 June 2005, web.archive.

org/web/20141010224625/http://abcmedianet.com/

web/dnr/dispDNR.aspx?id=062105_06.

talk about good practice: each subber translate a part of the episode then share it with the others. Normally there will be five or six persons per episode and the subber starts its translation thanks to the original translation. Then, an important part of the job is the synchronization. Even if this practice is not illegal, because subtitles are distributed apart from the video, there are still some copyright problems since the original dialogs are property of the authors. The truth is that it doesn’t seem to be some famous confrontations registered between translators and copyright holders. On the one hand, since the distribution is done via Internet, a medium in which borders, and nationalities are difficult to be delineated, the copyright legislation became complex to understand and to be applied. Moreover, this practice tends to be out of the commercial circle. Secondly, the revelation of the subtitles allows different media to be accessible to a wider public, otherwise the media could be directly thrown into a never-never land by the producer. In this way it gives better chance to retain viewers before the producers decide so.

38

I’ve found interesting to introduce the world of subbers since it’s a world of amateur translators, that have done some changes to the way subtitles were done and seen. By creating a collective way of working and then, by taking more freedom in the way of translation. Their comportment could inspire real translators to push their limits off the classic subtitles and to have a better understanding of the reception on the audience.

What we need are translators who are unruly, not transparently naked. Not sober but intoxicated—and positively abusive. «We want translators with attitude» (Baldo de Brébisson, 2016)

2.2. Issue mapping on the web

I was interested to see what Google can tell me about Music Translation for AV. I’ve used some digital methods to help me understand what the matter on Internet was.

On a first hand, before going deep into my researches, being in Italy I did a first research with Google.it. My main goal was to understand the general words that keep coming up when you used the query ‘traduzione audiovisuale’, the Italian words for Audio-visual Translation. I’ve used a Google Incognito Window to collect the first pertinent links . For the second step I’ve used a digital tool called Tag Cloud Generator that takes the raw text of the links and extract the words that are repeated many times. In the end, I collected a total of 114 different words but most of them could be mated between them such as translation with translator or film with cinema. That’s why I decided to classify them into ten categories; profession, duration, speaking, researches, law, translation, education field, communication media, area and accessibility. The conclusion about this first research gave me a good general map of the issue on internet; the word music or even its synonym almost never appeared in the 20 links, instead we could find a lot of words related to the speech. In general, the main topics were about educational field such as AVT masters and AVT education programs. That’s what gave me a start for the second experience.

39

For the second experience I wanted to focus my researches on the AVT Master since they are forming the next generation of translators, moreover I wanted to develop a new kind of research. This time I decided to see what was going on in www.google.fr, the main reason of this change was because I realized how many translation researches were followed by French searchers. This time I took the first ten links that responds to the query ‘Traduction Audiovisuelle Master’. Then, I treated them in a new digital tool called Google Scraper. To make it work you have to give some URLs and a list of keywords, Google will be asked if each keyword occurs in each URL and count it. I’ve created different glossary based on my previous researches, here again my goal is to see how often music came up as part of AVT. The glossary music was kind of disappointing, out of 7 words from the glossary music, only the word sound appeared and really few times. Here again the glossary speech won against the other one. Finally, to conclude the work I’ve also checked the glossary accessibility and it was a good surprise, even if it doesn’t include the accessibility of the music, it was good to see that it makes part of the main topics in this series of URL’s. This small research was for me a good start to dive in the topic of design for accessibility.

For details about this research, see Appendix 1 and 2.

40

3. THE CHALLENGE OF MUSIC ACCESSIBILITY__________

Accessibility has for a number of years been a legal and technical issue in various countries, with a view to ensuring that disabled persons can enjoy physical access to transport. Facilities and cultural venues. (Gambier, 2009) Services must be available and usable by all users, independently of their physical and mental capacity or neither their level of experiences.

Elise Roy, a disability rights lawyer and design thinker, talks about benefits that people without disabilities could take from an inclusive design. For her, design should be thought first for disability before than for the norm, in that way we could stumble upon solutions that are even better19. She believed unique experiences that people with disabilities have is what’s going to help us make and design a better world for everyone- both for people with and without disabilities.(Roy, 2015)

As seen, the distribution of AV media has growth and the accessibility must follows the trend as it could since it’s important to cater for the needs of user and to offer the equivalent

19 For instance, the famous OXO potato peeler was first design for people

with arthritis. Even better, the text messaging was originally design for people

who are deaf!

41

information to different audience. Even though, the work of the translators and designer is not so easy because it requires multi-tasks skills and they often focus on the linguistic importance of the translation instead of the effects produce on the audiences. Indeed, the viewers’ needs, the reading habits and the reception capacity should be taken into account when designing accessible technologies.

In conclusion of this first largely descriptive step, I’ve tried to answer those following questions: How important is the music in audio-visual media? Who are the actors present in the evolution of the translation for Death and Hard of Hearing? And finally, in which measure do this evolution impact the accessibility?

First of all, my researches focused on the question of new techniques applied to captioning and the expectations they bring about. The habits evolving, it seems time to use the digital wisely in order to improve the quality of accessibility services considering the expectations of DH spectators.

Secondly, more and more widespread, the special or alternative subtitles should not be treated as a competitor of the classical subtitle but as a continuity; A stimulating evolution, imprinted with freedom, to binding norms. It is not designed to be simply read: It is to look at, appreciate and sometimes even admire; A bit of emotion.

As we have seen with Gambier, one of the major issues with the subtitle’s translation is that we don’t effectively know about the reception on the audience and their satisfaction. I would ask myself, what about finding or at least testing some alternatives, in my case, about music subtitling?

« No matter whether we look at technical or literary translation, film subtitling or conference interpreting, most translators see themselves as common soldiers in the battlefield, rather than armchair strategists calmly considering their next move. » (Schjoldager, 2008)

42

They may be not ethically correct, but that’s where fansubbers have created a change. Being not completely legal, they took the freedom to personalize their subtitles and add some special effects in the rendering. For instance, they didn’t wait for special subtitles to be famous, they were already using kinetic technique in the subtitles of the Japanese anime. But that doesn’t mean that professional could not try to push the boundaries and find alternatives.

Finally, the music in an AV work must be considered as an important part of the comprehension of the film, its semiotic aspect has a great impact on the viewing. It is therefore essential to treat it in the same way as the rest of the information of the work and I would say even more, to look for a more adapted translation that could fit in the image. It is a difficult challenge, but it is an important step for the accessibility of the AV media. Music being invisible, a first step would probably be to take its impact and emotions into account. The whole focus is definitely on the way the receivers perceive the message and much less on the way that message resembles the original.

43

PART 2

COMMUNICATETHROUGH VISUAL STIMULIVisual exploration to hear music

Cover retrieved in Fischinger (2013)

45

4. INTERACTION BETWEEN SENSES __________

1. One sense triggers another

Audiovisual translation is not an easy world, as its name implies, the audiovisual is filled with codes and signs touching different senses. We cannot talk about linear communication for example between what is communicated in audio and the organ of the hearing or, on the contrary, between the visual and the view, it is a multisemiotic and multisensory whole that is communicated to the spectators. Mostly, people tend to focus on verbal in any translation, while the senses, genres in movies and television programs emerge from the complexity woven between language, images, music, sounds, colors, rhythm, etc. Interdisciplinarity, under these conditions, is inevitable: questions about language transfer cannot ignore the reflections film studies. As is often seen in design studies, Gestalt theory is applicable here as well. The whole, or better the form (gestalt in German) is more important than the parties taken separately. The processes of perception and mental representation spontaneously treat phenomena as global forms rather than the addition or juxtaposition of the simple elements. This theory is, of course, a simple summary of the complexity of the links that are developed in the audiovisual world, but it exalts in few words the complications associated with audiovisual translations.

46

One sense triggers another one but are we the owner of our own perception? The perception multi-sensorial is hard to dissect. It’s immediate, first receive by our senses and then judge by our brain. Could we understand the path that a sensorial perception is taking? The attempt to answer this question comes with another concept; synesthesia. This perceptual phenomenon of automatic associations is not so well-known from the general audience, indeed, even synesthetes themselves are often unconscious of their uniqueness and discover this particularity by coming across an article.

Even though I’m not synesthete myself, thanks to a laboratory in my first year of master, I discovered this world where we could taste by touching and see by hearing. I decided to go deeper with the approach of synesthesia and multisensoriality to understand alternatives to the AV translation. I believed this phenomenon is a great way of rethinking design and accessibility. It’s a support that could help us understand the multiple facets of a project and how to approach it on an unusual way. Obviously, lift the veil on this incredible intern imagery is a delicate exercise. If words could be often abstract and perception subjective, create an experimental and multi-sensorial zone where people get submerge is a good way to disrupt and to question the primary perception of the spectators. Numerous contemporary artists primed those experimentation based on a synesthetic mechanism or on its metaphors: the Russian painter Vassili Kandinsky, abstraction beast, is a great example to quote. He dedicated himself to translating what he was hearing. His forms and colors were his vocabulary and the composition were his grammar. Great understanding can be found in his work, more precisely about how to connect different senses and to understand correspondences in order to represent the invisible.

To claim a conceptualization and the creation of a complete experience created by the designer, it’s essential to master the target: the perception of the subject. From a scientific point of view to a philosophic one, the introduction of the neurologic phenomenon synesthesia will bring a new definition of the perception.

47

2. Sinestesia: cross-sensory implication

II est des parfums frais comme des chairs d’enfants,Doux comme les hautbois, verts comme les prairies,— Et d’autres, corrompus, riches et triomphants,

Extract from the poem Correspondances de Charles Baudelaire. In his poem, Baudelaire explain the theory of correspondences that is an essential aspect for his world’s perception. For him, a sense triggers another one, a kind of horizontal correspondence that could evoke a feeling or an idea.

This theory of correspondence between senses or images have been studied for long time and could be considered under different names even though they often represent the same concept of intersection between senses, such as mutltisensoriality, multimodal perception, cross-modal perception, intersensoriality, synesthesia. All the prefix multi-, pluri- or pluri- should outline the sensorial quality and the prefix inter- and sin- underline the relationship between the sensation. (Riccò, 2008, pp.61) The title of one of those concepts given to the artefact is determined as little useful in the sense that we already perceive an artefact as multisensorial since we detect it in its all. The interesting aspect would then be the relationship that we maintain at a given moment with this artefact. Nevertheless, the synaesthesic project is characterized as a coherent relationship between sensations. (Riccò, 2008)

The condition we today name synesthesia, an involuntary phenomenon, in which a small percentage of the population reports extraordinary additional sensations triggered by reading, listening to music, eating and so on, had different names throughout the history. There was a long process of the development of its name, from ‘obscure feeling’ (1772) to ‘synesthesia’ (1892). – VI International Congress Synaesthesia, Science & Art

48

Obscure feeling comes from the German poet and philosopher Johann Gottfried Herder, in 1772, he first talked about those phenomena of association with the sound and the colour. Afterwards lots of artists or philosopher used synonym to talk about the same strange feeling that they have with colour or sounds. But in 1892, Jules Millet defines this phenomenon of associated sensations with the term synesthesia as we intend it today. (Jewanski et al., 2018)

The word comes directly from the Greek syn-, union, and asthesis for sensation. In his website, Sean Day, ASA’s president , explains that the most common forms of cognitive synesthesia, out of 80 different types of synesthesia, involve such things as colored written letter characters (graphemes), numbers, time units, and musical notes or keys.

2.1. Sensoriality and accessibility

Synesthesia is not only about recurrent correspondence between specific sound and associated color, it’s a way to transfer or sensorially translate what surround us. It’s in a way, something that we all have is us. Some recent studies shows that synesthesia can be a much richer and a more extendible phenomenon and that synesthesia has to do with the process of extracting meaning of the stimulus by the process of learning, mostly when dealing with abstract concepts.

«These hypotheses explain for example, why the most common forms of reported inducers in synesthesia are graphemes (e.g., letters, numbers) and time units (e.g., weekdays, months in a year); graphemes and time-units are about the first abstract concepts that children are faced with through the educational systems.» (Kirschner et al., 2017)

It has been proposed the very name of the phenomenon—synesthesia—is fundamentally misleading. Instead, a more correct name for the phenomenon would be ideasthesia, from the Ancient Greek word idea standing for concept. Ideasthesia then means sensing concepts. If synesthesia only occurs in some

49

the senses, the distinction between simple perception and the interpreted sensation, led by the cognitive and affective mind, is questioned. Admitting that the perception is a tool to represent the world not as it seems to be, but as our senses understand it. He nourishes his work with his knowledge and the cases of neurological patients. His book delivers what looks like a collection of biographical narratives to lead to the paradigm of the vision of our mind.

2.2. Visual representation of the sound

Apart from the scientific point of view, synesthesia has a heavy artistic background. There is a true synesthetic heritage born from influences between literature, painting and music. A synergy between perception and know-how which is motivated by the ambition of re-transcription of these particular connections. Whether or not the creator is synesthetic, his ability to distort the public’s primary perception, to go beyond the categorizations of the senses, testifies to his power of openness to a new world. Manipulator of perception, the artist has mastered many tools that have the power to touch the sensitivity of his contemporaries, the ability to handle the senses to better exceed them and thus invite them to a new journey. Inspired by the great masters of perception such as Wassily Kandinsky for his abstract musical language or as the mystic Alexander Scriabin for the virtuosity of his colourful and ecstatic musical style, designer can

people, although it may be more common than previously thought, ideasthesia itself could be a fundamental part of our everyday life. Our everyday experience of colors sounds and other stimuli do not live on separate sensory islands but are organized in network of associations similar to our language network.

This theory is a very important point for my thesis and could give a clue to accessibility for DH. Indeed, as seen in the part 1, the importance has to be done on the way the receivers perceive the message and not on the message taking apart. Yves Delaporte, deaf’s ethnologue, explain that being deaf doesn’t mean to be sentenced to not ear but to live in a world that doesn’t understand deafs. Emmanuelle Laborit, deaf since birth, says that her eyes are her ears, that she doesn’t miss nothing, society make her disabled and dependant.

«La musique est un arc-en-ciel de couleurs vibrantes. […] La musique est un langage au-delà des mots, universel. C’est l’art le plus beau qui soit, il réussit à faire vibrer physiquement le corps humain. »

(Laborit, 2003)

Indeed, music can be “heard” in numerous ways and medical and sociological reports (Sacks, 1990) prove that the world of the deaf is all but silent. Sacks is at the same time doctor, patient, professor and author. In his book Des yeux pour entendre, transports his readership to an internal world where the interdependence of

50

admit synaesthesia as a new communication process for the senses. His qualities of master of the image also give him many possibilities of translation of the real, his palette of instruments to operate the image or the text give him great possibility of creations as infinite as the synesthetic associations.

In this art world, we can find plenty of representation of the sound with color, image or shapes. The history of synesthesia artists is consistent, we could start with the Renaissance Era, with the creation of melodious architecture, when artists start to believe that the musical proportion pleasant for the ears could do the work for the eyes too. Continue with Newton and his correspondence of color with sounds and then Castel’s keyboard Le clavecin oculaire that would turn on colorful light bulb instead of piano hammer. And finally goes to see how Kandiskij defines his music paintings , Today we could find plenty of artists that took the rains and are worth being quoted. For instance, Melissa McCracken paints music. As a synesthete, the sounds that she hears every day are then translated into vibrant beautiful colors that follows the tempo of the melodies. Her oil and acrylic paintings express her spectacular sights of an everyday life.

«Basically, my brain is cross-wired, I experience the ‘wrong’ sensation to certain stimuli. Each letter and number are colored and the days of the year circle around my body as if they had a set point in space. But the most wonderful ‘brain malfunction’ of all is seeing the music I hear. It flows in a mixture of hues, textures, and movements, shifting as if it were a vital and intentional element of each song.»

__________

Figure 7. Radiohead – Lucky by Melissa McCracken

51

In Progetto Grafico, M. Haverkamp, present at the congress, explained the correlation between hearing and visual attribute, his approach is related to the industrial design, for him synesthetic phenomena offer the useful materials to understand the mechanism of the multi-sensorial perception.

On the same level of idea of the ideasthesia or synesthesia accessible to everybody, he talked about intermodal analogy. Diverse sensitive parts could react in a specific context to describe an artefact. The type of analogies could work on different levels,

During the VI International Synesthesia, Science and Art Congress in Alcala La Real in May 2018, I had the chance to meet synesthete artists, searcher and designer such as Timothy B. Layden who describes his artistic work as the shape of sound, a series of sound and visual art that explore how the invisible world of sound can manifest itself visually through an overlapping of sense.

« I go to my studio and listen to music like Charles Mingus which creates shapes like exploding crystals, dark satin dripping like hot chocolate and swirling cream in a hot cup of coffee.»

__________

Figure 8. Bolero, by Timothy B. Layden

52

here are some of them useful to this paper.

• The pitch of a sound: An frequent analogy is done with the frequency of the sound and the height of the space.

• Musical graphic: analogy on the idea of a translation in form and color, in a spontaneous way as Melissa McCracken does or eventually in a studied way as a revolution to the music sheets.

Bolero Collective Art Performance

Bolero is a good way to illustrate those analogies. It’s a collective performance run during the Synesthesic Conference by Ninghui Xiong. The idea was to gather 18 volunteers in order to express visually what the ears tell us. Bolero is a genre of slow tempo Latin music that starts pianissimo and rise continuously to fortissimo. Each participant had to listen to his part and during the workshop all the minds were put together in order to create a continuous flow of visual and unique representation of the sound with colour, texture and forms.

__________

Figure 9,10,11,12. Retrieved in the Bolero Collective Art Performance by Ninghui Xiong.

53

5. EYES TO HEAR__________

1. Typography and visual sensation

1.1. The typographic experiment

In a world made of digital tools and communication with new technologies and way of reaching people, typography remains rudiment for good graphics. It has been and is still a great area of experimentation that have over decades influenced the development of new visual and theoretical vocabulary for the discipline. In the 19th century, Stéphane Mallarmé made some experiment with “formal” typography including Un coup de dés jamais n’abolira le hazard (1897), his work is considered as one of the first typographic poetry, the first state of the visual poetry that came into being with the poems of Guillaume Apollinaire from the 1920’s. But the radical innovation arrived within the territory of the avant-garde, operating outside dominant traditions. Experiment were done to deconstruct and create non-linear approaches to the contemporary narrative form in order to develop new ways of seeing, or more precisely of reading.

In The typographic Experiment from Teal Triggs (2003), the contemporary typographic experiment in graphic is explained with «the advances in computer technology that offer new aesthetic possibilities and great democratization of design and print production which led to a plethora of personalized typefaces and designers parading their new roles as graphic authors.» Experimental type design deals with two disciplines: the design, production of typefaces and the investigation of the type use in layouts. Triggs

54

explained that those experiments are written in the historical movements -isms of the twentieth century: Futurism, Constructivism, Dadaism and Modernism. Movements that are accompanied by scientific and technological discoveries, where modern industry were radically transformed, and typography became the visible artefacts of the new social, cultural and political life.

In 1909, Marinetti published in the French journal Le Figaro, the Manifeste du Futurisme to define new concepts of art and design in order to express reaction against the status quo. For him, futurism is a movement anti-cultural and anti-philosophical, a movement with idea and intuition that come from the instinct, based in the human sensibility and the help of the emergent technologies. In 1912, Marinetti published Zang Tumb Tumb, a sound poem that destroy completely the original syntax, where words are in freedom. The work had an enormous influence on the emerging culture of European avant-garde print.

«I call for a typographic revolution directed against the idiotic and nauseating concepts of the outdated and conventional book, with its handmade paper and seventeenth century ornamentation of garlands and goddesses, huge initials and mythological vegetation, its missal ribbons and epigraphs and roman numerals. The book must be the Futurist expression of our futurist ideasm... even more: My revolution is directed against what is known as the typographic harmony

__________

Figure 13. Zang Tumb Tumb, Filippo Tommaso Marinetti (1914)

of the page, which is contrary to the flux and movement of style.» (Marinetti, 1913)

Following the idea of revolution and anti-tradition, DaDa movement tried to search for new experience for audience members. Their main goal was to empty nationalism language and the sense given by the journalism that in their eyes has lost all the content. The subversive DaDa’s poem take shape with the typography and its disposition, they express their need to a layout’s resurgence without fixing any rules. Page layout for the artist Tzara takes entirely part in the poem, it doesn’t use typography and graphic composition

55

and based on 25 regularly used font families. The survey randomly assigned anonymous font and asked the viewer a series of descriptive questions such as ‘What are the 5 main values you think the typeface above reflects?’ The results of the font’s personalities were then categorized by its attributes personality, values and style. (See figure 14)

Shapes and metaphor

Glyphs are composed of attributes such as serifs, down-strokes and handles that could create a connotation with the physical world. The interpretation of the type shapes could create a connection through facial human or animal expression, through nature, gestures, object and so on. Type can be seen as

as a superficial ornamentation but as guide to give sense to the text.

1.2. Personality of the typography

Fonts are important visual communication, regardless of the words they mention, they convey a certain feeling of mood which can either increase or break the effectiveness of the message you are passing. We can talk about personality of the font because, like human personalities, they all have different history, shapes and style.

Sarah Hyndman has created an online Font Census in order to understand if we could determine the personalities of the font from a designer and non-designer point of view. The survey was done online

__________Figure 14. Type Testing online Font Census survey results identification of personality attributes, Hyndman (2016)

56

mirror to our emotion we show when confronted to the real world. Our happy face will be more open and rounder than the angry face that we will see as sharper, like the teeth of a dog when he barks at you. The inclination of the typography could also manipulate de perception; the right inclination of the text is inspired by the italic invented in the XV’s century in order to reduce the size of the text and the cost of printing. It’s indeed inspiring the fast writing, like a runner ready for a sprint race. It’s not surprising that lot of sports brand use it. On the opposite, the contrary inclination is almost never used by those brands since it gives the idea of slowing down.

SPRINT !STOPPP

The direction of the text is by default in Occident from the left to the right, thus changing it can give other connotation to the text than the wording taken separately.

Kerning corresponds to the space between each type. Even though it’s an invisible component to the reader, it’s fundamental to typography. Visual effect can be created to modify the signifié of the word: the space could reinforce the precious aspects or the delicacy and the strengthening decomposes the meaning of the word.

LUXE P U N K L U X E PUNK

57

In term of boldness, typography has something to say. Type family package almost always includes a great range of bold and light type (that goes from condensed to light, regular, bold or even sometimes extrabold). Those kinds of games are often play in the food industry, bold type illustrate the rich content and light type is more adapted for fitness product.

M I A M M I A M L I G H T L I G H T Every single one of us is already an expert in typography as a type consumer. All of these codes of references only work because you and I have been learning to recognize them all of our lives.

1.3. Sense of voice and musicality

Tone of voice

Fonts may speak a little louder than we think they do. In the 2013, the filmmaker and author Errol Morris and the New York Times20 presented an essay that was supposedly written by the filmmaker and ask the reader’s perception about it. The text was actually from David Deutsch and it was randomly present in a different typeface: Baskerville, Computer Modern, Georgia, Helvetica, Comic Sans or Trebuchet. The questions provided data about how readers perceive the essay and determine whether typeface choice influenced our perception of the truth of the extract. The conclusion shows that text that use type such as Comic Sans or Helvetica were the one less trustable. Indeed, Comic Sans is often seen as funny, humorous and comic, as the name actually indicate it. If types were human-voice, then Comic Sans would be a children voice. It means that using it to

20 Retrieved in the New York Times’ Opinionator, the opinion pages in

https://opinionator.blogs.nytimes.com/2012/08/08/hear-all-ye-people-hearken-o-

earth/

58

qualities of your voice or even the phonemes visually. A bold font particularly ink covered will looks louder than an italic word with a higher kerning. The arrangement of the letters on the page could be transform in a sheet music. Type designer Simon Stratford like to inspire his design by music22, here are fonts from Simon’s portfolio and the musical band behind them:

Inspired by Ziggy Stardust, David Bowie

Inspired by Balham to Brooklyn, Turin Brakes.

22 Retrieved in Simon Stratford website,

https://simonstratford.com/

communicate an important information could create discord and being seen as a font breach: indeed, in 2012, the CERN scientists announce the discovery of the Higgs boson21, an essential scientific find, with the Comic Sans font. On Twitter, the announcement became immediately more viral for its form that for its content.

Musical type

Type could reproduce the tone-of-voice and the intonation of the voice through the typeface as seen before or thanks to its style of the way designer dispose it on the frame. Sarah Hyndman imagined a ‘Type Karaoke’ to test the musicality of the word Hello written and graphically disposed in different ways. To play the game you just have to say out loud what you see.Type could transform the tones, the

21 Retrieved in The Guardian, Higgs boson and

Comic Sans: the perfect fusion.

__________

Figure 15. Hello, Type Karaoke, Hyndman (2016)

59

included great practice of the comic book page, such as Spider Man’ s signature onomatopoeia to achieve a stylized, kinetic visual work. The animation actually also gave a great tribute to Lichtenstein and Warhol works, using similar pop art visual for the characters but also in the intonation of the whole movie. Lichtenstein was another great artist that used typography to visually report sound in a comic book style. He was known as one of the most important pioneers of contemporary Typographic art.

__________

Figure 16. Oasis Definitely Maybe, Simon Stratford (2017)

In his collection, Stratford also proposed typo-illustration. For instance he represents visually every tracks from Definitely Maybe, the album of the English rock band Oasis, released way back in 1994.

Visual Onomatopoeia

The sound imitation by word and visual grew with the comic strips to illustrate the great action of the characters. This culture of onomatopoeia was even reproduced in some movie such as recently in Spider-Man: New Generation that received last February (2019) an Academy Award for Best Animated Feature. Texture, colour and drawing were underline positively by the critic. The producer seemed to matter about the origin of Spider Man and

__________

Figure 17,18,19. Retrieved in Spider-Man: New Generation (2019)

60

1.4. Kinetic typography: (e)motion

Typographic message could be seen on different angles; semantic denotative representation, colour, texture and shape. These dimensions are stimuli to subjects that activate mental imagery in both semantic and episodic memory systems. Those representations trigger complex sequence of reactions known as emotions (Stone et al., 2004). The intensity of the associative bonds could be amplified by the addition of motion, scale change, metamorphosis and context to typography. Diverse studies on Kinetic typography or animated text23 support that it recreates a broader range of pre-linguistic meaning and emotion than is possible with the static text alone.

Kinetic type grew with the emergence of the cinema and was first employed in the opening credits of films to set the spectators by establishing the mood of the movie. Saul Bass revolutionized the opening credits by giving a narrative and artistic dimension to it. His work could almost be seen has a short-film but that was at the same time completely part of the all movie. He didn’t intent to illustrate the informative phase of the credits but to stage graphically (with line, forms and typography) the elements that will be then used in the film.

Stone et al. (2004) runs an experiment called Type in Motion to understand the added value of the animation in the text. They tested emotions at appropriate points using a mood rating scale. The procedure consisted of showing stimuli to the subject and asking them to fill a quantitative and qualitative mood form in order to establish an emotional baseline. The first figure

23 Various names could be to refer to kinetic typography such as animated

text, moving text or type in motion.

« My initial thoughts about what a title could do was to set mood and to prime the underlying score of the film to express the story in some metaphorical way. I saw the title as a way of conditioning the audience, so that when the film actually began, viewers would already have an emotional resonance with it. I had a strong feeling that films really began on the first frame» (Haskins, 1996)

61

situations such as closed captioning for television or in live theatre. Situations where temporal restriction doesn’t give the possibility to describe the language. They support that the use of symbols, animation or moving text could express sound information and decrease the amount of text-based and the rate at which captions move.

«These alternatives could also be used to capture some sound information such as music and sound effects that cannot easily be described using text (often because there is no time or space in text captions for additional descriptions).» (Rashid et al., 2006)

Their study where developed based on four emotions; fear, anger, sad and happy. The animation properties consisted of a set of standard kinetic text properties. These were: the text size, the position on screen, the opacity of the text, the speed of the animation, the duration and the vibration effect. Table 1 summaries all of the property descriptions for each primitive emotion24.

The conclusion of this research was positive in a sense that the animation did work better on the audience, but the team raised the issue of the duplication of the process and the creation of automatic animation tools that could be implement this framework with standard video editing tools.

24 Emotions are chosen by the experts on

basis of Plutchick’s research Emotion, a psycho-

evolutionary synthesis (1980)

presented to the audience was always the still image of a typographic message, the follow sequences were animated message. In addition to the form, the searchers recorded the facial expressions and body language of participants to complement and give more elements to the questionnaires. In conclusion, from the quantitative form they noted the significant difference of efficiency in favor to the motion typography. Secondly, in the qualitative form the results appear to support the idea that motion typography influence reactions within the viewer and make typographic communication a richer and memorable experience.

At the University of Tokyo in the same year, Want et al. start from the assumption that information communicates by words represent only the 20-30% of the information conveyed in a conversation. They then studied and showed as Stone et al. that animated text was effective at conveying a speaker’s tone of voice, affect and emotion. In this study the difference was made with the skin response sensors (GSR) that gives to the study a stronger support that joined the results of Stone at al. studies; the emotional response of the user was heightened for animated text compared with static text.

In the study Expressing Emotions Using Animated Text Captions (Rashid et al., 2006) searchers suggest that kinetic typography could help to communicate specific sub-textual meaning without resorting to narrative description in

62

__________

Table 1. Summary of the relevant animation properties for anger, fear, sadness, happiness and disgust

Emotions

Fear

Anger

Sad

Happy

Relevant Properties (effect)

Size: Repeated and rapid expansion and contraction of the animated text.

Rate: Expansion and contraction occur at constant rate.

Vibration: Constant throughout the effect.

Size: Abrupt expansion and con- traction of the animated text (through one cycle).

Vibration: Occurs at largest size in cycle.

Onset: Fast onset

Duration: Pause at largest size in cycle and vibration occurs here.

Size: Vertical scale of text decreases

Position: Downward vertical movement from baseline.

Opacity: Decrease in opacity

Offset: Slow offset.

Size: Vertical scale of text increases

Position: Upward vertical movement from below baseline. Follows a curve (fountain-like).

Intensity of Effect

Low: Size of animated text is the same as non-animated text at onset. Low level vibration.

High: Size of animated text is larger than non-animated text at onset. High level of vibration.

Low: Size of effect is smaller. Slower onset.

High: Size of effect is larger. Faster onset.

Low: faster offset

High: slower offset.

Low: Slower onset. Offset text size is smaller.

High: Faster onset. Offset text size is larger.

63

2.1. Motion, emotion and shapes

The shape of sound

As seen with the chapter on typography, visually, it would be possible to relay the idea of what is listened. Researches about sound symbolism and phono-semantics are profuse and can be applied not only to letters but also to forms. Those researches have often been developed in the field of linguistic, marketing and artistic.

The sounds and shapes of the objects would have characteristics in common that can be abstracted, for instance a word can be defined as sharp or cutting, such as defined in what Ramachandran calls the Bouba Kiki effect25, an experiment run in 2001 with Edward Hubbard (Ramachandran et al., 2001) in order to understand if the human brain do attach abstract meanings to the shapes and sound. They asked American college undergraduates and Tamil speakers in India to relate the nonsense words bouba and kiki to two abstract shapes. More than 95% of both groups selected the curvy shape as Bouba and the sharp one as Kiki. Ramachandran points out that cross activation could explain our ability to make metaphors. More recently, research on ideasthesia indicated that Kiki and Bouba have an entire semantic-like network of associated cross-modal experiences. We use metaphor to describe our sense of sound such as visual sensation (dark, bright, deep sound), or tactile sensation (soft, smooth, hard).

25 This effect was first studied by Wolfgang Köhler, a German-American

psychologist in 1929 on the island of Tenerife, and the shape were called Takete

and Baluba or Maluma.

64

2.2. The emotional value of shapes

In 1924, Poffenberger and Barrows explored how lines are able to communicate emotion. They studied stimulus-response, for them our eyes move along the line and don’t see it as a whole but read it. It is about a physical experiment that remember the body language that we use to express our feelings. In their paper The feeling value of lines (Poffenberger et al., 1924), the searchers asked 500 participants to link emotional adjectives to a set of very simple lines ranging from graceful curves to sharp-edged blades. There was surprising universality in the result. For instance, sad was seen as a soft and descending line and on the opposite merry would be seen as a rising line with tight bend.

Some theories of aesthetics explained that our response are deeply inbuilt biological responses that come from circuits designed to fight or escape against predators but in the paper, the scientists mentioned another interesting possibility. Based on the theory of Ethel Puffer about aesthetics, they suggested that when we observe a line we make a series of movements to

__________

Figure 20. Line and suggested typography. Retrieved in Why Fonts Matter, Hyndman (2016)

65

any shape and a proof that attitudes can achieved with the simplest of shapes.

Talking about three-dimensional shapes to represent emotion, in 2016, Edward Melcer (New York University) and Katherine Isbister (University California, Santa Cruz) run an experimentation to understand the emotion by the metaphor of creation of shape. This creation of shapes was made with clay in order to give access to a wide range of participants and not only designer that could create digital models with Computer Aided Design (CAD) tool. A total of 14 persons participated in the clay shape construction, none of them were professional designers. The experiment run different phases, but the main goal

carry out the observation, as explained earlier, we read the line internally but also physically. The movements of our eyes as we follow the line produce feelings which we attribute to the lines themselves. That would explain why the sharp-edges line with tight repetition would agitate us. In Why Fonts Matters (Hyndman, 2016), the designer interestingly reports this study and confront it with a suggestion of typeface equivalent. (See Figure 20)

Interestingly, the expression of emotion through forms appears to be studied in other field such as Disney Flour Sack, a range of forms created by a flour sack that represent emotions without defined human forms. It’s a guide that was originally created to maintain volume in

__________

Figure 21. Disney Flour Sack, an illustration of emotions despite the absence of a clearly articulated human figure.

66

was to replicate with the clay a single emotion or a mixed-emotion, the total generated was of 42 clay shapes. Moreover, the audio was recorded during the test and then analyse to identify themes in the descriptions of shape and its relationship to certain emotions. Based on the result of both shape and discussion with participant, it appears that ordinary individuals can embody emotion within form to some extent but more than this they also found an unexpected aspect of embodiment: the importance of motion and tactile manipulation. The use of motion-oriented forms helps tie together understanding of the relationship between motion, emotion and form. (See Table 2 for conclusion)

__________

Table 2 Retrieved from the conclusions of Melcer (2016)

SurpriseThin, Misshapen, Twisted, Irregular

FrustrationStacked, Enclosed, Pointy, Spiky, Sharp, Ridges, Unbalanced, Fragmented, Shredded.

ExcitementGoing Everywhere, Arch, Connected, Spiral, Extending Away, Reaching Upwards

BoredomFlat, No Form, Repetitive, Droopy, Spreads Out, No Structure, Melting

ContentStable, Balanced, Symmetric, Open, Blunted, Round

DisinterestPhysical Distance (between objects or from observer), not connected, Long.

High Arousal Big, Thick, Curved, Central Mass, Size, Variation, Visual Variation, Tall

Low Arousal Little or No Visual Variation, Simple, Curved Up, Smooth, Small

67

2.3. Case studies

Hugo Meier Thur narrated by M. Haverkamp

The German Hugo Meier Thur was one of the great artists of the 1920’s highly involved in illustrating the synesthesic perception of music and sounds in music-painting, graphic art and typography. He crafted an impressive series of images but the wars caused the destruction of many of his works. Today it’s still difficult to find information or sources about this artist. During the International Congress of Synaesthesia, I had the chance to discover his work thanks to the studies of Michael Haverkamp. Meier-Thur’s artwork are interesting researches about how playing with non-objective forms and complex spatial arrangement. Moreover, his work remains often in black and white, he actually worked and studied the perception of colour but for a reason of print media production, great part of his research appears in black and white. This particularity makes his work even more interesting because it allows us to focus on the power of visual forms.

__________

Figure 22. Retrieved in Haverkamp Hugo M. Thur – Synesthesia and Life in Black and White (2018)

68

Many characteristics could be point out from his work. Graphically, dotted tones (e.g. bell, piano, harp) are distinguished from linear tones (e.g. violin, flute, trumpet). (See figure 24) Violin are represents with ornament lines meanwhile grand piano with circular forms open on a one side. Fortissimo as well as low pitch cause larger image than the high pitch and pianissimo. We can also notice the movement from left to right and from bottom. «According to Meier-Thur’s descriptions, synesthetic image and those perceived visually cause different emotions.» (M. Haverkamp, 2018).

In the next figures we can see what his imagination tell us about the curves assigned to drums and the exploded sounds of cymbal. Figure 23 shows his perception of the ticking sound (above) and the striking (below) of a wall clock.

__________

Figure 23 and 24. Retrieved in Haverkamp Hugo M. Thur – Synesthesia and Life in Black and White (2018)

The work of Meier-Thur results limited or uncompleted for an obvious limit of time but still deserve to be mention. It brings into light that synaesthetic phenomena can show remarkable complexity and could be a basis for further and broader studies.

69

Oskar Fischinger’ New perspectives of visualisation

« My film work operates on a completely different level. It’s all about the purely optical manipulation of light, color, shapes, and movement, which derives not from the subject matter, but from the thing itself and the materials. A new, not-real world of color and movement first has to be discovered through an intense, creative effort.» (Fishinger, 2013)

Oskar Fishinger is seen as the father of Visual Music, but what does it really mean? What were his thoughts on ‘visual music’? words that were not actually found in his writings until the mid-1940’s, along with ‘optick music’ and ‘augmenmusik’ (Fischinger, 2013)

Interest in the correspondences of sound to color, and, music to image, extend back to Aristotle, Pythagoras, and Isaac Newton. Today, definitions of Visual Music rapidly change as the field expands. William Moritz, who devoted thirty-five years to the study of Fischinger and Visual Music described in 1986:

«A music for the eye comparabletotheeffectsofsoundfortheear»

(Fischinger, 2013)

Yet, Fischinger explain that his films are no illustrations of music but an architectural ground plan from which he took the time

__________Figure 25. Experiments in Cinematic Abstraction (2013)

70

The Center for Visual Music has in its archives a collection of musical scores used or produced by Fischinger as part of the creative process in making his films. These scores allow to study the musical aspects of Fischinger’s work in detail.

This segment of a graphic score (Figure 26) depicts the first two bars of Sousa’s Stars and Stripe march, used by Fischinger for this film. The central black and red section represents the music in four distinct representations. The top one shows the main melody, with next line down showing the accompaniment. The third shows a rhythmic breakdown of the music. Above this is an area detailing the animation. Although here there is no graphical representation, considerable

and rhythm. Indeed, I believe this concept is an interesting point since the goal is not to represent music straightly but to take it as a bases in order to develop a translation able to deliver the same effect.

Fischinger’s tight synchronization to jazz in a film like Allegretto demonstrates that music demands more than just a rhythmic beat to suggest the structural dynamics of key signatures and development patterns in music.

«Rather a complex, layered symphony of integrated parts, with rhythmic background, harmonic supports, melodic bravura solos and some overall integrity of color and for [...]»

__________

Figure 26. Experiments in Cinematic Abstraction (2013)

71

timing information is given. This gives us a great picture of Fischinger’s interpretation of the music.

It’s seem obvious that Fischinger has chosen to use these four bars as an exposition to reveal the flag; the first two-bar melodic phrase representing the stars and the second the stripes.

«This interpretation has logic and charm, the first phrase being rythmically angular and jaunty, like the graphical form of a star; and the second being entirely regular, both in terms of pitch and rythm, evoking the uniformity of a series of stripes.»

Fischinger does indeed illustrate the first phrase with five star, which would seem to reflect the fact that there are five notes in the melody of this phrase. The second phrase is accompanied by annotations referring to six stripes. It is not immediately obvious how the six lines appear in the animation for this section, which seems on first viewing to consist of a bottom-top wipe revealing the American flag.

__________

Figure 27. Experiments in Cinematic Abstraction (2013)

72

ICE Cinema and the peripheral vision

Last but not least, the ICE cinema! I’ve chosen this last case study in order to introduce the world of immersive design. Without going too deep in the immersive technology, ICE Cinema is a new technology that deserve to be taken into account for my thesis. I believe 3D and immersive typography could be a next step in this kind of researches.

The immersive media gather all the technology possible around one user in order to dive him in the core of an experience always more realistic and sensational. All the senses are called to create an emotional link between a specific universe and the user. It puts the subject in a psychological state where he doesn’t realise about his physical state anymore. The subject is focus and loose his notion of time. We could draw a parallel with the idea of a trip in another space-time.

__________

Figure 28 ICE Cinema

73

ICE Cinema give you the possibility to enjoy a movie with an optimal vision without being forced to enter another dimension, such as 3D glasses often provoke. It’s proven that the peripheral vision could modulate emotion (Rigoulot et al.,2012) without taking too many attention of the viewer. It other words, viewer even if passively viewing the periphery of the visual field, still build an affective modulations. (De Cesarei et al., 2009). For this reason the ICE projection is interesting. They even often blurred the lateral image because they don’t expect viewer to turn their head but to capture visual information unconsciously. That’s what make it special, because it doesn’t require extract effort. Here we could talk about expanded cinema in the way that the immersive environment pushes the boundaries of cinema and rejects the traditional one-way relationship between the audience and the screen.

Nowadays those experiences are often use in Marketing and communication, for instance Virtual Reality, Augmented Reality, 3D, Projection, Video Mapping and so on. The main reason to use those kinds of experience is because they heavily involved the customers touching different levers. Such as the cognitive lever; immersive experiences leave a memorable print in the mind of the customer, they also play with the emotional lever; they send back to the customer a precise image, and, finally, they give the yen to consume the product or the services, playing with the connotative lever. The implication of the subject is what make the success of those kind of experiences, but the achievement is not an easy work, it requires advanced technologies and understanding of the human being. But what about using those technologies in an accessible perspective, in order the enhance senses or reinforce the one missing?

Peripheral vision (120° -180° )

Acute vision (3°-10°)

Fairly clear (10°-12°)

74

Cover retrieved in Birdman (2014)

PART 3

KINETIC SUBTITLINGPROJECTAn accessible alternativeto translate music in films

Cover retrieved in Birdman (2014)

77

6. KINETIC SUBTITLES’ PRODUCTION__________

Already quoted in the first part, Elisa Roy was an inspiration for me in the way that she opens minds on all the benefits that we can have by designing first for disabilities. Her speech motivated me to underline alternatives to subtitling in our world of digital

and immersive experiences.

My researches have been useful to understand where to start the project; On the one hand the first part helped me understand how much music is important on the narratively point of view but also its great emotional role. It helps to rhythm and conduct the narration and to understand the feelings of movie and characters. It was also interesting to point out that music is hard to be taken separately; the complication of a movie makes its beauty and music works also with sounds, noises and voices. On the other

«I believe that these unique experiences that people with disabilities have is what’s going to help us make and design a better world for everyone - both for people with and without disabilities» (Roy, 2015)

78

hand, I was pleased to note that alternative subtitle researches have been run in the field of typography and forms, even if it stayed at an experimental state. The visual can be reinforce by translating what cannot be heard by DH. Many attempts have been conducted by researcher to find how kinetic and still form was powerful to the human vision and it pushes me to gather all those informations and try to develop my own project of translation.

Emotion is at the centre of my project in the way that it frequently comes back as a reference of characterisation. First subjectively when translators where putting under the spotlight and asking to translate more emotionally what they hear, in order to help the viewers understand better the movie. In a second hand, objectively, to describe the role of music in film but also to explain the emotional valour of visual works links to typography and forms. I will therefore use it has a vector of translation from the music in movie, using the characterisation of JR Julien, into typography and forms, using the bibliography of my second chapter.

For a matter of time and resources, my project, that can be defined as an experiment, will stay in the field of hypothesis, putting limit on how this experiment could be developed to construct a new way of immersive subtitling. I define my project immersive in the way that the visual will be reinforce by the animation itself but also by the way of displaying it. In a first part, I will explain my interpretation of subtitling with the input of kinetic typography and forms. This part will be test, in its limits, with a group of participants. In the second part, though, the way of displaying, will stay in the theoretical field.

With this experiment I hope to create keys to further translation in the field of Audio-Visual works for designer or translators itself. This method was based on my previous researches in the part 2 and will be explained in a specific context of translation. I hope also to determine, with the experiment, in which extension this project would be convertible.

79

7. GUIDELINE TO ALTERNATIVE SUBTITLES____________

1. Starting point

A practical example of what music narratively and emotionally represents in movie can be found in Birdman (The Unexpected Virtue of Ignorance), an American comedy directed by Alejandro G. Iñárritu. The movie received countless rewards by the film-making industry in 2014. It seems important to me to choose a movie relatively new in order to confront a closely recent reality of DH translation. Birdman talks about Riggan Thomson (Michael Keaton) a fade Hollywood actor who was internationally known for his role of a famous superhero. But of this celebrity there is not much left, and he tries today to set up a play on Broadway in the hope of reconnecting with his lost glory. During the few days before the first opening, he faces his past, his family and loved ones, his dreams and moreover, his strong ego that often overtake his personality.

80

Director Iñárritu has previously said the soundtrack, mainly comprised of Sanchez drumming solo, helped guide the audience through the film. “In comedy, rhythm is king, and not having the tools of editing to determine time and space, I knew I needed something to help me find the internal rhythm of the film,” he said26. The jazz drummer Antonio Sanchez rhythm the tension and the duality that the main character is facing along the all movie but, surprisingly, the director chose to replaced drums by more classic symphonies of the 19th century by Gustav Mahler, Tchaikovsky and Rachmaninov to emphasize such moments that I would describe has relief and bliss instants. For instance, the first time we heard such symphony arrives when Riggan first plays his monologue under the spotlights.

From the classification of JR Julien, studied in the chapter The induction of emotion of this thesis, I chose two different scenes were the emotion is dictated by the music. As a reminder, JR Julien insists on the essential contribution of the music in movies. He observed that the feelings expressed by the movie and for whom music is created for, can be divided in two types. Firstly, the emotional state strongly “musicalisable” (Julien, 1980). And then, secondly, the emotional stage lowly “musicalisable” because the production and the pictures doesn’t let the music support them. Between those two poles, he sets a series of emotion that the music produces on the audience; love, the music of danger, sorrowful, tenderness, furious and hatred and finally bliss (happiness). For this experience, I’ve retained hatred and bliss, two contrary emotions in order to underline the difference in translation.

26 Exclusive video with Iñarritu, retrieved in http://blogs.indiewire.com/,

consulted on March 2018

81

2. Creative process

The concept consists in generating an alternative translations to the Closed Caption for two different scenes. The process is based on my previous research about forms’ and typo’s induction of emotion. While there were natural limitations to this creative process, it seemed the most useful and productive method at this initial stage. The following account is a description of the process undertaken.

Hatred Scene, Schizo by Antonio Sanchez

The first scene chosen, at the minutes 55, works at the pace of Sanchez’s drums. Riggan has just discovered that his co-star Mike stole his limelight, he is really nervous and argue about it with his girlfriend, he also mentions his greatest fear about being humiliated. Upset her girlfriend leaves the room by slamming the door. At this point, music starts. But not only, his inner voice comes back to criticize him and tell him how humiliated he is by Mike. He then goes to him with anger and wake him up from a sun bed. The music composed by Sanchez is called Schizo not by accident, its inner voice gets the upper hand in rhythm with the drums, the first seconds are almost unsettling. The brushes on the snare drums are going back and forth, like a freebie’s movement and taking out our senses. Then the bass drum seems to take back the rhythm and push Riggan to act and fight with Mike.

__________

Figure 29. Retrieved in Birdman, Alejandro G. Iñárritu (2014)

82

0 5 10

Klute Lucida Bright Demibold Italic

[DOOR CLOSES]

BIRDMAN: Forget about her. He stole your front page.He’s stealing your show, he thinks you’re a joke.

Brush on the snare drumsCymbal

Constant bass drum

Photograms(1 frame every 5 sec.)

Seconds

Official English Closed CaptionBIRDMAN (Consolas Bold Italic)Reggan (Consolas Bold)Mike(Consolas Bold)

Waveform (dB)

-3-6

-12

-12-6-3

Music annotation

Suggested line / movement

Suggested animation Glitch and skew animation on the characters Jumping characters and office light effect

Suggested typography

Suggested subtitles

ExplanationSee text 1 2

83

10 15 20 25

Klute Lucida Bright Demibold Italic CONSOLAS BOLD (ALL CAPS)

He stole your front page.He’s stealing your show, he thinks you’re a joke.

Now two million people agree with him.

[BREATHING HEAVILY]Maye you are, Riggan, Maybe that’s what you are--a joke.

Brush on the snare drumsCymbal

Constant bass drum Steady pace rythm, jazzy soundsHi-hat

foot pedalsClear and

costant rythm

Glitch and skew animation on the characters Jumping characters and office light effect Glitch and skew animation + scalling animation on the square brackets

1 2 3

84

25 30 35 40

CONSOLAS BOLD (ALL CAPS) Consolas Bold

Woa what the hell is going on?

Punchy rythm, jazzy sounds Roll of drumsClear and

costant rythm

Raining CharactersGlitch and skew animation + scalling animation on the square brackets Spinning and overlay effect

3 4

85

40 45 50

Consolas Bold IMPACT (ALL CAPS)

Woa what the hell is going on?

Get up!Ow! that’s hurts!

Roll of drums Clear and constant rythm

Raining Characters Spinning and overlay effect

3 4

86

1 ----------------Klute is for me a good choice of typeface to represent the personality of Birdman. Birdman is indeed the alter-ego of Riggan, his inner voice mainly negative and unfriendly. Klute typeface can be defined has sour and unfriendly, a good way to represent this inner voice that arrives with the spiky drums sound. Unfortunately the font is almost unlegible so I decided to use it only to introduce the voice. Lucida Bright Demibold Italic is then used for the following words, it’s legible and remain an angular typography with sharply cut serif that gives a tense perception.

Original subtitle:

Suggested subtitle:

1. The proportion of the suggested font is inconstant

2. The characters are sloping on the right giving a sense of speed.

3. Angles and curves are sharply cut.

For the animation I chose to put glitch and skew effect when the brush is played on the snare drums, giving a sense of the so called music: schizophrenia.

Birdman

BIRDMAN11

1

2

2 31

2----------------On the one side, characters are following the bass drums by jumping and appearing by block. On the other side, when the cymbal ring, I added an office light effect.

3----------------In this part, the tension is rising, the music is getting punchy and at the limit of aggressive. I represent this feeling with glitch animation and skew effect. Moreover I decided to animate the brackets with the rising ticking almost inaudible at the beginning of the scene.

At the end, music explode with the ambient sound of the scene, to represent this explosion I used raining characters.

4----------------Riggan finally speak after 40 seconds of silence, he exploded and shout at Mike. The font Impact were used in all caps to underline his anger. The character are bold and compact, like a punch in the face.

The final version of this translation can be found on Youtube here: https://youtu.be/zwiPVIjvFXg

87

Bliss scene, Symphony No. 2 In E Minor, Op. 27-II Allegro Molto, Sergei Rachmaninov

The second scene starts at minutes 91, Riggan had a tough night drinking in the street after being criticized negatively by a reviewer that didn’t even see his show. His inner voice Birdman woke him up with some compliments, for once, and bring him up to the roof of the theatre. The extract starts upper there and the moment is interesting because for once the character decides when the music start and stop, it’s seems that for once he is dictating his happiness and his inner voice helps him. He jumps from the roof and starts to fly over New York busy streets, he is happy and laughing, Birdman supported his happy thoughts. The extract finish when he lands straight on the street and tell the doorman to stop the music.

The symphony composed by Rachmaninov was often described as spectacular because it deploys many colourful and rhythmic themes. This surprising balance between orchestral complexity and expressive clarity is rare: despite the huge orchestral waves, the whirlwinds of sounds and notes, the messages remain completely comprehensible and memorable for the listener. In this excerpt, the music gives the idea of something luminous and positive, the image instead could be contradictory in the way that it could also be seen as a suicide.

__________

Figure30. Retrieved in Birdman, Alejandro G. Iñárritu (2014)

88

Photograms(1 frame every 5 sec.)

Seconds 0 10

Waveform (dB)

Music.

[SOARING MUSIC BEGINS PLAYING][MUSIC STOPS]

You okay, man?

-3-6

-12

-12-6-3

Suggested typography

Suggested subtitles

VAG Rounded Consolas Bold

Music annotation Symphonic music Music sudden interruption

You want me to callsomebody for you? Do you know where to go?

Suggested line / movement

ExplanationSee text 1 2

Suggested animationRising characters then

suddenly falling Wavy characters then rising

Official English Closed CaptionReggan (Consolas Bold)BIRDMAN (Consolas Bold Italic)Stranger (Consolas Bold)

89

Photograms(1 frame every 5 sec.)

Seconds 0 10

Waveform (dB)

Music.

[SOARING MUSIC BEGINS PLAYING][MUSIC STOPS]

You okay, man?

-3-6

-12

-12-6-3

Suggested typography

Suggested subtitles

VAG Rounded Consolas Bold

Music annotation Symphonic music Music sudden interruption

You want me to callsomebody for you? Do you know where to go?

Suggested line / movement

ExplanationSee text 1 2

Suggested animationRising characters then

suddenly falling Wavy characters then rising

Official English Closed CaptionReggan (Consolas Bold)BIRDMAN (Consolas Bold Italic)Stranger (Consolas Bold)

20 30 40

VAG Rounded (modified Italic)

Few oboe’s notes Symphonic music

Do you know where to go?

Yeah.I know where to go.

Oh! Hey!

[SOARING MUSIC PLAYING] [ RIGGAN LAUGHS ] [LAUGHING]

Oh! Oh!

3 4

Wavy characters then rising Twirl effect

90

40 50 60

VAG Rounded (modified Italic) Consolas Bold

Symphonic music Music is almost turned off to let Birdman talk

Whoa!

[BELL TOLLING]

BIRDMAN: You see?

4

Twirl effect

91

70 80 90

Cooper Black

Symphonic music restartsMusic is almost turned off to let Birdman talk

BIRDMAN: You see? This is where you belong. Above them hall.

5

Soft transition between words Rising characters

92

4----------------For consistency, Riggan kept his font as the stranger do so. The focus is done to the happiness of Riggan, he is flying on the rithm of Rachmaninov’ symphony. The music is absorbing, the envelop of the sound has smoother curves such as the animation. Music is going with lightness and frivolity. Riggan is laughing and enjoying this moment.

5----------------The music is diminished suddenly to make Birdman talk. Compared to the hatred scene, this time he is happy and follows Birdman mood. For that I decided to used falling and rising transition with soft curves. Even so, Birdman remains the scary and sarcastic inner voice and the music underline it by going quieter as if a danger was pending. For this reason I decided to skew negatively the font Cooper Black when his name is cited in order to give his sarcastic personality back.

1 ----------------Riggan is really calm, he decides to be happy and calls for music to underline his mood. This part last only few seconds because a stranger stops his enthusiasm. VAG Rounded types is used to represent the peacefulness of Riggan. This typeface is often described as comfortable, cheerful and for doer and idealist person (Hyndman, 2016). The animation follows the rising symphony that start when Riggan says it and stops when the stranger touch him, causing the falling characters. 2----------------In this part, the music is suddenly stopped. A stranger tries to help Riggan and asks him some question. Since the music is not present, I decided to keep the original CC subtitles, with the original font Consolas Bold.

3----------------Riggan will start to speak again, and with it the music comes back with some oboe’s notes. The music underline the folly of Riggan, and seems to mock the stranger’s questions. The typeface VAG Rounded is used again to maintain consistency with the character Riggan but this time the typeface is slightly modified with a skew (italic) to accentuate the music that is going on and is animated with wavy and quick effect to follow the oboe’s notes.

93

Original subtitle:

Suggested subtitle:

1. I didn’t maintain the all-caps of the original version in order to follow the symphony curves.

2. The skew is lightly negative in order to keep the sarcasm of the character.

3. In the original typeface, the curves are mainly round and lightly sharps. With the Cooper Black font there is only curves.

The final version of this translation can be found on Youtube here: https://youtu.be/up7i8HMabT4.

BIRDMAN1

1

2

2

3

3Birdman

__________

Figure 31. Retrieved in Birdman, Alejandro G. Iñárritu (2014) (Modified subtitles)

94

8. EVALUATION FORM____________

1. Sampling and Method

The main goal of this form was to understand the efficiency of kinetic typography as an alternative to closed caption to translate music. The questionnaires were anonymous, but they were asked to indicate if they are prelecutive/postlocutive deaf/hard of hearing. The form was sent to French and Italian speakers, among others to the student and Belgian center Kap Signes and to the Italian Instituto Pio Istituto dei Sordi di Milano. This form was tested with the help of 24 persons whose 10 were deaf persons, 2 were hard-of-hearing persons and 10 were hearing persons, often linked with deaf persons.

Subjects were told that they would view the textual animations on the screen and be asked to answer quickly and instinctively. On the first part, the goal was to identify the efficiency of using typography to personalize the name Birdman as a happy or rather angry person. People were also asked to identify words to describe the animated typographies. On the second part the animation was test using a 5-point Likert rating scale where 1 = very happy; 2 = slightly happy; 3 = neutral; 4 = slightly angry; and 5 = very angry. The scale remains unchanged for all the questions. All the animations were taken from the final project, the idea was to first try the animations without the movie on the background.

95

On the third part, this time, animations were tested directly with the movie. Participants were asked to give the general emotion of the scene, to answer if, for them, kinetic subtitles were useful to understand it and their difficulty to read the subtitles.

After the subjects had completed the test they were asked general information in order to interpret correctly their answer such as ‘Generally, are you satisfied with music subtitling?’

The data were then insert in an excel table and, in a first time, visualize graphically. Secondly, they were interpreted and discuss.

2. Visualisation of the results

FIRST PART: Personality in the typographyWhat’sthefirstwordthatcomesintoyourmindwhenyousee this name?

1

Category of words

Superhero (3)

Link to the movie (6)

Hatred (3)

Worrying (6)

Other (6)

Batman Super hero

Birdman Michael Kaeton

uccello

un film

uomo uccello

Effrayant exploser,Instabilité

inquiétant tensione

troublé

ça clignote

Intrigue Pazzo

competizione impersonnel

je n'aime pas

macchinina

piquant rage Énervé

96

2

Happy

10

1 2 4

neutralAngry

Happy

10

1 2 4neutralAngry

Category of words

Happiness (2)

Frivolity (11)

Sky (3)

Fun (2)

Link to the movie (2)

Cartoon (2)

Other (2)

Legend:

Deaf and hard-of-hearing persons

Hearing persons

10 9 9

2 2 2

10

Aereo Douceur Leggero

Légèreté

giocoliere

rigolote

Allegria Joyeux

Arcobaleno nuvola

volo

Dessin animé

dessin animés enfant

Misterioso

tondo

fun

sympathique

un film

uomo uccello

2 2 2 23 3

5 5

97

SECOND PART: KINETIC TYPOGRAPHY

Which emotion do you received seeing those animations?

2 4 5

2 4 5

10

1 2 5

10

1 2 4 5

10

1 2

10

1 2 4 5

10

1 2

10

1 2 4 5

Happy

Happy

Happy

Happy

Happy

10 10

10

10

10

Happy

Happy

Happy

Neutral

Neutral

Neutral

Neutral

Neutral

Neutral

Neutral

Neutral

Angry

Angry

Angry

Angry

Angry

Angry

Angry

Angry

10

10

10

Legend:

Deaf and hard-of-hearing persons

Hearing persons

2

2

8 9

3

2 2 2 23 4

66

134 55

1

1 1

2

56 6

2 21 2 1

75 5

4

54 4

1

5 5

3 342 2

5 5

12 2

68

5

98

THIRD PART: KINETIC SUBTITLESIn this part the two extract of the movie Birdman were presented, the “Hatred” and the other “Bliss” extracts.

Have you already seen the film?

Si: .....%No: .....%

What’s the general emotion perceived in this extract?

1 The final version of this translation can be found on Youtube here: https://youtu.be/zwiPVIjvFXg

colere

rancœur

énervement, tension, instabilité

énervement

rabbia

stress, peur

angoisse

ambiance tendue

stress colere

suspens malaisant

angoscia

ansia thriller

agitazione

haine

curiosità

distraente

normale

Category of words

Anger (10)

Stress (10)

Hatred (1)

Other (3)

99

Subtitles have been animated on the base of the music. Do you think, they help to understand the general emotion of the scene?

83,2% agree (20 out of 24)

Didyouhavedifficultiestoreadthesubtitles?

79% didn’t have difficulties to read it (19 out of 24)

What’s the general emotion perceived in this extract?

2The final version of this translation can be found on Youtube here: https://youtu.be/up7i8HMabT4.

plénitude compassion émerveillement

Joie légèreté De la joie joie

Témérité et joie gioia

La légereté

legerta volare

leggerezza

leggerta

liberté

libération Libération

liberazione

libertà

Comicità

Category of words

Satisfaction (4)

Frivolity (7)

Release (7)

Happiness (5)

Fun (1)

100

Subtitles have been animated on the base of the music. Do you think, they help to understand the general emotion of the scene?

83,2% agree (20 out of 24)

Didyouhavedifficultiestoreadthesubtitles?

83,2% didn’t have difficulties to read it(20 out of 24)

FOURTH PART: GENERAL INFORMATION

Are you...

Do you have professional experience with typography?

12 out of 14 from birth

average of 55%Note that we only took into account the answer of

deaf and hard-of-hearing persons (10+2).

average of 48%Note that we only took into account the answer of

deaf and hard-of-hearing persons (10+2).

How old are you?

When did they discover your deafness?

Are you...

Ingeneral,areyousatisfiedwithclosed caption subtitles?

Ingeneral,areyousatisfiedwithclosed caption subtitles for music in film?

w

man

woman

man

62,5% (15)

37,5% (9)

no

yes

66,6% (16)

33,3% (8)

51-70

41-50

31-40

21-30

16,6% (4)

16,6% (4)

20,8% (5) 45,8% (11)

hearing person

hard-of-hearing

deaf

49,9% (12)

8,2% (2)

49,92% (10)

101

to French and Italian speakers. For the general emotion and words perceived in the Hatred scene, the answer were mainly linked to anger and stress as I expected. In the Bliss scene, participants find satisfaction, release, frivolity and happiness.

About the general information, a consistent majority of woman answer it and a consistent minority were linked to the word of design and typography. Majority of the participants were in the 20’s and middle of them were deaf. For the visualisation I regroup hard-of-hearing persons with the category deaf because the participants were only two and were not then enough to represent a category by themselves. Finally I asked the satisfaction about closed caption subtitles, for this question only answer of deaf and hard-of-hearing persons were taken into account. In general, they were satisfied at 55% with the closed caption and only 48% with the general translation of the music, which prove that improvement has to be done on both side to make accessibility in the cinema greater.

4. Limitations

Emotion is a field difficult to measure and really subjective, for that reason I decided to stay closed to JR Remy classification of emotion given by music in films, but I believe the answer could have been enhanced. I realized the classification

3. Discussion

In the first part linked to the personality of the typography, participants seem kind of confuse with the use of the name Birdman since many of the participants had already seen the movie and thus associate the name with the movie in general or with a super-hero. Even though, we can underline the use of stressful words to describe the characters such as ‘blinking’, ‘worrying’, ‘explosion’. Instead, with the happy Birdman, it seems that the results were unanimous; they noticed the difference and qualify it more as something that glide. Words like ‘plane’, frivolity, happiness, softness, we’re used. Regarding the Linkert rating scale, majority answered that the first Birdman was angry and that the second one was happy, as planned but we could say the second one is more obvious.

Concerning the kinetic typography taken apart from the movie, it seems that the majority of the participants had perceive the shades of the animation even if it seem easier to be understood when the movement were accentuate, for instance with the words Breathing Heavily or the rising animation Above them all. When the kinetic typography is put on the movie, as subtitles, participant seem to all agree that it helps to understand the emotion of the movie and appreciate it. Only few of them find difficulties to read the subtitles, but it has to be said that the translation was done in the original language (English) and then proposed

102

had some limitation in the way that the music played in a scene can’t be only from one unique emotion but is filled with shades of emotions and it could have been interesting to collect. In that way I could also avoid neutral answer because of a lack of options.

Some useful comments also helped me to understand my limitation, for instance it would have been interesting to study the position of the subtitles, in my case I remained stick to the original position of the CC. Another comment was about the sound, for instance for hard-of-hearing or even for deaf, it’s useful to have the sound since the music can be still perceived, on a certain degree, with the help of vibration for instance.

Then, it would be interesting to compare directly with the participants the original static version with the alternative version, in order to weight the importance of future work in the environment of kinetic typography.

Another main issue needs to be taken into consideration: more attention should be paid to the characteristics and needs of the different audiences that are considered to constitute a unique group, that is the deaf, the Deaf and the hard of hearing. In each of these subgroups, there are differences in the level of deafness of each person.

The results and the original forms can be found in the Appendix 3 and 4 at the end of this thesis.

103

9. CONCLUSION AND POSSIBLE FUTURE APPLICATIONS____________

The world of the deaf is all but silent. The perception is a tool to represent the world not as it seems to be, but as our senses understand it. I believed we couldn’t talk about a specific sense as a simple perception but as cross- sensory implications. For Emmanuelle Laborit, deaf since birth, her eyes are her ears, she explained she doesn’t miss nothing but society makes her disabled and dependant. It was important for me to dig into the world of visual to see if its characteristic could enhance the sensations of hearing that one could received.

In developing the project, I have been interested in creating new type of visible language representation that brings question to some of our assumptions about forms, characters and meaning. We tried to give an answer to the hypothesis of kinetic typography as a tool to communicate a broader range of paralinguistic sensation and emotion that music delivers. More than it is possible with the static text. It seems that this pilot study conclude positively this research although its limitation. Deaf, hard-of-hearing and hearing persons went almost all on the same road and those participants were surprisingly positive about the all process.

104

My goal was not to find the best solution to describe music in movie but to open interest in this field, first because I believed music is an important and essential part of the movie and second because it’s has been proved in many ways that it’s accessibility was not enough developed. Music in an audio-visual work must be considered as an important part of the comprehension of the film, its semiotic aspect has a great impact on the viewing. Nevertheless, I need to precise that the angle of the film score was taken to give a specific direction to this thesis but I do think study about kinetic typography could also be run considering the tone of voice of the actors and as a way to improve static subtitles in general in order to enrich cinematic experience not only for hearing-impaired persons but for the whole audience.

A next step would be to make research in how to implement this technology in cinema, I think about the case study ICE Cinema where the lateral screen could be used in order to project animated typography.

Finally, it would be to understand how to codify that kind of subtitles and study in which measure this translation could enter in the budget of a movie considering the limitation of automation of the process. As closed caption subtitles, kinetic subtitles couldn’t be a literal translation because of the shades and distinctive features I believe only human could perceive.

105

106

BIBLIOGRAPHY____________

BALDO DE BRÉBISSON S. (2016), Formes, sens et pratiques du sous-titrage special, Signata, pp. 255-284. (Retrieved in http://journals.openedition.org/signata/1229).

BAULE G., CARATTI E. (edited by) (2018), Design is Translation: The translation paradigm for the culture of design. Franco Angeli Edizioni

CARAVALLO D. (2013), Synesthesia and the Arts, Mc Farland & Company, Jefferson, North Carolina.

CHIARO D. (2008), Audiovisual Translation, in “The Rouledge Companion to Translation Studies”, Routledge, London, pp. 169 – 170.

CHION M. (2014), L’audio-Vision: Son et image Au cinéma (3rd ed.), Armand Colin, Paris.

COHEN A. J. (2001), Music as a Source of Emotion in Film, in JUSLIN P. N., SLOBODA J. A. Sloboda(eds.), “Music and Emotion”, Oxford University Press, Oxford, pp. 249-272.

DAY S. (2005). Some Demographic and Socio-cultural Aspects of Synesthesia, in ROBERTSON L., SAGIV N., “Synesthesia: Perspectives from Cognitive Neuroscience”, Oxford University Press, New York.

DE CÓRDOBA SERRANO M.J., RICCÒ D. DAY S.A., SIDOROFF DORSO A., JEWANSKI J., LEYDEN T., PARRA V., LÓPEZ DE LA TORRE J., CARLUCCI, et al. (2018) VI International Congress Synaesthesia, Science & Art, Editorial: Fundación Internacional Artecittà, Granada.

DÍAZ CINTAS J., MUÑOZ SÁNCHEZ P. (2006), Fansubs: Audiovisual Translation in an Amateur Environment, in “The Journal of Specialised Translation”, vol. 6. (Retrieved from www.jostrans.org).

107

FISCHINGER O. KEEFER C. (edited by) GULDEMOND J. (edited by)(2013), Experiments in Cinematic Abstraction, Eye Film Museum, Amsterdam.

GAMBIER Y. (2003), Screen Transadaptation: Perception and Reception, in “The Translator”, vol. 9, Routledge, London, pp.171–189.

GAMBIER Y. (2009), Perception and Reception of Audiovisual Translation: Implications and Challenges, in CHE OMAR H., HAROON H., GHANI A. (eds.), “The Sustainability of the Translation Field”, ITBM, Malaysia, pp. 50–57.

HASKINS P. (1996). Saul, Can You Make Me a Title? In Film Quarterly, University of California, California, pp. 12–13.

HAVERKAMP M. (2007), Percezione sinestesica e design del rumore, in “Progetto grafic”o, vol.10, Milano, pp.162-169.

HYNDMAN S. (2016) Why fonts matter, Virgin books, China.

HOÉRÉE A. (1950), Histoire et fonction de la musique de film, in “Polyphonie: La Musique Mécanisé”e, vol. 6, Paris, pp. 59-66. ISBISTER K., MELCER E. (2016) Motion, Emotion, and Form: Exploring Affective Dimensions of Shape, in “CHI Conference Extended Abstracts on Human Factors in Computing Systems”, California, pp. 1430-1437.

JULIEN J.R. (1987), Défense et illustration des fonctions de la musique de film, in Vibrations: Les musiques des films, vol.4, pp. 28-41 (Retrieved from www.persee.fr).

JULIEN J.R. (1980), Éléments méthodologiques pour une typologie de la musique de film, in “Revue de musicologie”, vol. 66, pp. 179-202.

KANDINSKY W. (1989), Du spiritual dans l’art, et dans la peinture en particulier, Éditions Folio Essais, Paris.

KANDINSKY W. (1979), Point and line to plane, Dover Publications, New York.

108

LIPSCOMB S, KENDALL R. (1994), Perceptual Judgment of the Relationship Between Musical and Visual Components in Film, in “Psychomusicology: A Journal of Research in Music Cognitio”n, Vol 13, pp.60-80

LIPSCOMB S, TOLCHINSKY D. (2004), The Role of Music Communication in Cinema, in “Music Communication”, Oxford University Press, United Kingdom, pp. 383-404.

MARINETTI F.T. (1913), Typographic Revolution in DRUCKER J. (1997), “The Visible Word”, University of Chicago Press, Chicago.

MERLEAU-PONTY M. (1964), Le Visible et l’Invisible, Éditions Gallimard, Paris.

MELCER E. ISBISTER K. (2016), Motion, Emotion, and Form: Exploring Affective Dimensions of Shape. San Jose, CA, USA.

ORERO P. (2004), Audiovisual Translation: A New Dynamic Umbrella, in P. ORERO (eds.), “Topics in audiovisual Translation”, vol. 56: VII–XIII, John Benjamins Publishing Company, Amsterdam, Philadelphia.

PARKE R., CHEW E., KYRIAKAKIS C. (2007), Quantitative and Visual Analysis of the Impact of Music on Perceived Emotion of Film, ACM Computers Entertaint. (Retrieved from www.cie.acm.org).

POFFENBERGER A., BARROWS B. (1924). The Feeling Value of Line, in “Journal of Applied Psychology”, vol.2, pp.187-205.

RAMACHANDRAN V.S., HUBBARD E.M. (2001). Synaesthesia- A window into Perception, Thought and Language, in “Journal of Consciousness Studies”, vol. 8 No. 12, pp.18-20

RASHID R., AITKEN J., FELS D.I. (2006). Expressing Emotions Using Animated Text Captions, International Conference on Computers. Helping People with Special Needs (ICCHP), Austria.

RICCÒ D. (2018) Synesthetic translation. A theoretical framework, Proceedings of the VI International Conference Synesthesia: Science and Art, Alcalà la Real de Jaén, España, 18–21 May 2018,

109

Editorial Fundación Internacional Artecittà, Granada, April, 2018, pp. 1-10 (eBook, pp. 37-47)

RICCÒ D. (2008). Sentire il design: Sinestesie nel progetto di comunicazione, Carocci editore, Roma.

RICCÒ D. (1999). Sinestesie per il design. Le interazioni sensoriali nell’epoca dei multimedia, Etas, Milano.

SACKS O. (1989). Seeing voices, A journey into the world of the deaf, University of California Press, California.

SERBAN A. (2008). Les Aspects linguistiques du sous-titrage, in LAVAUR J. M. SERBAN A. (eds), ”La Traduction Audiovisuelle”, De Boeck, Bruxelles, pp. 85-99.

SCHJOLDAGER A., GOTTLIEB H., KLITGARD I. (2008). Understanding Translation, Authors and Academica, Aarhus, pp. 39-66.

SPIEKERMANN E. (2013). Stop stealing sheep & find out how type works, Adobe Press.

STONE B., ALENQUER D., BORISCH J. (2004). Type, Motion and Emotion: A Visual Amplification of Meaning, in McDONAGH D. et al.(eds), “Design and Emotion: The Experience of Everyday Things”, Taylor & Francis, London, pp. 190-4.

TAN W. (1996). Emotion and the structure of narrative film: Film as an emotion machine, Lawrence Erlbaum, Mahwah.

TEAL T. (2003) The Typographic Experiment, Thames & Hudson, London.WANG H., PRENDINGER H., IGARASHI T. (2004). Communicating Emotions in Online Chat Using physiological Sensors and Animated Text, in “Late Breaking Results Paper”, Viena, pp.1171-1174.

WIDMANN C. (2000). Il simbolismo dei colori, Edizioni Magi, Roma.

110

Sitography

Angelo Badalmenti Bio. Retrieved from < http://www.angelobadalamenti.com/ >

Dada – typographie & langageRetrieved from < http://indexgrafik.fr/ >

Emmanuelle Laborit, au-delà du silenceRetrieved from < https://www.franceculture.fr/emissions/series/emmanuelle-laborit-au-dela-du-silence >

Leitmotif in The New Grove Dictionary of Music online Retrieved from < www.grovemusic.com >

The monster meets ... Filmmaker David Lynch. Retrieved from < http://www.lynchnet.com/monster.html >

Timothy B Layden WebsiteRetrieved from < http://theshapeofsounds.com/ >

Sarah Hyndman WebsiteRetrieved from < https://www.sarahhyndman.com/ >

SynesthesiaRetrieved from < http://www.daysyn.com/ >

Synesthesia and Cross-Modal PerceptionRetrieved from < https://serendipstudio.org >

When we design for disability, we all benefit. Roy Elisa. Retrieved from < https://www.ted.com/talks/ >

Filmography

2001: A Space Odyssey (1968), Stanley Kubrick

8 Mile (2002), Curtis Hanson, Music by Eminem

Avatar (2009), James Cameron

Baby Driver (2017), Edgar Wright

111

Birdman (2014) Alejandro G. Iñárritu. Music by Antonio Sanchez.

Blue Velvet (1986) David Lynch (1986). Music by Angelo Badalamenti. Breakfast at Tiffany’s (1961), Blake Edwards.

Casablanca (1942), Michael Curtiz.

Cinema Paradiso (1988), Giuseppe Tornatore. Music by Ennio Morricone.

Dead Poets society (1989), Peter Weir. Music by Maurice Jarre and Peter Weir.

Dr. Jivago (1965), David Lean. Music by Maurice Jarre.

Dunkirk (2017), Christopher Nolan. Music by Hans Zimmer.

Eraserhead (1977), David Lynch. Music by David Lynch.

E.T. (1982), Steven Spielberg.

Eternal Sunshine of the Spotless Mind (2004), Michel Gondry.

Exorcist (1973), William Friedkin.

Inception (2010), Christopher Nolan. Music by Hans Zimmer et al.

Interstellar (2014), Christopher Nolan. Music by Hans zimmer.

Jaws (1975) , Stephen Spielberg. Music by John Williams.

La vita è bella (1997), Roberto Benigni. Music by Nicola Piovani.

Le Fabuleux Destin d’Amélie Poulain (2001), Jean-Pierre Jeunet. Music by Yann Tiersen.

Mommy (2014), Xavier Dolan.

Mulholland Drive (2001), David Lynch. Music by Angelo Badalamenti.

Once Upon a Time in America (1984), Sergio Leone. Music by Ennio Morricone.

112

Once Upon a Time in the West (1969) Sergio Leone. Music by Ennio Morricone.

Paramount on Parade (1930), Joseph L. Mankiewicz.

Pearl Harbour (2001), Michael Bay.

Psycho (1960), Alfred Hitchock. Music by Bernard Herrmann.

Romeo+Juliet (1996), Baz Luhrmann.

Sailor and Lula (1990), David Lynch. Music by Angelo Badalamenti. Slumdog Millionare (2008), Danny Boyle.

Star Wars sage (1977-1983), George Lucas.

The Dark Knight (2008), Christopher Nolan. Music by Hans Zimmer and James Newton Howard.

The departed (2006), Martin Scorsese.

The Green Mile (1999), Frank Darabont. Music by Thomas Newman.

The Jazz Singer (1927), Alan Crosland.

The Lion King (1994), Rob Minkoff, Roger Allers. Music by Hans Zimmer.

The Little Prince (2015), Mark Osborne. Music by Hans Zimmer and Richard Harvey.

Twin Peaks (1990), David Lynch, Mark Frost . Music by Angelo Badalamenti.

113

114

APPENDIX 1: DIGITAL METHODS RESEARCH 1.__________

Appendix 1: Digital Methods Research #1 Methods

1. On a first hand, before going deep into my researches, being in Italy I did a first research with Google.it. My main goal was to understand the general words that keep coming up when you used the query “traduzione audiovisuale”, the Italian words for Audiovisual Translation.

Tool Google Incognito Window Query Traduzione Audiovisuale URL www.google.it

2. I’ve collected the pertinent links of the first three page of www.google.it. That means that I

didn’t take into account the e-commerce and the PDFs.

Link_# URLs Typology Visit

Link_1

http://www.traduzione-testi.com/traduzioni/servizi-di-traduzione/che-cos%E2%80%99e-la-traduzione-audiovisiva.html Translation Agency 15/06/2018

Link_2

https://www.e-schooloftranslation.org/classes/traduzione-audiovisiva/

University (European School of Translation) 15/06/2018

Link_3 https://www.mediazionelinguistica.it/master/master-traduzione-audiovisiva-mba

University (Scuola Superiore Per Mediatori Linguistici) 15/06/2018

Link_4 http://www.mastraduvisual.com/it/modalita-in-presenza.php University (Universidad de Cadiz) 15/06/2018

Link_5

http://www.doppioverso.com/traduzione-e-audiovisivi-un-po-di-cose-che-credevamo-di-sapere-e-invece-no/ Translation Agency 15/06/2018

Link_6

http://www.iulm.it/wps/wcm/connect/scuoladicomunicazione/sdc/home/Master-per-professionisti/Doppiaggio-adattamento-traduzione-di-opere-cinetelevisive University (IULM) 15/06/2018

Link_7 http://www.yellowhub.it/servizi-linguistici/desk-traduzioni-audio-video/ Language consultants 15/06/2018

Link_8

http://www.fondazionemilano.eu/lingue/pagine/corso-traduzione-adattamento-sottotitolazione-2018

University (Civica Scuola Interpreti e Traduttori Altiero Spinelli) 15/06/2018

Link_9 http://www.fondazionemilano.eu/lingue/content/cinzia-di-barbara

University (Civica Scuola Interpreti e Traduttori Altiero Spinelli) 15/06/2018

Link_10 https://www.emagister.it/corsi_traduzione_audiovisiva-ek38852.htm

Information website (Piattaforma per i studenti) 15/06/2018

Link_11 http://www.professionedoppiaggio.com/corsi/traduttore-di-audiovisivi

University (Scuola professione doppiaggio) 15/06/2018

Link_12 https://www.unipr.it/node/17909 University (Universita di Parma) 15/06/2018

Link_13

http://www.terminologia.it/index.php/lifelong-learning/worshops/italiano-workshop-in-traduzione-audiovisiva/?lang=it

University (Scuola Laboratorio di Terminologia e Traduzione Assistita) 15/06/2018

115

Link_14 https://it.wikipedia.org/wiki/Traduzione_multimediale information website 15/06/2018

Link_15 https://it.jooble.org/lavoro-traduzione-audiovisiva platform job 15/06/2018

Link_16 https://www.lipsie.com/it/area_entertainment.htm Translation Agency 15/06/2018

Link_17 https://linguaenauti.com/tag/traduzione-audiovisiva/ blog 15/06/2018

Link_18

https://www.unicollegessml.it/it/master/mediazione-linguistica-culturale-con-migranti-ii-edizione/?gclid=EAIaIQobChMI2P-49Nzm2wIVkW4bCh1lIAe6EAMYASAAEgKRgfD_BwE

University (Scuola Superiore Mediatori Linguistici) 15/06/2018

3. I’ve passed those 18 pertinent links into Tag Cloud Generator. The main objective of this

digital tool is to discover the reccurent words and thus the main topics of the urls. Afterwards I’ve organized them by typology and finally by field.

Digital Tool Tag Cloud Generator1

word typology # field

esperienza skills 11 profession

marketing skills 21 profession

lavoro work 61 profession

contatti contact 23 profession

professionale professional 23 profession

professionisti professional 17 profession

competenze skills 20 profession

aziende agency 17 profession

management professional 17 profession

conoscenza skills 14 profession

profesional professional 13 profession

mercato market 13 profession

strategie skills 13 profession

manager professional 12 profession

produzione work 12 profession

editing work 12 profession

professione professional 11 profession

iva price 11 profession

1 https://tools.digitalmethods.net/beta/tagcloud/

116

tariffe price 11 profession

riferimento skills 11 profession

giorni day 27 duration

durata time 26 duration

time time 11 duration

italiano language 27 speaking

inglese/english language 66 speaking

lingua language 44 speaking

lingue language 34 speaking

dialoghi dialogs 24 speaking

voice voice 23 speaking

testo text 21 speaking

linguistica/o linguistic 32 speaking

voce voice 15 speaking

francese language 14 speaking

tedesco language 13 speaking

spagnolo language 12 speaking

studio/i research 74 search field

strumenti instruments 26 search field

ricerca research 29 research field

analisi analisys 14 research field

diritto diritto 13 law

norme norme 11 law

leggi leggi 15 law

traduzione/i, translation, traduttore, tradurre traduction 522 general traduction

doppiagio dubbing 108 general traduction

sottotitolazione/sottotitolaggio subtitles 108 general traduction

adattamento adaptaion 52 general traduction

sottotitoli subtitles 32 general traduction

audiodescrizione audiodescription 26 general traduction

copione scipt 22 general traduction

adattatore adaptation 21 general traduction

interpreti dubbing 15 general traduction

traduccion traduction 15 general traduction

sopratitoli subtiltes 14 general traduction

descrizione subtitles 14 general traduction

117

doblaje dubbing 11 general traduction

interpretazione dubbing 11 general traduction

audiovisiva/i/o, audiovisual 175 general communication

master master 110 education field

laurea laurea 48 education field

corsi education progam 74 education field

corso education progam 53 education field

studenti education progam 44 education field

ECTS education progam 40 education field

formazione education progam 37 education field

scuola education progam 33 education field

informazioni education progam 32 education field

iscrizione education progam 29 education field

modulo education progam 27 education field

univ/ università university 49 education field

programma/i education progam 47 education field

workshop education progam 25 education field

segreteria education progam 24 education field

docenti/e education progam 34 education field

stage education progam 21 education field

esercitazioni education progam 32 education field

processo education progam 19 education field

livello education progam 19 education field

orientamento education progam 18 education field

pratica education progam 18 education field

lezioni education progam 16 education field

test education progam 16 education field

ammissione education progam 14 education field

ateneo education progam 15 education field

percorso education progam 13 education field

bibliografia education progam 13 education field

didattica education progam 13 education field

moduli education progam 12 education field

webinar education progam 11 education field

erasmus Erasmus 11 education field

lettura education progam 11 education field

118

studente education progam 11 education field

borse education progam 11 education field

video video 36 communication media

web web 34 communication media

cinema cinema 34 communication media

film cinema 27 communication media

comunicazione communication 27 communication media

media communication 23 communication media

online web 22 communication media

storia communication 22 communication media

software web 21 communication media

multimediale communication 17 communication media

opere/a communication 31 communication media

documentari cinema 16 communication media

blog web 16 communication media

linguaggio communication 14 communication media

videogiochi video games 13 communication media

mediazione communication 12 communication media

msml web 14 communication media

milano milano 30 area

localizzazione general words 25 area

italia Italia 20 area

madrid Madrid 19 area

pisa Pisa 17 area

parma Parma 16 area

lombardia Lombardia 15 area

roma Roma 15 area

regione general words 12 area

firenze Firenze 11 area

sorde deaf 15 accessibility

119

4. I’ve created a visualization of the data.

120

APPENDIX 2: DIGITAL METHODS RESEARCH 2.__________

Appendix 1: Digital Methods Research #1 Methods

1. For the second experience I wanted to focus my researches on the AVT Master since they are forming the next generation of translators, moreover I wanted to develop a new kind of research. This time I decided to see what was going on in www.google.fr, the main reason of this change was because I realized how many translation researches were followed by French searchers. This time I took the first ten links1 that responds to the query “Traduction Audiovisuelle Master”.

Tool Google Incognito Window Query Traduction Audiovisuelle Master URL www.google.fr

Link_# URLs Typology

Link_1 https://uclouvain.be/prog-2018-intp2m-lintp803o University

Link_2 https://itiri.unistra.fr/masters-traduction/traduction-audiovisuelle/ University

Link_3 http://www.mastraduvisual.com/fr/descriptif-du-master.php University

Link_4 https://formations.univ-lille3.fr/fr/fiche/admission/17traductionetinterpretation/33502 University

Link_5 http://www.cinemadfilms.com/pages/un-metier-des-formations/traducteur-traductrice-auteur-de-doublage-de-sous-titrages-et-de-voice-over.html

information website

Link_6 https://uclouvain.be/prog-2018-intp2m-lintp803o University

Link_7 http://unice.fr/formation/formation-initiale/hmll12136 University

Link_8 http://www.profession-traducteur.net/etudiant/classement/spe_audio.htm

information website

Link_9 https://www.letudiant.fr/metiers/secteur/langues/traducteur-audiovisuel.html

information website

Link_10

https://www.parisnanterre.fr/offre-de-formation-/master-arts-lettres-langues-br-mention-traduction-et-interpretation-br-parcours-traduction-anglaise-specialisee-94321.kjsp University

2. Then, I treated them in a new digital tool called Google Scraper. To make it work you have

to give some URLs and a list of keywords, Google will be asked if each keyword occurs in each URL and count it. I’ve created different glossaries2 based on my previous researches, here again my goal is to see how often music came up as part of AVT.

Digital Tool Google Scraper3

1 See Appendix2

2 Glossaries were created thanks to the dictionary http://crisco.unicaen.fr/

3 https://tools.digitalmethods.net/beta/scrapeGoogle/

121

Word Total Glossary

musique(s) 0 Music Glossary

bruit(s) 0 Music Glossary

chanson(s) 0 Music Glossary

concert(s) 0 Music Glossary

mélodie(s) 0 Music Glossary

musicalité(s) 0 Music Glossary

son(s) 6 Music Glossary

parole 0 Speaking Glossary

voix 1 Speaking Glossary

discours 2 Speaking Glossary

dialogue(s) 2 Speaking Glossary

sourd(s) 8 Accessibility Glossary

malentendant(s) 4 Accessibility Glossary

aveugle(s) 0 Accessibility Glossary

handicap(s) 6 Accessibility Glossary

accessibilité(s) 5 Accessibility Glossary

3. I created a visualization NB: the word “son” in French can also mean “its, her, his”.

122

APPENDIX 3: KINETIC TYPOGRAPHY FORMITALIAN VERSION__________06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 1/10

Tipografia cinetica

Buongiorno, sono Alicia Gonzalez, una studentessa belga di design della comunicazione al Politecnico di Milano. Questo questionario ha l’obiettivo di testare l’efficacia di un nuova forma di sottotitoli in movimento che permetta di fare percepire l'andamento della musica alle persone sorde.

Passez à "Prima parte".

Prima parteNella prima parte, sono presenti una serie di animazioni tipografiche: ti chiediamo di rispondere istintivamente alle domande.

Passez à la question 1.

Personalità della tipografiaBirdman è il nome di una persona

1. Qual'è la prima parola che ti viene in mente

quando vedi questo nome?

Secondo te, Birdman è una persona...

1= felice, 3=neutro e 5=odiosa.

2. Une seule réponse possible.

1 2 3 4 5

Felice Odiosa

123

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 2/10

Personalità della tipografia (seconda parte)

3. In questo caso, qual'è la prima parola che ti

viene in mente?

Quindi diresti che Birdman è una persona...

1= felice, 3=neutro e 5=odiosa.

4. Une seule réponse possible.

1 2 3 4 5

Felice Odiosa

Passez à la question 5.

Transizione1= felicità, 3= neutro, 5= odio

124

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 3/10

5. A che emozione associ questo movimento?

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

6.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 4/10

7.

Une seule réponse possible.

1 2 3 4 5

Felicità Odio

AnimazioneQuale emozione associ a queste animazioni? 1= felicità, 3= neutro, 5= rabbia

8.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

125

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 6/10

11.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

12.

Une seule réponse possible.

1 2 3 4 5

Felicità Rabbia

Sottotitoli AnimatiSono presenti due estratti del film Birdman di Alejandro Iñarritu, sono stati tradotti con sottotitoli animati (in lingua originale). Ti chiediamo di rispondere dopo la visualizzazione a brevi domande. Il primo video ha una durata di 45 secondi, il secondo di 90 secondi.

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 7/10

13. Hai già visto questo film?

Une seule réponse possible.

Si

No

14. Quale emozione ti suggerisce questa

sequenza?

15. I sottotitoli sono stati animati sulla base della musica originale del film. Ti sono di aiuto a

capire l'emozione della scena?

Une seule réponse possible.

Si

No

Autre :

16. Hai fatto fatica a leggerli?

Une seule réponse possible.

Si

No

Autre :

http://youtube.com/watch?v=zwiPVIjvFXg

126

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 8/10

17. Quale emozione ti suggerisce questa

sequenza?

18. I sottotitoli sono stati animati sulla base della musica originale del film. Ti sono di aiuto a

capire l'emozione della scena?

Une seule réponse possible.

Si

No

Autre :

19. Hai fatto fatica a leggerli?

Une seule réponse possible.

Si

No

Autre :

Informazione generaleIl test è anonimo. Ci servirebbe qualche informazione per potere interpretare il risultato.

20. Sei

Une seule réponse possible.

Un uomo

Una donna

http://youtube.com/watch?v=up7i8HMabT4

127

06/04/2019 Tipografia cinetica

https://docs.google.com/forms/d/12v8SNDrIlxvAVue7ZpbXu-vqVtnunBplpTKiVpukzMM/edit 9/10

21. Hai mai lavorato nel graphic design?

Une seule réponse possible.

Si

No

22. Qual'è la tua età?

23. Sei

Plusieurs réponses possibles.

Sordo

Persona affetta da qualche deficit uditivo

Supporto professionale per persone sorde

Autre :

24. A che età ti hanno diagnosticato la sordità?

Une seule réponse possible.

dalla nascita

Autre :

25. In generale, sei soddisfatto dei sottotitoli che trovi nei film?

Une seule réponse possible.

0 1 2 3 4 5 6 7 8 9 10

no si

26. In generale, sei soddisfatto del sottotitolaggio della musica?

Une seule réponse possible.

0 1 2 3 4 5 6 7 8 9 10

no si

27. Vuoi lasciare un commento?

Grazie!Grazie del aiuto, non esitare a contattarmi per ulteriori informazione a questa e­mail: hello@alicia­gonzalez.com

128

APPENDIX 4: KINETIC TYPOGRAPHY FORM FRENCH VERSION__________

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 1/10

Typographie cinétique

Bonjour, Je m'appelle Alicia Gonzalez, je suis étudiante en communication design au Politecnico de Milan. Ce questionnaire a pour but de tester l’efficacité d’une nouvelle forme de sous­titres en mouvement qui permettrait de transmettre les nuances de la musique aux personnes sourdes.

Première partieDans la première partie, vous trouverez une série d’animations typographiques suivie de questions pour lesquelles vous devrez répondre de manière instinctive.

Personnalité de la typographieBirdman” est le nom d’une personne

1. Quelle est la première impression qui vous

vient en tête en voyant son nom?

2. D’après vous, Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Personnalité de la typographie (deuxième partie)

129

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

130

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 2/10

3. Quelle est la première impression qui vous

vient en tête en voyant son nom?

4. Ici, vous diriez que Birdman est une personne...

1= joyeuse, 0= neutre, 5= odieuseUne seule réponse possible.

1 2 3 4 5

Joyeuse Odieuse

Transition1= joie, 0= neutre, 5= colère

5. Quelle émotion vous transmettent ces animations?

Une seule réponse possible.

1 2 3 4 5

Joie Colère

131

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 6/10

12.

Une seule réponse possible.

1 2 3 4 5

Joie Colère

Sous­titres animésIci, vous trouverez deux scènes du film Birdman (de Alejandro Iñarritu). Ces deux extraits ont été traduits grâce à des sous­titres animés (en langue originale). Il vous sera demandé de répondre ensuite à quelques petites questions. La première vidéo dure 45 secondes et la deuxième 90s.

13. Avez­vous vu le film auparavant?

Une seule réponse possible.

Si

Non

Autre :

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 7/10

14. Selon vous, quelle est l'émotion générale de

cet extrait?

15. Les sous­titres ont été animés sur base de la musique du film. D'après vous, aident­ils à

comprendre l'émotion de la scène?

Une seule réponse possible.

Oui

Non

Autre :

16. Trouvez­vous des difficultés à lire les sous­titres?

Plusieurs réponses possibles.

Oui

Non

Autre :

http://youtube.com/watch?v=zwiPVIjvFXg

132

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 8/10

17. Selon vous, quelle est l'émotion générale de

cet extrait?

18. Les sous­titres ont été animés sur base de la musique du film. D'aprés vous, aident­ils à

comprendre l'émotion de la scène?

Une seule réponse possible.

Oui

Non

Autre :

19. Trouvez­vous des difficultés à lire les sous­titres?

Plusieurs réponses possibles.

Oui

Non

Autre :

Informations généralesLe test est anonyme. Cependant pour pouvoir interpréter les résultats, ces quelques informations nous serviraient.

http://youtube.com/watch?v=up7i8HMabT4

133

05/04/2019 Typographie cinétique

https://docs.google.com/forms/d/14HtuXUIg5pfAeyTtyfGzE_HlVN1J-dXxaAzuXGyp8EM/edit 9/10

20. Vous êtes

Une seule réponse possible.

Un homme

Une femme

Je ne souhaite pas le préciser

21. Avez­vous déjà travaillé dans le domaine du graphisme?

Une seule réponse possible.

Oui

Non

22. Quel âge avez­vous?

23. Vous êtes...

Une seule réponse possible.

sourd

malentendant

Professionnel s'occupant de personnes sourdes et malentendantes

Autre :

24. À quel âge a‐t‐on découvert votre surdité?

Une seule réponse possible.

À la naissance

Autre :

25. En général vous êtes satisfait des sous­titres dans les films?

Une seule réponse possible.

0 1 2 3 4 5 6 7 8 9 10

non oui

26. En général,vous êtes satisfait du sous­titrage de la musique?

Une seule réponse possible.

0 1 2 3 4 5 6 7 8 9 10

non oui

27. Voulez­vous laisser un commentaire?

134