GAZE AND FACIAL EXPRESSIONS WHEN TALKING ABOUT ...

15

Psychologia, 2011, 54, 15–26

GAZE AND FACIAL EXPRESSIONS WHEN TALKING ABOUT

EMOTIONAL EPISODES

Mitsue NOMURA and Sakiko YOSHIKAWA

Kyoto University, Japan

This study investigated whether an emotional valence conveyed by verbal content

would influence the production of facial displays. Undergraduate students were

asked to describe three kinds of intense emotional episodes (involving joy, anger, or

sadness) to an experimenter (listener). Their facial displays were simultaneously

videotaped via a prompter. The results revealed that overall, speakers maintained

positive facial displays as they talked, regardless of the type of emotional episodes.

While they described a positive emotional episode (i.e., joy), their verbal protocols

were likely to co-occur with a gaze directed at the listener and a positive facial

expression. In contrast, while they described a negative emotional episodes (i.e.,

anger or sadness), their verbal protocols were less clearly associated with gaze and

facial expression. These findings implicate that during social interactions, gaze and

facial expression reflect both the emotional valence of a verbal message and the

attitudes toward a listener.

Key words: emotional episode, facial expression, gaze

Expressing and sharing emotions are important aspects of building and maintaining

good relationships with others. We often tell others about emotional episodes, even when

the event is not directly related to the listener (Rimé, Mesquita, Philippot, & Boca, 1991).

Empirical data have revealed that emotional sharing can create a bond between narrator

and audience and can generate a social triad of narrator, audience, and the target of the

emotional episode (Peters & Kashima, 2007). Given that emotional sharing creates and

changes our social relationships, it is important to elucidate how we communicate our

emotional experiences to others. This study investigated the role of gazes and facial

expressions during verbal communication.

Previous studies have identified a precise correlation between speech stream and

body motion. For example, Condon and Ogston (1967) found that the body parts of a

speaker, including head movement, mouth opening, gaze direction, etc., were

synchronized with the articulatory segmentation of his/her speech. Bavelas and Chovil

(2000) developed the “Integrated Message Model,” in which moment-by-moment audible

and visible communicative acts are treated as symbolic messages that are used to convey

meaning to others.

Numerous studies about emotional recognition have also documented interactions

between multi-modal expressions of emotion. For example, gaze perception is modulated

by emotional facial expressions of the target person (Lobmaier, Tiddeman, & Perrett,

Correspondence concerning this article should be addressed to Mitsue Nomura, Department of Cognitive

Psychology in Education, Graduate School of Education, Kyoto University, Yoshida-honmachi, Sakyo-ku,

Kyoto 606-8501, Japan (e-mail: [email protected]).

NOMURA & YOSHIKAWA16

2008). In addition, interpretation of facial expression is also influenced by the gaze

direction of the target person (Adams & Kleck, 2003). It is still unclear, however, how

these expressions work in conjunction with utterances. Another line of study has revealed

that humans use visual information such as mouth opening when perceiving audio

information (McGurk & MacDonald, 1976) and that this visual–audio interaction in

speech perception emerges at a very early developmental stage (Kushnerenko, Teinonen,

Volein, & Csibra, 2008). However, these studies did not examine how the semantic

dimension of verbal messages is recognized by non-verbal behaviors.

A few studies have focused on gazing behavior in relation to the emotional valence

of messages. Kimble and Olszewski (1980) used role-play, and asked participants to

memorize and repeat verbal messages for a video camera. The results showed that

regardless of the valence of the message (anger vs. liking), directed gaze increased when

participants intensified the valence of the message. Kimble, Forte, and Yoshikawa (1981)

replicated this effect using a person as a target. They suggested that gaze plays a role in

verbal communication by intensifying the emotional intensity of a verbal message

(intensifier hypothesis).

Taken together, these previous studies have shown that facial displays play an

important role in social interactions, but they leave several questions unanswered. First, it

is questionable whether Kimble et al.’s intensifier hypothesis can be applied to emotional

states other than liking or anger. Previous studies have indicated that both gazing behavior

and facial expressions have a common behavioral tendency toward the external stimuli

that evoked that emotion. For example, approach-oriented emotions (such as anger or

liking) tend to occur in conjunction with a directed gaze, whereas avoidance-oriented

emotions (such as embarrassment, sorrow, and disgust) tend to be expressed with an

averted gaze (see Argyle & Cook, 1976; Kleinke, 1986). Some empirical studies have

found that facial expressions such as anger or joy were recognized faster when the face

also had a directed gaze than when the face had an averted gaze. Conversely, facial

expressions of avoidance-oriented emotions (such as fear and sadness) tended to be

recognized faster when the face had an averted gaze (Adams & Kleck, 2003). Therefore,

Kimble’s results may not apply for avoidance-oriented emotion.

Second, it remains unclear whether the results of Kimble and Olszewski (1980)

could be replicated in a more natural setting for two reasons. First, the emotional

messages were prepared before testing began: participants read and memorized a prepared

text before talking. Although this manipulation was required to control the mental

processing of text information across participants, it might have expanded the discrepancy

between the experimental context and normal everyday communication. In daily life,

humans decide independently what to say when conveying emotions. Words are often

chosen immediately before they are spoken, and can arise spontaneously. In addition, it is

plausible that participants in a conversation are more involved in their own events than in

those of others, especially when talking about an emotional event. Second, participants in

a real-life conversation may suppress emotions during face-to-face interaction.

Expressing anger can be unhealthy physically and psychologically, and humans often hide

emotions for a prosocial function. For example, we may suppress anger with a friend to

FACIAL DISPLAYS AND EMOTIONAL EPISODES 17

preserve the relationship (Tavris, 1984). Previous findings have suggested that Asian

cultures in particular encourage individuals to refrain from expressing emotions directly

when such expression might hurt another (Wierzbicka, 1994). In laboratory settings,

individuals with Asian cultural values seem to automatically reduce their emotion-

expressive behavior (Butler, Lee, & Gross, 2007). With this in mind, it is questionable

whether Japanese subjects directly exhibit negative facial expressions even when they are

instructed to convey a negative emotion.

The present study examined how facial displays are configured when conveying an

emotional message. We decided to use an interview-based method rather than a role-

playing method. The interview method allowed participants to relate their own emotional

episodes so that they could intensify the episode in a natural way. In addition, we included

not only approach-related (anger and joy) but also avoidance-related emotion (sadness) to

clarify how verbal content and facial displays interact to convey an emotional experience1.

METHODS

Participants

A total of 39 Japanese undergraduate students (18 women and 21 men, mean age 19.4 years)

participated as speakers. All participants were unfamiliar to the listener (a female graduate student).

Apparatus

Figure 1 displays the experimental setup. In the experimental room, a speaker sat in a chair in front of

a prompter that enclosed a monitor and a two-way mirror. The video camera was hidden behind the monitor,

so the speaker’s face was videotaped without the speaker’s being overly aware of the camera. The speaker’s

facial images were transmitted to a second prompter, which was located in front of the listener. As the

speaker talked, the speaker and the listener viewed video images of each other’s face from the shoulders up.

Procedure

First, participants were instructed to talk to the listener about emotional episodes when they had

experienced intense anger, joy, and sadness. They were asked to relate the event in such a way as to convey

1 Negative and avoidance-related emotions include fear and disgust as well as sadness. Of these emotions,

we assumed that sadness was the most generally experienced and could be intense enough to discuss in

experimental settings.

Fig. 1. Apparatus and setup.


as clearly as possible the emotions they experienced at the time. They were also asked to describe a typical

school day. This request for a non-emotional story was inserted after the discussion of the three emotional

episodes to minimize the discomfort experienced by participants when talking about negative emotions. To

prevent the participants from pretending or exaggerating their facial expressions, we used a cover story. We

explained that the purpose of the study was to investigate prosodic features used when discussing emotional

episodes, and asked subjects if we could videotape their faces in case prosodic features could not be specified

through voice data alone. This cover story enabled us to videotape facial displays that were as natural as

possible, and also ensured that participants were aware they were being videotaped and that they granted

permission for videotaping.

Subjects were given sufficient time to choose and think about an emotional episode before relating it.

This manipulation was intended to reduce the cognitive load of speakers as they talked, as cognitive load is

well known to influence gazing behavior (e.g., Exline & Winters, 1965; Glenberg, Schroeder, & Robertson,

1998). Subjects were asked to recall and write a brief version of events from their past when they had

experienced intense anger, joy, and sadness, as well as a typical school day. For each emotional episode,

subjects were required to assess the intensity of the associated emotion on a 10-point scale from 1 (not at all

intense) to 10 (extremely intense).

Speaking began after the participant had prepared all three emotional episodes and filled out the

questionnaire. Just before they began to speak about each emotional episode, they were reminded to express

clearly what they had felt at the time. The order of the three emotional episodes was counterbalanced across

participants. After each episode was completed (or after four minutes), the listener wrapped up the talk and

moved on the next episode2.

After speaking about all four episodes, the subjects filled out a questionnaire to answer the following

statements: Did you feel at ease when talking, compared to how you usually feel? Did you fully express what

you meant to say? Did you get through to the listener? Did you feel comfortable? Did you feel as if you

were reliving the event? Did you worry about the video recording? They answered the questions on bipolar

scales ranging from very much (1) to not at all (7). Finally, participants were debriefed about the true purpose

of the study and thanked.

Analysis

After the video images were captured and transferred to a PC, each participant’s facial displays were

coded frame by frame (15 frames/sec) using Adobe Premiere Pro. Gaze directions were coded into four

categories: gaze directed at the listener’s face, gaze directed elsewhere, eyes closed, or eyes hidden. Before

participants began to speak about each episode, the experimenter confirmed that eye contact had been

established via the prompter. At that time, the participant’s gaze direction was classified as “gaze at the

listener’s face;” other gaze directions were categorized as “gaze elsewhere”. At that time, participants

touched their faces or hair while talking. Frames in which participants hid their eyes behind a raised hand

were classified as “eyes hidden.”

The Facial Action Coding System (FACS: Ekman, Friesen, & Hager, 2002) was applied to evaluate

facial expressions. Three kinds of action units (AU) were evaluated, corresponding to the three emotional

valences: cheek lifting (AU6), which is typical of happy expressions; brow lowering (AU4), which is thought

to be prototypical of angry expressions; and inner brow raiser (AU1), which is thought to be typical of sad

expressions. When one of these categories of movement was observed, its intensity was also assessed and

classified as slight or intense. The criteria for differentiating between these two levels corresponded to those

set out in FACS (2002): “a trace” and “more than a trace,” respectively.

Coding was initially conducted by the first author and was confirmed by a student unfamiliar with the

goals of the study, who coded frames for each emotion condition from a random sampling of video data of the

six speakers. The coder was trained by using written descriptions of the expression categories and pictures of

prototypical facial changes (Ekman et al., 2002). The average Cohen’s kappa was 0.81 for gaze direction and

0.64, 0.78, and 0.76 for AU6, AU4, and AU1, respectively.

2 During this talking phase, the listener simultaneously reacted to the speaker. This manipulation allowed

us to examine natural and realistic facial displays, similar to those displayed during daily communication.

Some might argue that the listener should have maintained a neutral expression to avoid confounding

behavior, but we were concerned that the speaker would not be able to speak smoothly if the listener

completely restricted her behavior.


RESULTS

We investigated the kind of experiences participants spoke about, their intensity

ratings, their gazing behavior and facial expressions, and the interactions among these

measures. We also noted correlations between subjective feelings and facial displays.

Types of Emotional Episodes

Episodes described by participants varied greatly: anger episodes included

unreasonable actions by others, unfair advantages for others, a feeling of inability, etc.

Typical joyful episodes included successful entrance exams or other goal achievements,

romantic feelings, and reminiscences about the past. Sad episodes included death of

grandparents or pets, separations from significant persons, heartbreak, personal failure,

etc.

Intensity of Emotional Experience

Figure 2 presents the subjective ratings of emotional intensity for each type of

emotional episode.

A one-way ANOVA revealed a main effect of emotion (F(2, 58) = 13.3, p < .001);

ratings for episodes of anger were significantly lower than those for joy or sadness. This

is partly because nine participants rated at least one of the three emotional episodes as “5”

or lower, even though the experimenter had requested that participants talk about an

experience during which they had a strong emotion. Therefore, we excluded data from

these nine participants from further analyses.

Selection of Emphatic Phase

The emotional episodes included stories about how the emotion arose as well as a

description of the emotion itself. This study focused on verbal content with a highly

emotional valence. Therefore, we divided each episode into two parts: the Emphatic

phase and the Baseline phase. To identify the Emphatic phase, in which participants place

the greatest verbal stress on the explanation of their emotions, each episode was

transcribed based on voice data from the videos of each speaker. For the three types of

emotional episodes, four raters (graduate students who were not aware of the purpose of

the study) selected the part they considered to be the most emphatic. For each episode, the

portion of the video data that all raters agreed was the most emphatic was classified as the

Emphatic phrase of that episode. The rest of the episode (after the Emphatic phase was

removed) was defined as the Baseline phase of that episode.

The Emphatic phase lasted for approximately 5% of all episodes (Me = 62.5, 51.50,

and 61.5 frames for anger, joy, and sad episodes respectively). A one-way ANOVA

revealed no significant differences among three types of emotional episodes (F(2, 58) =

1.11, p > .10).

Gazing Behavior

First, we calculated the proportion of time each participant spent looking at the


listener during each episode. The arcsine-transformed proportion values were subjected to

a repeated measure ANOVA with episodes (four levels) as within-subjects factors. This

resulted in a significant effect (F(3, 87) = 6.33, p < .001). Post-hoc comparisons using

Ryan’s method revealed significant differences only between joy and the other three

conditions (ts > 2.15, ps < .05), whereas differences between other pairs were not

significant (ts < 1.56, ps > .10).

Next, we calculated the proportion of time each participant spent looking at the

listener during the Emphatic and Baseline phases of each episode (Figure 3). We

conducted a 3 (emotional valence of episode; anger, joy, and sadness)×2 (phase; Emphatic

vs. Baseline) repeated-measures ANOVA on the arcsine-transformed proportion values,

which revealed that emotional valence had a significant main effect (F(2, 58) = 7.50,

p < .005), indicating that speakers looked at the listener more when speaking about a

joyful episode than about an angry or sad episode (ts > 2.10, ps < .05). Differences

between the angry and sad conditions were not significant (t(58) = 1.77, p > .08). Phase

also had a significant main effect (F(1, 29) = 15.35, p = .001), indicating that speakers

looked at the listener more during the Emphatic phase than during the Baseline phase.

The interaction between emotional valence and phase was not significant (F(2, 58) = 2.01,

p > .10).

To examine these data more closely, we compared differences in the proportions of

time spent looking at the listener between the first and second halves of the Emphatic

phase. A 3 (emotional valence; anger, joy, and sadness)×2 (part; first vs. second)

repeated-measures ANOVA on the arcsine-transformed proportion values revealed that

part had a significant main effect (F(1, 29) = 30.98, p < .05), suggesting the speakers

looked at the listener more during the second part of the Emphatic phase. The main effect

of emotional valence was also significant (F(2, 58) = 5.41, p < .05).

Finally, we calculated the average number of gaze shifts (looking away from the

listener or looking back to the listener) per frame within the Emphatic phase (Table 1).

These data did not differ by the emotional valence of episodes (F(2, 89) = 0.70, p > .10).

Fig. 2. Mean intensity ratings for each emotional

episode.

Fig. 3. Mean proportion of directed gaze as a

function of episode phase.


Facial expressions

Table 2 lists the number of participants who exhibited each facial expression, i.e.,

AU4 (brow lowering), AU6 (cheek lifting), and AU1 (inner-brow raising), at least once

during the Emphatic phase. Even after the Emphatic phase was identified, patterns of

facial expressions in the Emphatic phase were similar to those in entire episodes;

regardless of the emotional valence, negative expressions (AU1 or AU4) were observed

rarely, whereas positive expressions (AU6) were frequently observed. In the Emphatic

phase dataset, no cases of AU1 were observed, and cases of AU4 were observed in only 11

frames. Therefore, we conducted a further analysis of AU6.

AU6 was observed in almost all frames (mean ratio = 0.97, 1.00, and 0.92 for

episodes of anger, joy, and sadness, respectively). We calculated the observed proportion

of intensified AU6 by dividing the number of all frames from each phase (Figure 4).

Arcsine-transformed proportion values were subjected to a 3 (emotional valence; anger,

joy, and sadness)×2 (phase; Emphatic vs. Baseline) repeated-measures ANOVA. The

emotional valence of episodes had a significant main effect (F(2, 58) = 8.84, p < .001),

suggesting that AU6 was intensified when subjects spoke about a joyful episode. Phase

also had a significant main effect (F(1, 29) = 8.02, p < .05), suggesting that AU6 was

intensified during the Emphatic phase. The interaction did not reach a level of

significance (F(2, 58) = 2.59, p = .08).

Table 1. Mean Number (SD) of Gaze Shift (DG → nonDG,

or nonDG → DG) per Sec During Emphatic Phase.

Anger Joy Sadness

0.87 (0.83) 1.06 (0.95) 0.88 (0.89)

Table 2. Number of Participants Who Showed Each Facial Expression at Least Once

During Emphatic Phase.

emotional episode

Anger Joy Sadness

AU4 intense 0 0 0

slight 1 0 0

AU6 intense 10 18 8

slight 30 30 30

AU1 intense 0 0 0

slight 0 0 0

max = 30


Gaze and Facial Expressions

We developed an index to determine whether the co-occurrence of gazing behavior

and facial expressions of emotion varied by the emotional valence of the verbal content.

The proportion of co-occurrence of directed gaze and intensified AU6 was defined as the

number of frames in which we observed both a gaze directed toward the listener and

intensified AU6; this was then divided by the number of frames in which we observed

intensified AU6 (Fig. 5). Figure 5a shows that AU6 tended to be accompanied by a

directed gaze. This tendency was more obvious during the Emphatic phase (Fig. 5a), and

the connection between gaze and facial expressions was more evident when AU6 was

intensified (Fig. 5b). The tendency did not occur when the participants talked about sad

emotional episodes, in contrast to episodes of joy or anger. Instances of AU4 during sad

emotional episodes were accompanied by an averted gaze or closed eyes. Due to the small

number of these frames (see also Table 2), it was not possible to test these associations

statistically.

Table 3 shows the relationship between gaze shifting and occurrences of intensified

AU6. When subjects talked about an angry or joyful emotional episode, they tended to

look at the listener first and then smile (i.e., intensified AU6). They also tended to

Fig. 4. Mean proportion of intensified AU6 as a function of episode phase.

Fig. 5. Mean proportion of intensified AU6 with directed gaze.


continue looking at the listener while exhibiting intensified AU6. However, these

correlations did not occur as subjects talked about sad episodes.

Correlation Between Subjective Feelings and Facial Displays

Figure 6 shows that participants’ subjective evaluations of their feelings as they

spoke differed depending on the emotional valence of the episode. For example, subjects

rated the ease of speaking and the level of comfort as significantly higher for joyful

episodes than for sad or angry episodes. They rated their ability to get through to the

listener and their ability to relive the event as higher for joyful and sad episodes than for

angry episodes (all Fs > 3.50, ps < .05). The other items did not reveal any significant

differences in rating scores as a function of the emotional valence of the episode, but it

should be noted that the participants did not report feeling uncomfortable being videotaped

(mean rating scores were 2.0, 1.8, and 2.2 for angry, joyful, and sad episodes respectively).

Table 3. Temporal Relationship Between Change of Facial Expression and

Gaze Shift During Emphatic Phase

Anger Joy Sadness

DG → AU6i 12 15 1

AU6i → DG 2 6 5

AU6i without DG 3 2 5

number of observed events

AU6i = AU6(intense)

DG = Directed gaze toward the listener

Fig. 6. Participants’ subjective ratings of their speaking experience.


To analyze whether subjective evaluations about feelings were correlated with

behavioral measures, we computed correlations between each rating (range 1–7) and three

kinds of behavioral measures: the proportion of observed directed gaze, the proportion of

intensified AU6, and the proportion of co-observed directed gaze and intensified AU6

during the Emphatic phase. The results did not reveal any particularly strong correlations

(rs < .48), except that the proportion of co-observed directed gaze and intensified AU6

were positively correlated with the ease in speaking (r = .662) and the level of comfort

(r = .703) when participants related sad episodes.

DISCUSSION

This study investigated whether the emotional valence conveyed by the verbal

content would influence the production of facial displays. Participants described three

different kinds of personal emotional episodes and tried to convey an intense joyful, angry,

or sad emotion. We recorded their facial expressions and any gazes directed at the

listener, and we evaluated these with respect to emotional condition.

The results revealed that gazes and facial expressions varied depending on speakers’

intention to intensify the emotion as they related an emotional episode. For all three

episodes, speakers were more likely to look at the listener during the Emphatic phase than

the Baseline phase. We found that facial expressions also varied depending on the

intention to emphasize an emotion; the number of frames with intensified AU6 increased

when speakers emphasized their emotions verbally.

This pattern of gazing behavior supports previous reports of gazing behavior during

role-play experiments (Kimble & Olszewski, 1980; Kimble, Yoshikawa, & Zehr, 1981).

Kimble and his colleagues suggested that a directed gaze of a sender to a recipient can

function to intensify an emotional message, but their study addressed only approach-

related emotions, such as anger and joy. We extended their research by including

avoidance-related emotions (i.e., sadness) and by incorporating a more natural

communication setting.

Previous studies focusing on gazing behavior during dialogue have reported that

speakers tend to look at the listener when they want to request and monitor a response

from the recipient (Beattie, 1978; Argyle & Cook, 1976). Considering these previous

findings, our result can be interpreted as speakers’ monitoring responses or requesting

positive reactions by the recipient. Our data, which reveal intensified AU6, support this

‘requesting’ theory, as positive facial expressions may evoke the same positive expression

on a perceiver’s face (Dimberg, Thunberg, & Elmehed, 2000). However, intensifying

messages and checking for responses from recipients could be mental processes that are

naturally highly correlated or difficult to separate.

In addition to the results that support the ‘intensifier hypothesis,’ our findings

demonstrate that the type of emotional episode influenced speakers’ facial expressions

3 N = 8 (see Table 2).


and the proportion of their gaze that was directed toward the listener. They tended to look

at the listener more when talking about a joyful episode compared to when relating a sad

or angry emotional event. This difference in gazing behavior between positive (i.e.,

joyful) and negative (i.e., angry and sad) emotional episodes was not restricted to the

Emphatic or Baseline phase. With regard to facial expressions, the proportion of

intensified AU6 also differed between positive (i.e., joyful) and negative (i.e., angry and

sad) emotional episodes. In contrast, AU4 and AU1 were not observed even when

speakers described angry or sad emotional episodes.

These findings seem to contradict Kimble and Olszewski (1980), who reported that

negative emotional messages were accompanied by more directed gazing than were

positive emotional messages. It is possible, however, that these findings were influenced

by the fact that Kimble and his colleagues used a camera or a target person who never

showed any response. In our study, we instructed subjects to convey their emotions

clearly, which might have facilitated the regulation of emotional expression according to

the listener’s response. Friesen (1972) reported that Japanese students were more likely

than American students to suppress negative expressions when a stressful film was shown

in the presence of the interviewer, even though both groups of university students were

equally likely to show negative expressions when they watched the film alone. Therefore,

Japanese participants may be more influenced by social context.

Taken together, our findings indicate that when we communicate with others in face-

to-face situations, our facial displays can send multi-layered messages. For example, one

message is closely related to the verbal emotional content: the speaker’s intention to

intensify a verbal emotional message. In 1969, Ekman and Friesen termed this an

‘illustrator’ effect, and Eibl-Eibesfeldt (1989, 2001) expanded on their research by

showing that speakers tend to raise and hold their eyebrows to emphasize a sentence.

However, to date, very little empirical evidence about the illustrator effect has been

available. Further research will be required to obtain quantitative evidence about specific

relationships and interactions between facial displays and verbal content.

Facial displays can also convey a meta-message, including messages that

communication is going well, expressions of politeness toward another, and positive

expressions invoked by enjoying the dialogue. It is possible that this meta-message might

overshadow or weaken the facial expressions related to negative emotions, such as AU4 or

AU1. It is also possible, however, that the negative expression communicated by verbal

content may be reflected in the relationships between facial expressions and gaze

direction. Our experiments did not investigate how temporal relationships or co-

occurrence rates between facial expressions and gaze direction modulate the intensity of

emotional meanings, so future studies should address this issue.

Finally, it should be noted that speakers might have applied differing strategies to

intensify their emotions as they related the episodes. For example, some speakers may

have tried to re-experience the emotional event and to express in words how they were

feeling; this strategy would likely cause speakers to smile as they described a joyful

episode. Other speakers might have described the story behind the emotion, trying to help

the listener to imagine the situation that evoked the intense emotion; this strategy might


cause speakers to have a more serious facial expression when describing a joyful episode.

More precise investigation of the relationships between a verbal message and facial

displays could help clarify human interpersonal relationships and how we communicate.

REFERENCES

Adams, R. B., Jr., & Kleck, R. E. 2003. Perceived gaze direction and the processing of facial displays of

emotion. Psychological Science, 14, 644–647.

Argyle, M., & Cook, M. 1976. Gaze and mutual gaze. Cambridge University Press.

Bavelas, J. B., & Chovil, N. 2000. Visible acts of meaning. An integrated message model of language in

face-to-face dialogue. Journal of Language and Social Psychology, 19, 163–194.

Beattie. G. W. 1978. Floor apportionment and gaze in conversational dyads. British Journal of Social and

Clinical Psychology, 17, 7–15.

Butler, E. A., Lee, T. L., & Gross, J. J. 2007. Emotion regulation and culture: Are the social consequences of

emotion suppression culture-specific? Emotion, 7, 30–48.

Condon, W.S. & Ogston, W.D. 1967. A segmentation of behavior. Journal of Psychiatric Research, 5, 221–

235.

Dimberg, U., Thunberg, M., & Elmehed, K. 2000. Unconscious facial reactions to emotional facial

expressions. Psychological Science, 11, 86–89.

Eibl-Eibesfeldt, I. 1989. Die Biologie des menschlichen Verhaltens: Grundriβ der Humanethologie [Human

ethology: ningenkoudou no seibutsu gaku, Minerva shobou, 2001. (Japanese)]

Ekman, P., & Friesen, W. V. 1969. The repertoire of nonverbal behavior: Categories, origins, usage, and

coding. Semiotica, 1, 49–98.

Ekman, P., Friesen, W. V., & Hager, J. C. 2002. Facial action coding system. Salt Lake City, Utah: Research

Nexus.

Exline, R. V., & Winters, L. C. 1965. Affective relations and mutual glances in dyads. In S. S. Tomkins & C.

E. Izard (Eds.), Affect, cognition and personality (pp. 319–350). New York: Springer.

Friesen, W. V. 1972. Cultural differences in facial expressions in a social situation: An experimental test of

the concept of display rules. Unpublished doctoral dissertation, University of California, San

Francisco.

Glenberg, A. M., Schroeder, J. L., & Robertson, D. A. 1998. Averting the gaze disengages the environment

and facilitates remembering. Memory & Cognition, 26, 651–658.

Kimble, C. E., Forte, R. A., & Yoshikawa, J. C. 1981. Nonverbal concomitants of enacted emotional

intensity and positivity: Visual and vocal behavior. Journal of Personality, 49, 271–283.

Kimble, C. E., & Olszewski, D. A. 1980. Gaze and emotional expression: The effects of message positivity–

negativity and emotional intensity. Journal of Research in Personality, 14, 60–69.

Kimble, C. E., Yoshikawa, J. C., & Zehr, H. D. 1981. Vocal and verbal assertiveness in same-sex and mixed-

sex groups. Journal of Personality and Social Psychology, 40, 1047–1054.

Kleinke, C. L. 1986. Gaze and eye contact: A research review. Psychological Bulletin, 100, 78–100.

Kushnerenko, E., Teinonen, T., Volein, A., & Csibra, G. 2008. Electrophysiological evidence of illusory

audiovisual speech integration in human infants. Proceedings of the National Academy of Sciences of

the United States of America, 105, 11442–11445.

Lobmaier, J. S., Tiddeman, B. P., & Perrett, D. I. 2008. Emotional expression modulates perceived gaze

direction. Emotion, 8, 573–577.

McGurk, H., & MacDonald, J. 1976. Hearing lips and seeing voices. Nature, 264, 746–748.

Peters, K., & Kashima, Y. 2007. From social talk to social action: Shaping the social triad with emotion

sharing. Journal of Personality and Social Psychology, 93, 780–797.

Rimé, B., Mesquita, B., Philippot, P., & Boca, S. 1991. Beyond the emotional event: Six studies on the social

sharing of emotion. Cognition and Emotion, 5, 435–465.

Tavris, C. 1984. On the wisdom of counting to ten: Personal and social dangers of anger expression. Review

of Personality & Social Psychology, 5, 170–191.

Wierzbicka, A. 1994. Emotion, language, and cultural scripts. In S. Kitayama & H. R. Markus (Eds.),

Emotion and culture (pp. 133–196). Washington, DC: American Psychological Association.

(Manuscript received January 14, 2010; Revision accepted July 12, 2010)

GAZE AND FACIAL EXPRESSIONS WHEN TALKING ABOUT ...

Documents

Transcript of GAZE AND FACIAL EXPRESSIONS WHEN TALKING ABOUT ...