The differential contribution of facial expressions, prosody, and speech content to empathy

This article was downloaded by: [Christina Regenbogen]On: 03 January 2012, At: 13:10Publisher: Psychology PressInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office:Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Cognition & EmotionPublication details, including instructions for authors and subscriptioninformation:http://www.tandfonline.com/loi/pcem20

The differential contribution of facialexpressions, prosody, and speech content toempathyChristina Regenbogen a b , Daniel A. Schneider a b , Andreas Finkelmeyer bc , Nils Kohn a b , Birgit Derntl a b d , Thilo Kellermann a b , Raquel E. Gur e ,Frank Schneider a b e & Ute Habel a ba Department of Psychiatry, Psychotherapy, and Psychosomatics, RWTHAachen University, Aachen, Germanyb JARA Translational Brain Medicine, Aachen, Germanyc Institute of Neuroscience, Newcastle University, Newcastle Upon Tyne,England, UKd Institute for Clinical, Biological, and Differential Psychology, University ofVienna, Vienna, Austriae Department of Psychiatry, University of Pennsylvania, Philadelphia, PA,USA

Available online: 03 Jan 2012

To cite this article: Christina Regenbogen, Daniel A. Schneider, Andreas Finkelmeyer, Nils Kohn,Birgit Derntl, Thilo Kellermann, Raquel E. Gur, Frank Schneider & Ute Habel (2012): The differentialcontribution of facial expressions, prosody, and speech content to empathy, Cognition & Emotion,DOI:10.1080/02699931.2011.631296

To link to this article: http://dx.doi.org/10.1080/02699931.2011.631296

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantialor systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, ordistribution in any form to anyone is expressly forbidden.

http://www.tandfonline.com/loi/pcem20

http://dx.doi.org/10.1080/02699931.2011.631296

http://www.tandfonline.com/page/terms-and-conditions

The publisher does not give any warranty express or implied or make any representation that thecontents will be complete or accurate or up to date. The accuracy of any instructions, formulae,and drug doses should be independently verified with primary sources. The publisher shall notbe liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever orhowsoever caused arising directly or indirectly in connection with or arising out of the use of thismaterial.

Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

The differential contribution of facial expressions,prosody, and speech content to empathy

Christina Regenbogen1,2, Daniel A. Schneider1,2, Andreas Finkelmeyer2,3,Nils Kohn1,2, Birgit Derntl1,2,4, Thilo Kellermann1,2, Raquel E. Gur5,Frank Schneider1,2,5, and Ute Habel1,2

1Department of Psychiatry, Psychotherapy, and Psychosomatics, RWTH Aachen University,Aachen, Germany2JARA Translational Brain Medicine, Aachen, Germany3Institute of Neuroscience, Newcastle University, Newcastle Upon Tyne, England, UK4Institute for Clinical, Biological, and Differential Psychology, University of Vienna, Vienna, Austria5Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA

Background: Facial expressions, prosody, and speech content constitute channels by whichinformation is exchanged. Little is known about the simultaneous and differential contribution ofthese channels to empathy when they provide emotionality or neutrality. Especially neutralisedspeech content has gained little attention with regards to influencing the perception of otheremotional cues. Methods: Participants were presented with video clips of actors telling short-stories.One condition conveyed emotionality in all channels while the other conditions either providedneutral speech content, facial expression, or prosody, respectively. Participants judged the emotionand intensity presented, as well as their own emotional state and intensity. Skin conductance servedas a physiological measure of emotional reactivity. Results: Neutralising channels significantlyreduced empathic responses. Electrodermal recordings confirmed these findings. The differentialeffect of the communication channels on empathy prerequisites was that target emotion recognitionof the other decreased mostly when the face was neutral, whereas decreased emotional responsesattributed to the target emotion were especially present in neutral speech. Conclusion: Multichannelintegration supports conscious and autonomous measures of empathy and emotional reactivity.Emotional facial expressions influence emotion recognition, whereas speech content is important forresponding with an adequate own emotional state, possibly reflecting contextual emotion-appraisal.

Keywords: Empathy; Facial expressions; Prosody; Speech content; Galvanic skin conductance.

Correspondence should be addressed to: Christina Regenbogen, Department of Psychiatry, Psychotherapy, and Psychoso-

matics, Medical School, RWTH Aachen University, Pauwelsstrasse 30, D-52074 Aachen, Germany. E-mail: cregenbogen@

ukaachen.de

This work was supported by the Interdisciplinary Centre for Clinical Research of the Medical Faculty of the RWTH Aachen

University (IZKF, N4-4, N2-6) and the International Research Training Group (IRTG1328) of the German Research Foundation

(DFG), as well as DFG�KFO 112 and HA 3202/7-1.

We are very grateful to Jonas Albers, Anna Bartsch, Jorn Kramer, Carla Schirk, and Timur K. Toygar for their assistance and to

all our participants.

COGNITION AND EMOTION

2012, 1�20, iFirst

1# 2012, The Authors

http://www.psypress.com/cogemotion http://dx.doi.org/10.1080/02699931.2011.631296

Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

http://www.psypress.com/cogemotion

http://dx.doi.org/10.1080/02699931.2011.631296

Social cognition or ‘‘knowledge of the mind ofothers’’ (Adolphs, 2002, p. 2) has been a field ofinterest for a few decades (Penn, Sanna, &Roberts, 2008). Recent studies have investigatedsocially relevant processes such as empathy, attri-butional style, or agency judgements. In a socialcontext, emotional information is conveyedthrough the simultaneous use of multiple com-munication channels. Facial expressions, prosody,and speech content transfer emotional informa-tion, among other cues (e.g., body posture orgestures). This facilitates understanding of theother person and supports a shared and paralleledemotional experience of the other, so-calledempathy, which in turn promotes prosocial beha-viour. Being a social phenomenon, empathyincludes the presence of more than one individualand the distinction into ‘‘self’’ and ‘‘other’’ regard-ing the emotional experience. These two criteriareflect a common denominator in the heteroge-neous definitions and experimental implementa-tions of empathy (see Batson, 2009; de Waal,2008; Decety & Jackson, 2004).

In an experimental setting, empathy is difficultto assess. Using static or dynamic displays of facialexpressions, researchers have experimentally tar-geted one empathy prerequisite: emotion recogni-tion (Ekman & Friesen, 1976; Kilts, Egan,Gideon, Ely, & Hoffman, 2003; LaBar, Crupain,Voyvodic, & McCarthy, 2003; Sato, Kochiyama,Yoshikawa, Naito, & Matsumura, 2004; Traut-mann, Fehr, & Herrmann, 2009). Althoughexperiments using static emotional facial expres-sions have established a valuable foundation andhave shed light on the neural basis of theprocessing of emotional faces, they lack ecologicalvalidity (see Adolphs, 2002, for a review). Dy-namic stimuli, on the other hand, show advan-tages regarding recognition accuracy (Ambadar,Schooler, & Cohn, 2005), intensity and realismratings (Simons, Detender, Roedema, & Reiss,1999; Weyers, Muhlberger, Hefele, & Pauli,2006), as well as neural activation patterns(Trautmann et al., 2009). Numerous studieshave investigated the effects of emotional prosody,either by itself (Bostanov & Kotchoubey, 2004;Ethofer, Van De Ville, Scherer, & Vuilleumier,

2009; Hammerschmidt & Jurgens, 2007; Kotzet al., 2003; Paulmann & Kotz, 2008; Thonnessenet al., 2010; see Cutler, Dahan, & van Donselaar,1997; Friederici, 2002; Wildgruber, Ackermann,Kreifelt, & Ethofer, 2006, for reviews) or withrespect to the integration of multiple channels (deGelder & Vroomen, 2000; Massaro & Egan,1996; Nygaard & Queen, 2008; Stevenson, Van-DerKlok, Pisoni, & James, 2011; Werner &Noppeney, 2010; see Campanella & Belin, 2007;Koelewijn, Bronkhorst, & Theeuwes, 2010, forreviews). The involvement of multiple modalitiesnot only improves perceptual mechanisms on abasic sensory level (Collignon et al., 2008; Zee-lenberg & Bocanegra, 2010), but also involvesqualitatively different processes, as indicated byfunctionally different brain regions involved inmultimodal compared to unimodal processing(Kreifelts, Ethofer, Grodd, Erb, & Wildgruber,2007; Peelen, Atkinson, & Vuilleumier, 2010;Seubert et al., 2010).

When simultaneously presented with multi-modal signals, the processing of these is notseparate. Auditory and visual information, forexample, interact at early temporal stages (e.g.,Kreifelts et al., 2007) when presented simulta-neously. The integration of prosodic and semanticinformation (Friederici & Alter, 2004) involves adynamic interplay between right and left hemi-spheric processes. More and more studies haveinvestigated the interplay between prosody andsyntax on a sentence level (e.g., Humphries, Love,Swinney, & Hickok, 2005) where it has beenshown earlier that prosody influences morphosyn-tactic information and causes performance differ-ences in accuracy of semantic judgements andreaction times (Blasko & Hall, 1998).

In empathy, there are processes beyond recog-nising someone’s emotional expression that needto be taken into account. According to deVignemont and Singer (2006; see also Derntlet al., 2010; Singer & Lamm, 2009), this includesthe sharing of the emotion of the other person(see Batson, 2009, for the differentiation betweenempathy and sympathy; de Vignemont & Singer,2006) to some degree which includes perspective-taking processes (Derntl et al., 2010; Kramer,

REGENBOGEN ET AL.

2 COGNITION AND EMOTION, 2012, iFirst

Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

Mohammadi, Donamayor, Samii, & Munte,2010). Subsequent adequate responses are sup-ported by a contextual appraisal process (deVignemont & Singer, 2006), which can occur atthe same time at which emotional cues arepresented (early appraisal model), or at a laterstage, modulating an earlier automatically elicitedresponse to the emotional cue (late appraisalmodel). According to the appraisal models, emo-tional cues profit from a contextual embedding sothat they can be interpreted correctly and justifyempathy by the receiver.

To our knowledge, one aspect that has notbeen systematically investigated is the integrationof three simultaneously presented channels, aswell as their distinct contributions to differentemotional parameters when manipulating thefactor ‘‘emotionality’’. Especially regarding empa-thy, it is particularly unclear whether there arenoticeable differences in the impact of thesechannels on empathy.

To investigate multimodal effects as wellas distinct contributions of facial expressions,prosody, and speech content on empathy and itsprerequisites dynamic stimulus material was de-

veloped. Participants watched video clips ofactors who conveyed emotional information viaall three channels, and video clips in which oneof the three channels was held neutral (Figure 1).They judged the emotional state and emotionalintensity of the actor, as well as their ownemotional state as a response to the video clip.If participants attributed the same emotionalstate to themselves as to the recognised emotionalstate of the actor and if both matched thedisplayed target emotion this was defined as anempathic response. We further assessed intensitylevels as subjective emotional indicators, andmeasures of galvanic skin responses (GSR) aspsychophysiological arousal indicators.

We hypothesised that stimuli conveying con-sistent multichannel emotional information wouldbe responded to with more empathy than stimuliwith one neutral channel, due to multichannelintegration and facilitation through emotionalcongruence (Seubert et al., 2010) and convergingcontextual appraisal. We expected decreased em-pathy rates for items with a neutral channel,especially for stimuli with a neutral face or prosodyas they have been shown to be very powerful cues

Figure 1. Upper: Stimuli existed in four conditions. In ‘‘all emotional’’, facial expression, prosody and speech content are emotional (grey).

In ‘‘neutral prosody’’, ‘‘neutral face’’, ‘‘neutral speech content’’ one communication channel was each held neutral (white). Lower: Behavioural

variables.

FACIAL EXPRESSIONS, PROSODY, SPEECH CONTENT AND EMPATHY

COGNITION AND EMOTION, 2012, iFirst 3

Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

of emotionality. We further hypothesised higheremotion intensity and sympathetic arousal levels inour participants when all channels conveyed con-gruent emotional information.

Studies investigating sex differences regardingemotion related parameters reported heteroge-neous results. They range from stating females asthe ‘‘more emotional’’ sex outperforming males ondifferent behavioural and psychophysiologicalemotion measures (Diekman & Eagly, 2000;Hall et al., 2004; Kring & Gordon, 1998) tofindings that report no differences on psychophy-siological measures (e.g., Roman, Garcia-San-chez, Martinez-Selva, Gomez-Amor, & Carrillo,1989) or behavioural performance levels (Derntl,Habel et al., 2009; Habel et al., 2007). Based onthese results (see also Koch et al., 2007) wehypothesised no behavioural sex differences.

METHODS

Dynamic naturalistic video clips

The stimulus set consisted of 64 short standar-dised video clips (duration M�11.8 s, SD�1 s)displaying male and female actors (29 clips of fivedifferent males, 35 clips of 7 different females),telling a self-related story. Short stories withdisgusted, fearful, happy, sad, or neutral contentwere used. The emotional load of the clips was notevident in the first 3 s of the video clips, butunfolded over the course of the clip and reachedits apex towards the end.

We created four different conditions (seeFigure 1) regarding the emotionality of eachcommunication channel (facial expression, pro-sody, speech content). The condition ‘‘all emo-tional’’ consisted of the actor telling a self-relevantemotional story with accompanying emotionalfacial expression and prosody. In the conditions‘‘neutral face’’, ‘‘neutral prosody’’, and ‘‘neutralspeech content’’ the corresponding channel lackedemotionality while the two remaining channelsstill transferred the target emotion. This specificpresentation (two channels emotional, oneneutral) was chosen in order to (1) retain a relative

naturalness, and (2) allow detection of theemotion present in the majority of channels. Aswe were interested in differential effects ofchannels and their joint appearance in a multi-modal presentation, we did not completely elim-inate the third channel but kept it neutral.

For ‘‘neutral speech’’, this included a neutralstory accompanied by emotional facial expressionsand congruent prosody (e.g., smiling and talkingjoyfully while reporting something neutral).‘‘Neutral face’’ was created by originally filmingan actor telling an emotional story with neutralfacial expression and prosody and then dubbingthe original voice stream with the same actortelling the same story lip-synchronously to hisclip, but with emotional prosody. For ‘‘neutralprosody’’, the actor was filmed while displayingemotionality in all three channels, and these clipswere then dubbed with a neutral voice stream(sample clips are available, further description ofstimulus creation and validation can be found inthe online supplement).

Participants

Forty healthy native German speakers (20 fe-males, age M�32.18, SD�9.84 yrs, range 19�52) were recruited through local advertisementsand participated in the study. They were right-handed according to the Edinburgh HandednessInventory (Oldfield, 1971), and did not report anyhistory of psychiatric, neurological, or cardiovas-cular illness. All participants had normal orcorrected-to-normal vision. They were screenedfor alexithymia using the 20-item Toronto Alex-ithymia Scale (TAS-20; Bagby, Taylor, & Ryan,1986). The TAS-20 measures difficulties inidentifying and describing emotions as well as anexternal focus of attention and minimisation ofemotional experience using three subscales: ‘‘diffi-culty describing feelings’’ (DDF); ‘‘difficulty iden-tifying feelings’’ (DIF); and ‘‘externally orientedthinking’’ (EOT). The internal consistency mea-sure in the present sample (Cronbach’s a, seeTable 1) was comparable to previously reportedconsistency values (Parker, Taylor, & Bagby,

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

2003). All participants gave written informedconsent to the study protocol. The study wasapproved by the local ethics committee. One malewas excluded because of Klinefelter Syndrome andone female was excluded because of a TAS-valueof 61 (cut-off at 60), which resulted in a finalsample of N�38.

Data acquisition

After task instruction and presentation of threeexemplary video clips, we presented participantswith 64 clips in pseudo-random order (two orders,counter-balanced), interrupted by three shortbreaks. Participants were seated 0.7 m awayfrom a laptop computer screen (IBM,ThinkPad†; 15.6??, 800�600 pixels) and wereequipped with headphones. The participant’s lefthand was comfortably placed on a towel withpalms facing upward. We asked participants tomove their hand as little as possible during thewhole experiment. In order for the physiologicalmeasurements to reach baseline level, we startedGSR measurements�10 minutes prior to theactual start of the experiment. One investigatorstayed in the room with the participant and satoutside the participants’ field-of-view.

The instruction was to engage in the commu-nication situation by imagining the presentedactor as a personally familiar communicationpartner telling a self-related, short story. We

avoided any suggestion of pretending to be theperson or of deliberately sharing the emotionalstate (see Farrow et al., 2001, for comparison withtask instructions in empathy studies). After eachclip, the participants rated their emotional state asa response to the video clip (‘‘self’’) and they ratedthe emotion of the other person (‘‘other’’) bychoosing from four emotion categories (happi-ness, sadness, fear, and disgust) and neutral. Theyalso gave separate ratings of intensity levels ofthese emotions (self and other) as well asnaturality ratings on the video clip (scales 1�6).Participants were told to focus their eyes on awhite fixation cross set against a black backgroundbetween rating task and the next video clip(duration: 7 s).

Galvanic skin conductance was continuouslymeasured with two flat reversible silver/silverchloride (Ag�AgCl) electrodes (Ø 10 mm).Electrodes were attached to the thenar andhypothenar eminence on the left hand palmsand filled with molar isotonic electrode paste(NaCl). Recordings were measured with theVarioport† 3-channel bioamplifier (BeckerMEDITEC, Germany) and acquired with time-locked stimulus onsets using Variograph† soft-ware (ibid.) at a sampling rate of 16 Hz.

After the experiment, participants were indivi-dually tested with a neuropsychological battery,consisting of tests on crystallised verbal intelligence

Table 1. Participants’ demographics, questionnaire results, and empathy questionnaires (M9SD)

N Total Males Females t p

Age (yrs) 38 31.3999.47 31.1697.083 31.63911.58 �0.15 .88

Education (yrs) 38 12.7690.88 12.7991.03 12.7490.73 0.18 .86

Questionnaires (Cronbach’s a)

SPF 38 33.8295.69 (0.66) 32.9595.10 34.6896.25 �0.94 .35

E-Scale 38 3.1190.78 (0.94) 2.9090.57 3.3290.91 �1.70 .10

TAS20 38 38.5397.34 (0.74) 39.8497.04 37.2197.58 1.11 .28

WST (total) 38 33.5393.31 32.6894.14 34.3791.98 �1.60 .12

TMT-A (s) 38 18.2296.03 21.3796.24 15.0893.86 3.73 .001

TMT-B (s) 38 36.92916.07 46.70915.84 27.1498.85 4.70 B.001

RWT (total) 38 68.68913.10 63.21910.87 74.16913.09 �2.81 B.01

VERT (% correct) 37 80.6399.92 79.24911.98 82.1097.21 �0.87 .39

Notes: Data for one female were missing for the VERT. For abbreviations see text. Bold letters indicate significant t-test results.



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

executive functions (Trail Making Test, versions Aand B, TMT-A/-B; Aschenbrenner, Tucha, &Lange, 2000; Schmidt & Metzler, 1992), as well astwo empathy questionnaires; the German versionof the Interpersonal Reactivity Index, SPF (IRI;Davis, 1980) and the E-Scale (Leibetseder, Lair-eiter, & Koller, 2007). The SPF is a standardised5-point scale questionnaire measuring emotional(FS�fantasy, EC�emotional concern, PD�personal distress) and cognitive (PT�perspectivetaking) dimensions of empathy. The internalconsistency values (Cronbach’s a) ranged from.71 to .79 (Davis, 1980) and was a bit lower(a�.66) for the total score of the SPF. The E-Scale measures facets of empathic processingincluding the subscales ‘‘cognitive sensitivity’’(CS), ‘‘emotional sensitivity’’ (ES), ‘‘emotionalconcern’’ (EC) and ‘‘cognitive concern’’ (CC).The internal consistency measure (a�.94 for thetotal score) in the present sample was satisfactoryand in the range of the reported values in thestructural analysis of the E-Scale (Leibetsederet al., 2007).

Responses to video clips

The two responses (self and other emotion) pertrial were classified according to the followingscheme (see Figure 1): The emotion presented inat least two of the three channels was consideredthe target emotion. A match between the reportedother emotion and the presented target emotionwas classified as other-matching-target (OMT)response (i.e., correct emotion recognition). Ifthe reported self emotion matched the targetemotion, the emotional responses was called aself-matching-target (SMT) response. We alsoidentified cases in which the reported emotionalstate of the other person matched the ownreported emotional state as self-other-match(SOM). The indicator of an empathic responseand the primary measure of interest was theintersection of these responses, i.e., a self-other-match that also matched the target emotion (self-other-matching-target, SOMT). To explore therelationship of this experimental empathy and its

prerequisites (OMT and SMT) with externalquestionnaire assessments of empathy, we appliedtwo validated empathy questionnaires, describedin the methods section.

As empathic responses depend upon identifi-cation of the target emotion in the other person(OMT) we also computed the conditional fre-quency of SOMT given OMT, p(SOMTjOMT)as conditional self-other-match (cSOMT). It istherefore a measure of empathy that is indepen-dent from emotion recognition (OMT) and isbased purely on the own emotional reaction to thevideo once the target emotion in the other personwas identified.

Cases in which a matching between self andother emotion was achieved, but the targetemotion was not identified (non-target self-other-

match, ntSOM) were further assessed, represent-ing the difference between SOM and SOMTrates.

We further assessed participants’ evaluation ofnaturality and intensity levels of the video clips.

Additionally we analysed the rates of ‘‘neutral’’self (NS) and ‘‘neutral’’ other (NO) responses toinvestigate whether neutralising emotionality inone of the communication channels had a differ-ential effect on perceiving the emotion in theother, or reacting emotionally to the video. Assuch, they are a subset of the responses that wouldnot be classified as SMT or OMT, as the ‘‘target’’was always defined as one of the four emotions(see above). They are of interest because partici-pants may have simply based their response to theneutral channel as no specific instructions weregiven as to which channel(s) their emotionselection should be based on. (A detailed analysisof other non-matches, i.e., cases in which parti-cipants perceived or reacted with an emotion thatwas different from the target emotion, is pre-sented in the supplement.)

For all measures, relative frequencies werecomputed within the different communicationconditions. It is important to note thatthese measures are not fully independent fromone another, as, for example, SOMTs are, by

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

definition, a subset of the other three responsetypes, and all response types can be deducted fromknowledge of at most two.1

Naturality ratings and intensity levels of therated own emotions were averaged within eachcondition. We also analysed the intensity levels ofparticipants’ own responses to those stimuli towhich they responded empathically (Int_SOMT).

Physiological measures

Stimulus-driven changes in GSR were recorded foran overall time window of 1�13 s after stimulusonset; they were included into further analysis at aminimum response criterion of 0.02 mS. Thestimulus interval was divided into two intervals*time window (TW) 1: 1�3 s, TW2: 3�13 s*inorder to avoid confounding effects from measuringglobal means across the whole time course since theemotional load of the clips developed over time.

Raw GSR data were pre-processed and ana-lysed for event-related activation with Ledalab†

V3.2.8 software (Leipzig, Germany). This in-cluded electrode montage artefact removal,smoothing, down-sampling to 2 Hz, and move-ment-related artefact removal. Continuous de-composition analysis was applied to extract phasiccomponents from tonic activity based on standarddeconvolution (Benedek & Kaernbach, 2010).Outcome parameters included the number ofGSRs (nGSR) and their standardised meanamplitude, to account for positively skeweddistributions of skin conductance data*mAmp�log (1�Amp)/nGSR (Venables, Gart-shore, & O’Riordan, 1980). Due to technicalerrors associated with the vulnerability of GSCmeasurements (n�5), as well as some hypo- andhyper-responders (participants with data exceed-ing three standard deviations of stimulus-drivenGSRs across all video clips, n�9 for TW1, n�8for TW2),2 physiological data of n�13 (n�14

for TW1) participants were excluded. This re-duced the sample size for EDA measurements ton�24 for TW1 and n�25 for TW2.

Statistical analysis

All outcome measures (OMT, SMT, SOM,SOMT, cSOMT, ntSOM, Naturality, NO, NS,Int, Int_SOMT, Tonic (TW1), nGSR (TW1),nGSR (TW2), nGSR_SOMT (TW2), mApm_SOMT (TW2), were analysed using full factorialgeneralised estimating equations (GEEs) proce-dure in IBM† SPSS† Statistics to account fortheir potentially non-normal distributions and/orviolations of sphericity. The statistical modelsincluded main effect of the factors ‘‘Communica-tion’’ (within-subjects, four levels) and ‘‘Sex’’(between-subjects, two levels) and interaction.Post hoc pairwise comparisons between condi-tions were performed using paired t-tests andBonferroni correction.

For differences between normally distributedneuropsychological variables of males and females,independent samples t-tests were applied. Forcorrelation analyses, we used Pearson’s r or Spear-man’s r if parametric assumptions were violated.Correlation analyses were Bonferroni corrected.

RESULTS

Match of other and target emotion (emotionrecognition)

Across all experimental communication condi-tions, the average rate of other-matching-targetresponses (OMT; Table 2, Figure 2) wasM�83.39%, SD�17.47%. A GEE analysisshowed a significant main effect of Communica-tion, Wald x2(3)�87.36, pB.001. Pairwisecomparisons indicated significantly higher valuesin condition ‘‘all emotional’’ compared to ‘‘neutral

1For instance, if a response is classified as OMT (the perceived emotion in the other person matches the presented target

emotion), but not as SMT (the emotional self-response does not match the presented target emotion), it can be deduced that there

is no match between perceived emotion in the other and the self (SOM) or between all three categories (SOMT).2We compared hypo- and hyper-responders to normal responders regarding emotion recognition (OMT), emotional responses

(SMT) matching (SOM), and empathy (SOMT) ratings, and did not find significant differences regarding the means (OMT:

t� �1.58, p�.12; SMT: t�0.30, p�.76; SOM: t��.02, p�.99; SOMT: t��0.07, p�.95).



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

prosody’’ (t�5.52), ‘‘neutral face’’ (t�7.80), and

‘‘neutral speech content’’ (t�6.99, all psB.001).

‘‘Neutral prosody’’ received higher OMT rates

than ‘‘neutral face’’ (t�5.15, pB.001) but not

significantly higher than ‘‘neutral speech content’’

(t�2.15, p�.24), while ‘‘neutral speech content’’

and ‘‘neutral face’’ were also not significantly

different from each other (t�2.48, p�.72). The

main effect of Sex, Wald x2(1)�11.68, p�.001,

showed significantly higher OMT rates in women

compared to men while the interaction of Com-

munication and Sex was not significant, Wald

x2(3)�5.72, p�.13.

Match of self and target emotion (emotionalresponse)

Across all four conditions, the average rate of self-

matching-target (SMT; Table 2, Figure 2) was

M�61.88%, SD�25.74%. A GEE analysis

resulted in a significant main effect of Commu-

nication, Wald x2(3)�80.39, pB.001 (Table 2).

Pairwise comparisons indicated significantly high-

er SMT responses in ‘‘all emotional’’ than in

‘‘neutral prosody’’ (t�3.56), ‘‘neutral face’’

(t�6.82), or ‘‘neutral content’’ (t�8.51, all

psB.001). ‘‘Neutral prosody’’ received higher

SMT responses than ‘‘neutral face’’ (t�4.19,

Table 2. Mean values of subjects’ performance rates (in %9SD) within the different communication conditions of recognizing and

responding to the target emotion; constituting empathy

Communication condition

Rating variables Total All emotional Neutral prosody Neutral face Neutral speech

Other emotion matching the target (OMT)

Total 83.39917.47 96.7197.08 86.51912.29 69.74921.38 80.59914.00

Males 80.51918.06 94.0899.19 84.87914.33 66.78921.86 76.32912.60

Females 86.27916.49 99.3491.97 88.1699.97 72.70921.06 84.87914.33

Self emotion matching the target (SMT)

Total 61.88925.74 76.15921.79 68.26921.13 59.87923.05 43.26925.31

Males 56.83925.36 69.08922.87 64.47921.56 54.93922.20 38.82925.14

Females 66.94925.29 83.22918.64 72.04920.56 64.80923.40 47.70925.36

Self emotion �Other emotion (SOM)

Total 64.84923.66 76.64921.97 70.72918.67 60.69918.66 51.32926.84

Males 60.77923.64 69.74923.13 67.43918.11 57.57920.05 48.36927.55

Females 68.91923.12 83.55918.89 74.01919.12 63.82917.13 54.28926.52

Empathy (SOMT)

Total 57.03926.12 75.33921.84 63.98921.08 46.88921.93 41.94925.46

Males 51.73925.53 67.43922.40 59.87921.88 41.78921.66 37.83925.04

Females 62.34925.78 83.22918.64 68.09919.97 51.97921.56 46.05925.87

Conditional self-other match (cSOMT)

Total 67.64925.77 78.01921.97 73.89921.79 67.14923.28 51.51928.15

Males 63.37926.27 72.36924.07 70.28922.70 61.99924.58 48.86928.58

Females 71.90924.70 83.66918.57 77.49920.82 72.29921.31 54.16928.24

Non-target self-other match (ntSOM)

Total 7.81910.36 1.3293.61 6.7497.32 13.82913.01 9.38910.85

Males 9.05911.01 2.3094.76 7.5797.95 15.79913.24 10.53911.98

Females 6.5899.57 0.3391.43 5.9296.74 11.84912.82 8.2299.78

Perceived naturality (1�6)

Total 3.4491.06 4.1590.74 3.7690.80 3.0290.98 2.8291.11

Males 3.5590.99 4.2590.61 3.9590.58 3.2390.85 2.7891.09

Females 3.3291.12 4.0490.85 3.5790.94 2.8191.07 2.8691.15

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

pB.001) and ‘‘neutral speech content’’ (t�6.30,pB.001), and ‘‘neutral face’’ showed higher SMTrates than ‘‘neutral speech content’’ (t�4.47,pB.001). The main effect of Sex, Waldx2(1)�2.86, p�.09, as well as the interaction

(Communication�Sex), Wald x2(3)�4.05,p�.26, were not significant.

Match of self and other

The GEE analysis investigating a self-other-match

(SOM; Table 2, Figure 2) showed a significantmain effect of Communication, Waldx2(3)�40.55, pB.001. Pairwise comparisonsyielded higher SOM rates for ‘‘all emotional’’than for ‘‘neutral prosody’’ (t�2.63, p�.01),‘‘neutral face’’ (t�4.34, pB.001), and ‘‘neutral

speech content’’ (t�6.11, pB.001). ‘‘Neutralprosody’’ had significantly higher matching ratesthan ‘‘neutral face’’ (t�3.21, pB.01) and ‘‘neutralspeech content’’ (t�4.84, pB.001). There was nostatistical difference between ‘‘neutral face’’ and‘‘neutral speech content’’ (t�2.13, p�.19). The

factor Sex, and the interaction of Sex andCommunication did not have significant effects,Wald x2(1)�2.57, p�.11, and Waldx2(3)�4.21, p�.24, respectively.

Self-other-matching-target (empathy)

The GEE analysis investigating self-other-match-

ing-target responses (SOMT empathy; Table 2,Figures 2 and 3) showed a significant main effectof Communication, Wald x2(3)�100.22,pB.001. Compared to ‘‘all emotional’’, therewere significantly lower SOMT responses in‘‘neutral prosody’’ (t�5.20), ‘‘neutral facialexpression’’ (t�9.90), and in ‘‘neural speechcontent’’ (t�8.39, all psB.001). Pairwise com-parisons between the other communication con-ditions replicated the results from the previousanalysis (SOM): significantly higher rates in‘‘neutral prosody’’ than in ‘‘neutral face’’(t�6.35, pB.001) or ‘‘neutral speech content’’(t�5.25, pB.001), and no significant differencebetween ‘‘neutral face’’ and ‘‘neutral speech con-tent’’ (t�1.27, p�.21). The factor Sex, Waldx2(1)�3.55, p�.06, and the interaction (Com-munication�Sex), Wald x2(3)�5.71, p�.13,did not have significant effects.

Across communication conditions and partici-pants, SOMT rates showed small, but nonsigni-ficant correlations with self-report data, SPF (IRI,r�.21, p�20) and E-Scale (r�.27, p�.10),respectively. One prerequisite of empathy, emo-tion recognition (OMT), correlated significantly

Figure 2. Rating results of matching emotional responses between self and other (SOM, one-way striped, please refer to Figure 1) and

empathic responses (SOMT, three-way striped). In conditions with one neutral channel, matching stayed more consistent than empathy.

This was due to an increase in matching responses not meeting the target emotion (ntSOM, grey lined triangles) showing an increase once

communication channels were neutralised.



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

with the total scores of the SPF and the E-Scale(rs�.40 and .44, psB.01).

Conditional self-other match

The GEE analysis investigating the conditionalprobability of an emphatic response given targetemotion recognition (cSOMT; Table 2, Figure 3)showed a significant main effect of Communica-tion, Wald x2(3)�49.35, pB.001, with cSOMTin the ‘‘all emotional’’ condition showing thehighest rates. When prosody was neutral, cSOMTwas not significantly different (t�1.82, p�.42).However, cSOMT decreased in ‘‘neutral face’’(t� �3.61, pB.01), and in ‘‘neutral speech con-tent’’ (t� �6.69, pB.001), compared to ‘‘allemotional’’. ‘‘Neutral speech content’’ was alsosignificantly lower than ‘‘neutral face’’ (t��3.72,p�.001) and ‘‘neutral prosody’’ (t��5.23,pB.001) while ‘‘neutral prosody’’ and ‘‘neutralface’’ did not differ significantly from each other(t�2.09, p�.20). There were no significant maineffects of Sex, Wald x2(1)�1.91, p�.17, or theinteraction of Communication�Sex, Waldx2(3)�1.33, p�.72.

Emotional matching based on a differentemotion than the target emotion

A GEE analysis of non-target-self-other-match(ntSOM, Table 2, Figure 2) resulted in a maineffect of Communication, Wald x2(3)�34.65,pB.001, with significantly lower ntSOM ratesin ‘‘all emotional’’ compared to ‘‘neutral

prosody’’ (t��4.34), ‘‘neutral facial expression’’

(t� �5.38), and ‘‘neutral speech content’’

(t��4.66, all psB.001). The comparison be-

tween ‘‘neutral prosody’’ and ‘‘neutral facial expres-

sion’’ also showed a significant increase from the

former to the latter (t� �3.27, pB.01), while the

remaining pairwise comparisons were not signifi-

cant (‘‘neutral prosody’’ vs. ‘‘neutral speech con-

tent’’, t� �1.33, p�.99; ‘‘neutral face’’ vs.

‘‘neutral speech content’’, t�1.49, p�.79). Sex

showed a significant main effect, Wald

x2(1)�6.42, p�.01, with males having higher

rates of ntSOM than females. The interaction of

Communication � Sex was not significant, Wald

x2(3)�2.15, p�.54.

Perceived naturality

The GEE model investigating naturality rates

attributed to the video clips showed a significant

main effect of Communication, Wald

x2(3)�114.08, pB.001 (Table 2). ‘‘All emo-

tional’’ clips received higher naturality ratings

than ‘‘neutral prosody’’ (t�5.65), ‘‘neutral face’’

(t�9.75), and ‘‘neutral speech content’’ (t�8.03,

all psB.001) video clips. ‘‘Neutral prosody’’ had

significantly higher rates than ‘‘neutral face’’

(t�6.94, pB.001) and ‘‘neutral speech content’’

(t�5.40, pB.001). There was no significant

difference between ‘‘neutral speech’’ and ‘‘neutral

face’’ (t�1.47, p�.72). The factor Sex, and the

interaction of Sex and Communication, did not

Table 3. Mean values of subjects’ mean values (in %9SD) of not responding empathically, and neutrality ratings towards the other and

self emotion


Neutrality ratings Total All emotional Neutral prosody Neutral face Neutral speech

Neutral other rating (NO)

Total 11.23915.45 1.1592.85 10.36912.01 25.49919.89 7.89910.03

Males 12.42916.54 2.3093.73 12.50914.13 27.30921.17 7.57910.54

Females 10.03914.29 0.0090.00 8.2299.33 23.68918.93 8.2299.78

Neutral self rating (NS)

Total 33.10924.64 19.08920.18 28.62920.74 35.03921.23 49.67926.11

Males 37.01925.39 24.34923.93 31.25922.24 39.47920.63 52.96926.71

Females 29.19923.39 13.82914.37 25.99919.35 30.59921.44 46.38925.80

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

have significant effects, Wald x2(1)�0.93,p�.34 and Wald x2(3)�5.03, p�.17.

Non-empathic responses*neutral responses

Non-target responses were most often the resultof choosing ‘‘neutral’’ instead of an emotioncategory (supplement, Table S2). A further eva-luation of this tendency towards neutral was toanalyse how the different communication condi-tions influenced a neutral rating of self or otheremotion, respectively (Figure 4). For this, we setup two GEEs, investigating the rates of neutral-other (NO) and neutral-self responses (NS).

For NO, the GEE showed a significant maineffect of Communication, Wald x2(3)�577.09,pB.001. Pairwise comparisons indicated lowerNO responses in the ‘‘all emotional’’ conditioncompared to the ‘‘neutral prosody’’ (t� �5.38),‘‘neutral face’’ (t� �7.54), or ‘‘neutral speechcontent’’ (t� �4.16, all psB.001) conditions.‘‘Neutral prosody’’ resulted in lower NO ratesthan ‘‘neutral face’’ (t� �4.47, pB.001), butrates were not significantly different from ‘‘neutralspeech content’’ (t�1.10, p�.99). ‘‘Neutral face’’had higher NO rates than ‘‘neutral speech content’’(t�4.45, pB.001). The main effect of Sex, Waldx2(1)�134.19, pB.001, showed higher NO

response rates in males compared to females. Theinteraction failed to show a significant effect,Wald x2(3)�1.23, p�.54.

For NS, the GEE showed a significant maineffect of Communication, Wald x2(3)�72.05,pB.001. Pairwise comparisons indicated lowerNS rates in ‘‘all emotional’’ compared to ‘‘neutralprosody’’ (t� �4.75), ‘‘neutral face’’ (t� �6.81),or ‘‘neutral speech content’’ (t� �7.73, allpsB.001). ‘‘Neutral prosody’’ had lower NS ratesthan ‘‘neutral face’’ (t� �3.39, pB.01), and‘‘neutral speech content’’ (t� �5.31, pB.001).‘‘Neutral face’’ had lower NS rates than ‘‘neutralspeech content’’ (t� �3.98, pB.001). Neitherthe main effect of Sex, Wald x2(1)�1.90, p�.17,nor the interaction, Wald x2(3)�4.24, p�.24,were significant.

Intensity levels

The GEE analysis of subject’s own intensity levels(Int; Table 4) showed a significant main effects ofCommunication, Wald x2(3)�12.05, pB.01. Intrates were higher during ‘‘all emotional’’ comparedto ‘‘neutral speech content’’ (t�2.75, p�.03).‘‘Neutral prosody’’ (t��2.37, p�.10) or ‘‘neutralface’’ (t��1.19, p�.99) did not significantly

Figure 4. Participants’ attribution of neutral states regarding the

other person (NO, dark grey) and the self (NS, light grey), differing

between different communication conditions. ‘‘Neutral speech

content’’ influenced NS more than NO, while this effect was

smaller in ‘‘neutral face’’ as the influence of the latter condition on

emotion recognition came into play was present in ‘‘neutral face’’.

Figure 3. Rating results of empathic responses (SOMT, three-

way striped, please refer to Figure 1) and the influence of own

emotional responses on empathy (cSOMT, dotted). The values

reveal a decrease of both measures once one communication channel

is held neutral, for emotional responses, this was especially present

in ‘‘neutral speech content’’.



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

differ from ‘‘all emotional’’, nor did any otherpairwise comparison show significant effects(‘‘neutral prosody’’ vs. ‘‘neutral face’’, t� �0.64,p�.99; ‘‘neutral prosody’’ vs. ‘‘neutral speechcontent’’, t�1.30, p�.99; ‘‘neutral face’’ vs.‘‘neutral speech content’’, t�2.08, p�.21). Asignificant main effect of Sex, Waldx2(1)�6.90, pB.01, showed higher Int levelsfor males than for the female population. Theinteraction effect was not significant, Waldx2(3)�0.83, p�.84.

For own emotion intensity levels of empathicresponses (Int_SOMT), there was significant maineffect of Communication, Wald x2(3)�43.88,pB.001, showing significantly higher Int_SOMTlevels in ‘‘all emotional’’ compared to ‘‘neutralprosody’’ (t�2.28, p�.03) and ‘‘neutral speechcontent’’ (t�5.86, p�.001), but not compared to‘‘neutral face’’ (t�1.98, p�.28). Further, ‘‘neutralprosody’’ had higher Int_SOMT rates than ‘‘neu-tral speech content’’ (t�4.32, p�.01) but nothigher than ‘‘neutral face’’ (t�0.18, p�.99). Thedifference between ‘‘neutral face’’ and ‘‘neutralspeech content’’ was nonsignificant (t�3.59,p�.06). There was a significant effect of Sex,Wald x2(1)�5.12, p�.02, the interaction was notsignificant, Wald x2 (3)�4.15, p�.25.

Galvanic skin responses

Tonic skin conductance levels (Table 5) in the firstinterval (TW1) were not significantly differentbetween communication conditions*main effect

of Communication, Wald x2(3)�5.61, p�.13.Moreover, a significant interaction of Commu-nication � Sex, Wald x2(3)�11.04, p�.01, butno significant main effect of Sex, Waldx2(1)�0.81, p�.37, emerged. For the interac-tion, pairwise comparisons showed that for malesthe difference between ‘‘neutral prosody’’ and‘‘neutral speech’’ was significant only if no correc-tion was applied (t�2.24, p�.04, uncorrected).Only the difference in tonic conductance betweenconditions ‘‘neutral face’’ and ‘‘neutral speechcontent’’ (t�3.84, pB.01) survived Bonferronicorrection. In females, none of the pairwisecomparisons between conditions survived Bonfer-roni correction, and only the difference between‘‘neutral face’’ and ‘‘neutral prosody’’ was significantif no correction was applied (t��2.51, p�.04,uncorrected).

The GEE for the number of GSRs in TW1yielded a significant main effect of Sex, Waldx2(1)�4.05, p�.04, showing a higher number ofGSRs for males than for females. The main effectof Communication, Wald x2(3)�2.42, p�.49, orthe interaction, Wald x2(3)�0.03, p�.99, didnot yield significant results.

In the second time window (TW2), the GEE ofthe numbers of GSRs resulted in a main effect ofSex, Wald x2(1)�6.04, p�.01, showing signifi-cantly more GSRs in the male population. NeitherCommunication, Wald x2(3)�0.80, p�.85, northe interaction, Wald x2(3)�1.66, p�.65, had asignificant effect.

Table 4. Mean values of subjects’ reported own intensity levels (Int, upper part) and empathic intensity levels (Int_SOMT, lower part)


Intensity ratings Total All emotional Neutral prosody Neutral face Neutral speech

Int

Total 3.1790.83 3.3090.84 3.1590.73 3.2090.80 3.0190.94

Males 3.4690.83 3.5690.87 3.4190.79 3.5290.80 3.3590.92

Females 2.8790.72 3.0490.74 2.8990.59 2.8890.68 2.6890.86

Int_SOMT

Total 3.0290.91 3.2890.90 3.1290.83 3.1090.92 2.5790.87

Males 3.2490.92 3.4391.01 3.3490.86 3.2790.87 2.9490.93

Females 2.7990.85 3.1390.77 2.9090.75 2.9390.95 2.2190.64

Note: Int �Own intensity levels; Int_SOMT �Own intensity levels for empathically responded video clips. Scale ranging from 1 to 6.

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

For the number of GSRs (TW2) in empathic

responses (nGSR_SOMT), the main effect of

Sex, Wald x2(1)�4.52, p�.03, showed signifi-

cantly more GSRs in the male population. The

main effect of Communication, Wald

x2(3)�30.68, pB.001, showed ‘‘all emotional’’

evoking more nGSR_SOMT than ‘‘neutral face’’

(t�5.32, pB.001). There was no statistical

difference between ‘‘all emotional’’ and ‘‘neutral

prosody’’ (t�1.02, p�.99) or ‘‘neutral speech

content’’ (t�3.34, p�.11). ‘‘Neutral prosody’’

yielded more nGSR during empathic responses

than ‘‘neutral face’’ (t�4.96, pB.001). No other

pairwise comparisons resulted in significant dif-

ferences (‘‘neutral prosody’’ vs. ‘‘neutral speech

content’’, t�2.60, p�.99; ‘‘neutral face’’ vs.

neutral speech content’’, t� �0.88, p�.48).

The interaction Communication � Sex did not

reach significance, Wald x2(3)�0.77, p�.86.

The GEE analysis of mean amplitudes in

TW2 (mAmp_SOMT) did not show a significant

main effect of Sex, Wald x2(1)�2.11, p�.15.

The main effect of Communication, Wald

x2(3)�7.18, p�.07, and the interaction effect

Communication�Sex, Wald x2(3)�7.88, p�.05, were not significant.

DISCUSSION

Human communication situations and the role of

different communication channels in eliciting

empathic responses were investigated using nat-

uralistic dynamic emotional video clips. Facial

expressions, prosody, or speech content were

manipulated in such a way that one channel

would stay neutral while the remaining two

channels transported emotional information.

Table 5. Mean values of subjects’ physiological galvanic skin response parameters within the different communication conditions


GSR measure Total All emotional Neutral prosody Neutral face Neutral speech

Tonic activity in time window 1

Total 7.1094.68 7.0794.75 7.1594.74 7.1294.72 7.0494.81

Males 7.7394.82 7.6994.94 7.7594.97 7.8394.91 7.6494.97

Females 6.0594.29 6.0494.49 6.1594.43 5.9594.39 6.0594.64

nGSR time window 1

Total 4.0792.95 4.0092.89 4.0492.42 4.4292.95 3.8393.58

Males 4.8093.04 4.7393.04 4.7392.09 5.1392.95 4.6094.07

Females 2.8692.37 2.7892.28 2.8992.62 3.2292.68 2.5692.24

nGSR time window 2

Total 14.5998.39 14.2897.85 14.5298.43 14.6899.16 14.8898.58

Males 17.4397.86 17.0096.54 17.0099.18 18.0098.22 17.7398.05

Females 10.3397.36 10.2098.18 10.8095.73 9.7098.53 10.6097.85

nGSR_SOMT time window 2

Total 8.3697.28 10.6897.69 9.7297.64 6.0896.20 6.9696.91

Males 10.1098.02 12.2097.81 11.6798.83 7.8797.10 8.6798.16

Females 5.7595.07 8.4097.28 6.8094.32 3.4093.31 4.4093.41

mAmp_SOMT in time window 2

Total 0.0590.05 0.0590.04 0.0590.04 0.0690.05 0.0690.05

Males 0.0690.05 0.0690.04 0.0690.05 0.0690.05 0.0790.06

Females 0.0490.04 0.0490.03 0.0490.03 0.0590.05 0.0490.03

Note: nGSR �number of galvanic skin responses, nGSR_SOMT �number of GSRs for empathically responded video clips,

mAmp_SOMT �mean amplitude for empathically responded video clips.



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

They were compared to an emotionally congruenttri-channel presentation.

The advantage of multichannel emotionalinformation for empathy

In natural tri-channel emotional communication,participants responded empathically to threequarters of the video clips and were also influ-enced by the emotion on a personal level asindicated by their own emotion intensity levelsand galvanic skin response. This effect was alsopresent when controlling for differences in emo-tion recognition, and suggests a strong involve-ment of the participants’ own emotional responsesto the clips. Emotional mirroring, based on aperceived emotion other than the target emotion,barely happened. These findings support thenotion of facilitation of emotional transfer by acombination and integration of multiple channels(de Gelder & Vroomen, 2000; Dolan, Morris, &de Gelder, 2001; Ethofer et al., 2006; Massaro &Egan, 1996). They further support the use ofdynamic stimuli in studies investigating empathy.

Once a channel was ‘‘neutralised’’, the numberof empathic responses, intensity levels, and thenumber of GSRs decreased. This was mainlyattributed to the selection of ‘‘neutral’’ responses;either for the own emotional state, for the other’semotional state, or for both (please refer to thesupplement). This last aspect is especially impor-tant as we set the prerequisites for empathy torecognise and feel the defined target emotion, andnot simply to ‘‘mirror’’ what the participantsthought the other person was feeling. It has tobe kept in mind that although the target emotionwas transmitted via two communication channelsthe third channel transmitted neutrality. If theparticipants’ attention was drawn to this exactchannel because it ‘‘popped out’’ the answer‘‘neutral’’ may have been be sensible and yieldedinteresting results regarding the contribution ofdifferent communication channels to empathy.

Significant associations of questionnaire assess-ments of empathy with emotion recognition, andnonsignificant correlations with empathy mightbe due to conceptually different approaches of the

questionnaires (IRI and E-Scale) and emotionrecognition (OMT) on one side and experimentalempathy (SOMT) on the other. While emotionrecognition and questionnaire empathy are sug-gested to be influenced by social conformity/expectancy effects to a greater extent (e.g., genderstereotypes; see Derntl et al., 2010), the constructof empathy in this experiment relied on partici-pants’ spontaneous subjective emotional responsesin a specific situation (as further assessed by thevariable cSOMT). Questionnaire items inquiringabout everyday situations and the specific questionabout a presented person who was in an urgentemotional state might have potentially targeteddifferent aspects of empathy, explaining their lowcorrelations. While the items assess tendenciesand attitudes and lack specificity as well as reality,our paradigm requires immediate behavioural andspecific responses in a given situation. It is wellknown from research in such areas, that attitudesand real behaviour show only small to moderatecorrelations (e.g., Greenwald, Poehlman, Uhl-mann, & Banaji, 2009). Nevertheless, we arguethat empathy is a complex construct with differentcomponents, and we have demonstrated that someof those components are associated more stronglywith existing standardised questionnaire data thanothers.

Differential contribution of communicationchannels to prerequisites of empathy

As empathy ratings were attenuated once acommunication channel was held neutral, it isimportant to consider the contributions of thedifferent ‘‘prerequisites’’ to this construct. Foremotion recognition, this decrease was mostpronounced when facial expressions remainedneutral. Recognising emotions is highly depen-dent on the visual information communicated bythe face. Without this information the abilityto infer the emotional state of a communi-cation partner based on prosodic and contentinformation alone was drastically reduced. Neu-trality in the face leads to decreased rates ofempathic responses because of an attribution ofthe whole presentation as ‘‘neutral’’. This supports

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

the relevance of facial expressions for emotionrecognition, as has consistently been shown(Ambadar et al., 2005; Carton, Kessler, & Pape,1999; de Gelder & Vroomen, 2000; Etcoff &Magee, 1992; Weiss, Salloum, & Schneider,1999). Further, via cross-modal mechanisms,facial expressions improve the processing ofspeech comprehension (Massaro, 1987; Stevensonet al., 2011; Summerfield, 1979). Although,strictly speaking, the experimental design onlyallows us to make inferences about the lack ofemotional content of one communicationchannel*not directly its contribution*it is sug-gested that neutral facial expressions not onlycaused high degrees of artificiality, but might alsohave inhibited multimodal processing.

This was the case for prosody as well. Wefound consistent differences between ‘‘all emo-tional’’ and ‘‘neutral prosody’’, suggesting that alack of emotional prosody during a communica-tion situation had deteriorating effects on out-come parameters that lead to empathy. Severalstudies have shown that prosody contributes tothe lexical processing of language (Nygaard &Queen, 2008; Schirmer, 2010) and that syntax andprosody interact (see Friederici, 2002, for areview). Based on this background, we suggest alack of multimodal processing and its advanta-geous effects when prosody was neutralised,although the current study did not allow for thisstatement to be tested explicitly.

Whereas prosody and especially facial expres-sion were mainly important for inferring emo-tional states of the other, speech content seemedto be particularly involved regarding the partici-pants’ own emotion. Comparatively low empathylevels in ‘‘neutral speech content’’ were driven bythe participants’ own emotional responses to theclips, which stayed neutral instead of matching thepresented emotion that was conveyed via facialexpression and prosody.

Within the appraisal model of empathy (deVignemont & Singer, 2006) consistent emotionalinformation from multiple channels might bereciprocally evaluated. Depending on the situationfacial expressions, speech prosody and contenttake turns in representing cues and appraisal

variables. Once one communication channel offersincongruent and ‘‘disturbing’’ information, theactual appraisal processed might be biased towardsthis channel and interfere with the establishmentof an empathic response. This ambiguity mighthave been more subtle and resembled an actualnatural situation in ‘‘neutral prosody’’ and ‘‘neutralface’’ (imagine someone telling you a sad storywith neutral prosody). After all, the speechcontent always fits with one of the other twononverbal channels. In contrast, the ambiguitypresent in a mismatch of verbal and para- ornonverbal information (‘‘neutral speech content’’)might have been obviously deteriorating becausethe speech content mismatched both other chan-nels. Naturality ratings supported this by showingthat ‘‘neutral speech content’’ significantly differedfrom ‘‘all emotional’’ and ‘‘neutral prosody’’.

Based on these findings, we attribute a specificrole to speech content, contrary to several findingspreviously suggested (Argyle, Alkema, &Gilmour, 1971; Mehrabian, 1981). During theappraisal of several-channel information, neutra-lised speech might represent a context in whichthe bimodal emotional information cannot beintegrated. In other words, the neutral speechcontent did not give a rational explanation of theemotional visual-auditory perception and subse-quently might have inhibited an emotional per-spective change, which is a prerequisite forempathy. Subsequently, while still recognisingthe target emotion of the other person partici-pants did not respond emotionally to the pre-sentation and rated their own emotional state asneutral.

The latter also shed more light on thedifference between matching and empathy during‘‘neutralised’’ conditions (see Figure 2). Here,matching stayed more stable because the matchwas based on the selection of a neural responsewhile empathy, based on target emotion recogni-tion and response decreased.

Sex differences

Contrary to our hypotheses, which were basedupon studies from our group, we found signifi-



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

cantly higher rates of attributing the other emo-tion to the target emotion in females than males,as well as a statistical trend regarding empathy.These differences were not present for empathycorrected for the effects of emotion recognition.Participants’ own intensity ratings revealed oppo-site-sex differences. Psychophysiological measuresalso showed males to have significantly moreGSRs (generally and during empathic responses),and slightly higher local maxima (n.s).

Summarised, the factor sex keeps yieldingheterogeneous results. While some of the more‘‘external’’ variables, behavioural parameters relat-ing to empathy and intensity ratings, showedhigher values for females than for males, more‘‘internal’’ variables, GSR variables, displayed areversed effect. Results mostly based on externalparameters have resulted in characterising womenas being ‘‘more emotional’’ (Derntl et al., 2010;Diekman & Eagly, 2000; Kring & Gordon, 1998)and suggested women to have a more efficientstyle of (multisensory) emotion processing (Col-lignon et al., 2010). This was supported by studiesthat showed sex differences on both a physiolo-gical (Nater, Abbruzzese, Krebs, & Ehlert, 2006)and a neural level (Habel et al., 2007; Koch et al.,2007).

However, the central factors that might cause adifference in emotion-related parameters are so-cial influence, stereotypes, and cultural sex roles(Grossman & Wood, 1993). While these mightexert differential influence on the sexes resultingin tendencies to express and feel emotions, covertemotional experience (e.g., skin conductance)might not be influenced to the same extent byovert sociocultural factors (Codispoti, Surcinelli,& Baldaro, 2008). The present finding of higherperformance rates regarding recognition accuracyin females, but a higher number of galvanicresponses to emotional material in males mightreflect a qualitatively different processing style ofemotion (Buck, Miller, & Caul, 1974; Buck,Savin, Miller, & Caul, 1972).

A further analysis of the selection of neutralityinstead of the target emotion (see Table 3)revealed males’ higher tendency to select neutralfor the other emotion, but not for their own,

suggesting differences in only one prerequisite ofempathy, emotion recognition, but not emotionalresponses. The latter is also supported by non-significant effects of the factor sex when investi-gating empathy corrected for the effects ofemotion recognition (cSOMT; Table 2).

Against this background, potential differencesbetween the sexes should be carefully interpretedand differential effects of different aspects ofemotion processing should be assumed and in-vestigated to prevent feeding social stereotypesand to avoid oversimplification (see also Cahill,Gorski, Belcher, & Huynh, 2004).

Limitations and conclusion

We purposely chose isomorphic emotion cate-gories as a means of experimentally restrictingreactions to the stimulus material to meet thedefinitions of empathy given in the literature.This restricts the interpretation range of theresults due to a limited answer spectrum. As therestill is no clear agreement on the differentiationbetween the constructs of emotional contagion,empathy and sympathy nor their manifestationand phenomenology (Batson, 2009), it is not clearwhether non-isomorphic responses relate to ‘‘sym-pathetic reactions’’ (Singer & Lamm, 2009) or to awider interpretation of the empathy construct. Toour knowledge, the aspect of having differentialempathic responses to various emotional states hasnot been investigated. It is very likely that in oneemotion category (e.g., happiness), an empathicresponse results in an isomorphic feeling ofcontent or happiness in the receiver. For sadness,however, an empathic response might cause thereceiver to feel pity or concern rather than sadness.However, it depends on the definition whetherthis pity response actually meets the criteria ofempathy or of sympathy (Singer & Lamm, 2009)and if this holds true differential empathic reac-tions to various emotion categories should beconsidered as well in future studies.

Second, by relying on an extensive stimuluscreation procedure (please refer to the supple-ment) we wanted to target the factor ‘‘artificiality’’,which cannot be ruled out in experimental

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

investigations of social constructs. Still, it has tobe kept in mind that experimental stimuli cannever depict real-life situations, and there is nodoubt that the stimuli, whatever quality standardsthey meet, will represent posed emotions to acertain degree.

One last aspect considers the collapsing of fourdifferent emotion categories. The current stimulusmaterial made it impossible to assess both aspects(communication condition and emotion category)in one single design (yielding four cells each) andwe refrained from targeting the emotion-specifi-city at all. However, based on the literature, weexpect different effects for different emotions andwill target this in subsequent work.

Summarised, the present findings suggest thatin human communication, behavioural empathyrelies on consistent information from severalsources; facial expressions, prosody, and speechcontent. Omitting one channel generally results indecreased empathy judgements, lower intensitiesand fewer psychophysiological reactions. Therequirements for empathy are differently affectedby different communication channels. Whilemissing emotional information in the face de-creases a person’s ability to recognise someoneelse’s emotional expression displayed by otherchannels, omitting emotional information in thespeech content causes the largest drop in perfor-mance rates of adequate emotional reactions.

Our findings exceed the level of cognitivelydriven emotion recognition and its importance forempathy to emotional responses (Derntl, Finkel-meyer et al., 2009) and autonomic physiologicalparameters (i.e., GSR). While performance mea-sures were supported by subjective ratings andpsychophysiological measurement, we demon-strated a more complex interaction with sexdifferences. This should be taken into accountwhen investigating studies in social cognitionusing dynamic stimulus material and targetingseveral measurement levels.

Manuscript received 28 April 2011

Revised manuscript received 8 September 2011

Manuscript accepted 18 September 2011

First published online 5 January 2012

REFERENCES

Adolphs, R. (2002). Recognizing emotion from facial

expressions: Psychological and neurological mechan-

isms. Behavioral and Cognitive Neuroscience Reviews,

1(1), 21�61.Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005).

Deciphering the enigmatic face: The importance of

facial dynamics in interpreting subtle facial expres-

sions. Psychological Science, 16(5), 403�410.Argyle, M., Alkema, F., & Gilmour, R. (1971). The

communication of friendly and hostile attitudes by

verbal and non-verbal signals. European Journal of

Social Psychology, 1(3), 385�402.Aschenbrenner, S., Tucha, O., & Lange, K. W. (2000).

Regensburger Word Fluency Test [Regensburger Wort-

flussigkeits-Test, RWT]. Gottingen, Germany: Ho-

grefe.Bagby, R. M., Taylor, G. J., & Ryan, D. (1986).

Toronto Alexithymia Scale*Relationship with per-

sonality and psychopathology measures. Psychother-

apy and Psychosomatics, 45(4), 207�215.Batson, C. D. (2009). These things called empathy. In

J. Decety & W. Ickes (Eds.), The Social Neuroscience

of Empathy (pp. 3�16). Cambridge, MA: MIT

Press.Benedek, M., & Kaernbach, C. (2010). Decomposition

of skin conductance data by means of nonnegative

deconvolution. Psychophysiology, 47(4), 647�658.Blasko, D. G., & Hall, M. D. (1998). Influence of

prosodic boundaries on comprehension of spoken

English sentences. Perceptual and Motor Skills, 87(1),

3�18.Bostanov, V., & Kotchoubey, B. (2004). Recognition of

affective prosody: Continuous wavelet measures of

event-related brain potentials to emotional exclama-

tions. Psychophysiology, 41(2), 259�268.Buck, R., Miller, R. E., & Caul, W. F. (1974). Sex,

personality, and physiological variables in the com-

munication of affect via facial expression. Journal of

Personality and Social Psychology, 30, 587�596.Buck, R., Savin, V. J., Miller, R. E., & Caul, W. F.

(1972). Communication of affect through facial

expressions in humans. Journal of Personality and

Social Psychology, 23, 362�371.Cahill, L., Gorski, L., Belcher, A., & Huynh, Q.

(2004). The influence of sex versus sex-related traits

on long-term memory for gist and detail from an

emotional story. Consciousness and Cognition, 13(2),

391�400.



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

Campanella, S., & Belin, P. (2007). Integrating face

and voice in person perception. Trends in Cognitive

Sciences, 11(12), 535�543.Carton, J. S., Kessler, E. A., & Pape, C. L. (1999).

Non-verbal decoding skills and relationship well-

being in adults. Journal of Nonverbal Behavior, 23(1),

91�100.Codispoti, M., Surcinelli, P., & Baldaro, B. (2008).

Watching emotional movies: Affective reactions and

gender differences. International Journal of Psycho-

physiology, 69, 90�95.Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-

Amour, D., Lassonde, M., et al. (2008). Audio-

visual integration of emotion expression. Brain

Research, 1242, 126�135.Collignon, O., Girard, S., Gosselin, F., Saint-Amour,

D., Lepore, F., & Lassonde, M. (2010). Women

process multisensory emotion expressions more

efficiently than men. Neuropsychologia, 48(1), 220�225.

Cutler, A., Dahan, D., & van Donselaar, W. (1997).

Prosody in the comprehension of spoken language:

A literature review. Language and Speech, 40(2),

141�201.Davis, M. H. (1980). A multidimensional approach to

individual differences in empathy. JSAS (Journal

Supplement Abstract Service) Catalog of Selected

Documents in Psychology, 10, 85.de Gelder, B., & Vroomen, J. (2000). The perception of

emotions by ear and by eye. Cognition and Emotion,

14, 289�311.de Vignemont, F., & Singer, T. (2006). The empathic

brain: How, when and why? Trends in Cognitive

Sciences, 10(10), 435�441.de Waal, F. B. M. (2008). Putting the altruism back

into altruism: The evolution of empathy. Annual

Review of Psychology, 59, 279�300.Decety, J., & Jackson, P. L. (2004). The functional

architecture of human empathy. Behavioral and

Cognitive Neuroscience Reviews, 3(2), 71�100.Derntl, B., Finkelmeyer, A., Eickhoff, S., Kellermann,

T., Falkenberg, D. I., Schneider, F., et al. (2010).

Multidimensional assessment of empathic abilities:

Neural correlates and gender differences. Psycho-

neuroendocrinology, 35(1), 67�82.Derntl, B., Finkelmeyer, A., Toygar, T. K., Hulsmann,

A., Schneider, F., Falkenberg, D. I., et al. (2009).

Generalized deficit in all core components of

empathy in schizophrenia. Schizophrenia Research,

108(1�3), 197�206.

Derntl, B., Habel, U., Windischberger, C., Robinson,S., Kryspin-Exner, I., Gur, R. C., et al. (2009).General and specific responsiveness of the amygdaladuring explicit emotion recognition in females andmales. BMC Neuroscience, 10, 91.

Diekman, A. B., & Eagly, A. (2000). Stereotypes asdynamic constructs: Women and men of the past,present, and future. Personality and Social Psychology

Bulletin, 26(10), 1171�1188.Dolan, R. J., Morris, J. S., & de Gelder, B. (2001).

Crossmodal binding of fear in voice and face.Proceedings of the National Academy of Sciences,98(17), 10006�10010.

Ekman, P., & Friesen, W. V. (1976). Pictures of Facial

Affect. Palo Alto, CA: Consulting PsychologistsPress.

Etcoff, N. L., & Magee, J. J. (1992). Categoricalperception of facial expressions. Cognition, 44(3),227�240.

Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, L.,Saur, R., et al. (2006). Impact of voice on emotionaljudgment of faces: An event-related fMRI study.Human Brain Mapping, 27(9), 707�714.

Ethofer, T., Van De Ville, D., Scherer, K., &Vuilleumier, P. (2009). Decoding of emotionalinformation in voice-sensitive cortices. Current

Biology, 19(12), 1028�1033.Farrow, T. F., Zheng, Y., Wilkinson, I. D., Spence,

S. A., Deakin, J. F., Tarrier, N., et al. (2001).Investigating the functional anatomy of empathyand forgiveness. Neuroreport, 12(11), 2433�2438.

Friederici, A. D. (2002). Towards a neural basis ofauditory sentence processing. Trends in Cognitive

Sciences, 6(2), 78�84.Friederici, A. D., & Alter, K. (2004). Lateralization of

auditory language functions: A dynamic dual path-way model. Brain and Language, 89(2), 267�276.

Greenwald, A., Poehlman, T. A., Uhlmann, E. L., &Banaji, M. R. (2009). Understanding and using theImplicit Association Test: III. Meta-analysis ofpredictive validity. Journal of Personality and Social

Psychology, 97(1), 17�41.Grossman, M., & Wood, W. (1993). Sex differences in

intensity of emotional experience: A social roleinterpretation. Journal of Psychology and Social Psy-

chology, 65, 1010�1022.Habel, U., Windischberger, C., Derntl, B., Robinson,

S., Kryspin-Exner, I., Gur, R. C., et al. (2007).Amygdala activation and facial expressions: Explicitemotion discrimination versus implicit emotionprocessing. Neuropsychologia, 45(10), 2369�2377.

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

Hall, J., Harris, J. M., Sprengelmeyer, R., Sprengel-meyer, A., Young, A. W., Santos, I. M., et al.(2004). Social cognition and face processing inschizophrenia. British Journal of Psychiatry, 185,169�170.

Hammerschmidt, K., & Jurgens, U. (2007). Acousticalcorrelates of affective prosody. Journal of Voice,21(5), 531�540.

Humphries, C., Love, T., Swinney, D., & Hickok, G.(2005). Response of anterior temporal cortex tosyntactic and prosodic manipulations during sen-tence processing. Human Brain Mapping, 26(2),128�138.

Kilts, C. D., Egan, G., Gideon, D. A., Ely, T. D., &Hoffman, J. M. (2003). Dissociable neural pathwaysare involved in the recognition of emotion in staticand dynamic facial expressions. NeuroImage, 18,156�168.

Koch, K., Pauly, K., Kellermann, T., Seiferth, N. Y.,Reske, M., Backes, V., et al. (2007). Genderdifferences in the cognitive control of emotion: AnfMRI study. Neuropsychologia, 45, 2744�2754.

Koelewijn, T., Bronkhorst, A., & Theeuwes, J. (2010).Attention and the multiple stages of multisensoryintegration: A review of audiovisual studies. Acta

Psychologica, 134(3), 372�384.Kotz, S. A., Meyer, M., Alter, K., Besson, M., von

Cramon, D. Y., & Friederici, A. D. (2003). On thelateralization of emotional prosody: An event-re-lated functional MR investigation. Brain and Lan-

guage, 86(3), 366�376.Kramer, U. M., Mohammadi, B., Donamayor, N.,

Samii, A., & Munte, T. F. (2010). Emotional andcognitive aspects of empathy and their relation tosocial cognition*An fMRI-study. Brain Research,1311, 110�120.

Kreifelts, B., Ethofer, T., Grodd, W., Erb, M., &Wildgruber, D. (2007). Audiovisual integration ofemotional signals in voice and face: An event-relatedfMRI study. NeuroImage, 37(4), 1445�1456.

Kring, A. M., & Gordon, A. H. (1998). Sex differencesin emotion: Expression, experience, and physiology.Journal of Personality and Social Psychology, 74(3),686�703.

LaBar, K. S., Crupain, M. J., Voyvodic, J. T., &McCarthy, G. (2003). Dynamic perception of facialaffect and identity in the human brain. Cerebral

Cortex, 13(10), 1023�1033.Leibetseder, M., Laireiter, A.-R., & Koller, T. (2007).

Structural analysis of the E-scale. Personality and

Individual Differences, 3, 547�561.

Massaro, D. W. (1987). Speech perception by eye and by

ear: A paradigm for psychological inquiry. Hillsdale,

NJ: Lawrence Erlbaum Associates, Inc.Massaro, D. W., & Egan, P. B. (1996). Perceiving

affect from the voice and the face. Psychonomic

Bulletin and Review, 3, 215�221.Mehrabian, A. (1981). Silent messages: Implicit commu-

nication of emotions and attitudes. Belmont, CA:

Wadsworth.Nater, U. M., Abbruzzese, E., Krebs, M., & Ehlert, U.

(2006). Sex differences in emotional and psycho-

physiological responses to musical stimuli. Interna-

tional Journal of Psychophysiology, 62(2), 300�308.Nygaard, L. C., & Queen, J. S. (2008). Communicating

emotion: Linking affective prosody and word mean-

ing. Journal of Experimental Psychology: Human

Perception and Performance, 34(4), 1017�1030.Oldfield, R. C. (1971). The assessment and analysis of

handedness: The Edinburgh Inventory. Neuropsy-

chologia, 9(1), 97�113.Parker, J. D., Taylor, G. J., & Bagby, R. M. (2003).

The 20-Item Toronto Alexithymia Scale. III.

Reliability and factorial validity in a community

population. Journal of Psychosomatic Research, 55(3),

269�275.Paulmann, S., & Kotz, S. A. (2008). Early emotional

prosody perception based on different speaker

voices. Neuroreport, 19(2), 209�213.Peelen, M. V., Atkinson, A. P., & Vuilleumier, P.

(2010). Supramodal representations of perceived

emotions in the human brain. The Journal of

Neuroscience, 30(30), 10127�10134.Penn, D. L., Sanna, L. J., & Roberts, D. L. (2008).

Social cognition in schizophrenia: An overview.

Schizophrenia Bulletin, 34(3), 408�411.Roman, F., Garcia-Sanchez, F. A., Martinez-Selva,

J. M., Gomez-Amor, J., & Carrillo, E. (1989). Sex

differences and bilateral electrodermal activity: A

replication. The Pavlovian Journal of Biological

Science, 24(4), 150�155.Sato, W. S., Kochiyama, T., Yoshikawa, S., Naito, E.,

& Matsumura, M. (2004). Enhanced neural activity

in response to dynamic facial expressions of emo-

tion: An fMRI study. Cognitive Brain Research, 20,

81�91.Schirmer, A. (2010). Mark my words: Tone of voice

changes affective word representations in memory.

PLoS ONE, 5(2), e9080.Schmidt, K. H., & Metzler, P. (1992). Wortschatztest

(WST). Weinheim, Germany: Beltz Test GmbH.



Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

Seubert, J., Kellermann, T., Loughead, J., Boers, F.,Brensinger, C., Schneider, F., et al. (2010). Proces-sing of disgusted faces is facilitated by odor primes:A functional MRI study. NeuroImage, 53(2), 746�756.

Simons, R. F., Detender, B. H., Roedema, T. M., &Reiss, J. E. (1999). Emotion processing in threesystems: The medium and the message. Psychophy-

siology, 36, 619�627.Singer, T., & Lamm, C. (2009). The social neu-

roscience of empathy. Annals of the New York

Academy of Sciences, 1156, 81�96.Stevenson, R. A., VanDerKlok, R. M., Pisoni, D. B., &

James, T. W. (2011). Discrete neural substratesunderlie complementary audiovisual speech integra-tion processes. NeuroImage, 55(3), 1339�1345.

Summerfield, A. Q. (1979). Use of visual informationon phonetic perception. Phonetica, 36, 314�331.

Thonnessen, H., Boers, F., Dammers, J., Chen, Y. H.,Norra, C., & Mathiak, K. (2010). Early sensoryencoding of affective prosody: Neuromagnetic to-mography of emotional category changes. Neuro-

Image, 50(1), 250�259.Trautmann, S. A., Fehr, T., & Herrmann, M. (2009).

Emotions in motion: Dynamic compared to staticfacial expressions of disgust and happiness reveal

more widespread emotion-specific activations. Brain

Research, 1284, 100�115.Venables, P. H., Gartshore, S. A., & O’Riordan, P. W.

(1980). The function of skin conductance response

recovery and rise time. Biological Psychology, 10(1),

1�6.Weiss, U., Salloum, J. B., & Schneider, F. (1999).

Correspondence of emotional self-rating with facial

expression. Psychiatry Research, 86(2), 175�184.Werner, S., & Noppeney, U. (2010). Distinct func-

tional contributions of primary sensory and associa-

tion areas to audiovisual integration in object

categorization. Journal of Neuroscience, 30(7), 2662�2675.

Weyers, P., Muhlberger, A., Hefele, C., & Pauli, P.

(2006). Electromyographic responses to static and

dynamic avatar emotional facial expressions. Psycho-

physiology, 43, 450�453.Wildgruber, D., Ackermann, H., Kreifelts, B., &

Ethofer, T. (2006). Cerebral processing of linguistic

and emotional prosody: fMRI studies. Progress in

Brain Research, 156, 249�268.Zeelenberg, R., & Bocanegra, B. R. (2010). Auditory

emotional cues enhance visual perception. Cognition,

115(1), 202�206.

REGENBOGEN ET AL.


Dow

nloa

ded

by [

Chr

istin

a R

egen

boge

n] a

t 13:

10 0

3 Ja

nuar

y 20

12

The differential contribution of facial expressions, prosody, and speech content to empathy

Documents

Transcript of The differential contribution of facial expressions, prosody, and speech content to empathy