NeuroImage 47 (2009) 2005–2015

Contents lists available at ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r.com/ locate /yn img

Detecting sarcasm from paralinguistic cues: Anatomic and cognitive correlates inneurodegenerative disease

Katherine P. Rankin ⁎, Andrea Salazar, Maria Luisa Gorno-Tempini, Marc Sollberger, Stephen M. Wilson,Danijela Pavlic, Christine M. Stanley, Shenly Glenn, Michael W. Weiner, Bruce L. MillerUniversity of California San Francisco, USA

⁎ Corresponding author. Memory and Aging Center, U350 Parnassus Avenue, Suite 905, San Francisco, CA 9414800.

E-mail address: krankin@memory.ucsf.edu (K.P. Ran

1053-8119/$ – see front matter © 2009 Elsevier Inc. Aldoi:10.1016/j.neuroimage.2009.05.077

a b s t r a c t

a r t i c l e i n f o

Article history:Received 17 June 2008Revised 19 May 2009Accepted 21 May 2009Available online 6 June 2009

While sarcasm can be conveyed solely through contextual cues such as counterfactual or echoic statements, face-to-face sarcastic speechmay be characterized by specific paralinguistic features that alert the listener to interpretthe utterance as ironic or critical, even in the absence of contextual information. We investigated theneuroanatomy underlying failure to understand sarcasm from dynamic vocal and facial paralinguistic cues.Ninety subjects (20 frontotemporal dementia,11 semantic dementia [SemD], 4 progressive non-fluent aphasia, 27Alzheimer's disease, 6 corticobasal degeneration, 9 progressive supranuclear palsy, 13 healthy older controls)were tested using the Social Inference — Minimal subtest of The Awareness of Social Inference Test (TASIT).Subjects watched brief videos depicting sincere or sarcastic communication and answered yes–no questionsabout the speaker's intended meaning. All groups interpreted Sincere (SIN) items normally, and only the SemDgroup was impaired on the Simple Sarcasm (SSR) condition. Patients failing the SSR performed more poorly ondynamic emotion recognition tasks and had more neuropsychiatric disturbances, but had better verbal andvisuospatial workingmemory than patients who comprehended sarcasm. Voxel-basedmorphometry analysis ofSSR scores in SPM5 demonstrated that poorer sarcasm comprehension was predicted by smaller volume inbilateral posterior parahippocampi (PHc), temporal poles, and R medial frontal pole (pFWEb0.05). This studyprovides lesion data suggesting that the PHc may be involved in recognizing a paralinguistic speech profile asabnormal, leading to interpretive processingby the temporal poles and rightmedial frontal pole that identifies thesocial context as sarcastic, and recognizes the speaker's paradoxical intentions.

© 2009 Elsevier Inc. All rights reserved.

Introduction

Sarcasm is a type of ironic speech inwhich an implicit criticism of aspecific target is conveyed via contextual or paralinguistic cues. Itssocial function is to heighten dramatic effect (McDonald, 1999) whilesimultaneously increasing the perceived politeness of the speaker(Jorgensen, 1996) and decreasing the aggressiveness of the criticalcomment (Dews and Winner, 1995). While sarcasm can be conveyedsolely through contextual cues such as counterfactual or echoicstatements, and thus may be recognized in text communications,face-to-face sarcastic speech may be characterized by a specificparalinguistic profile that alerts the listener not to interpret theutterance sincerely, even in the absence of contextual information.Analysis of the vocal qualities of sarcastic speech suggests that it ischaracterized by an increased range and amplitude of fundamentalvoice frequency, higher emphatic stress, shorter pauses, and acaricatured lengthening of syllables compared to sincere speech

CSF Department of Neurology,43-1207, USA. Fax: +1 415 476

kin).

l rights reserved.

(Anolli et al., 2000; Rockwell, 2007). Sarcasm is “a technique thatplayswith the voice, not in a natural but in a studiedway” that is “bothpremeditated and affected” (Anolli et al., 2000). Analysis of sarcasm'snon-acoustic paralinguistic features suggests that it involves varyingorflattening the range and intensity of one's facial expression, and usingtechniques such as widened, rolling eyes, more rapid blinking,increased grimacing and smirks to help alert the listener that themeaning is ironic (Attardo et al., 2003; Rockwell, 2001).

The ability to recognize sarcasm from paralinguistic cues developsearlier (around age 5) than the ability to correctly interpret sarcasmfrom contextual cues (around age 7) (Laval and Bert-Eboul, 2005), andmultiple lines of evidence converge to suggest that the latter is a morecomplex, difficult task. Accordingly, patients with traumatic braininjury (TBI), schizophrenia, autism, and dementia have demonstrateddeficits interpreting sarcasm from contextual cues (Bara et al., 2000;Champagne et al., 2003; Channon et al., 2005, 2007; Dennis et al.,2001; Leitman et al., 2006; Martin and McDonald, 2004; Rajendranet al., 2005). However, some studies using either audio or audio-visual sarcastic stimuli suggest that these deficits may persist evenwhen subjects are presented with paralinguistic sarcasm cues(McDonald, 1996; McDonald et al., 2006, 2003). Schizophrenicsubjects not only fail to detect sarcasm in auditory stimuli, but are

2006 K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

biased toward identifying statements as sincere compared to controls(Leitman et al.,). The one study using dynamic stimuli to assesssarcasm comprehension in patients with frontotemporal dementiaused stimuli that mixed paralinguistic and contextual cues (Kippset al., 2009), thus the performance of patients with neurodegenerativedisease on sarcasm tasks using purely paralinguistic rather thancontextual stimuli remains unknown.

While poorer recognition of paralinguistic sarcasm cues shows somecorrelation with emotion recognition in patient groups (Leitman et al.,2006; McDonald et al., 2006; Shamay-Tsoory et al., 2005a), theirrelationship is unclear. Schizophrenic patients who show deficitsrecognizingparalinguistic sarcasmalso performpoorly onvoiceprosodytasks, suggesting that voice prosody may play a significant role insarcasm recognition (Leitman et al., 2006). Sarcasm comprehension hasalso been related to deficits in other cognitive areas such as slowedinformation processing speed, poorer workingmemory, reduced verbaland non-verbal new learning, and deficits in complex non-verbalexecutive reasoning, but the degree towhich these skills are involved inthe interpretation of the paralinguistic versus contextual aspects ofsarcasm has never been delineated (McDonald et al., 2006).

The right temporal lobe is involved in recognizing and categorizingvocal prosody and facial cues (Allison et al., 2000), and correct inter-pretation of textual irony appears to be partly mediated by righttemporal and dorsomedial frontal structures (Champagne et al., 2003;Eviatar and Just, 2006; Shamay-Tsoory et al., 2005a). However, neuro-anatomic studies of sarcasm recognition have primarily used textstimuli, and the anatomy underpinning paralinguistic sarcasm interpre-tationhas not beendirectly studied inhealthycontrols orpatient groups.

We investigated the neuroanatomic correlates of the ability to useparalinguistic cues to recognize sarcasm in patients with neurode-generative disease by first testing subjects with a psychometricallyvalidated measure of sarcasm comprehension, then performingquantitative analysis of structural MRI scans. The aim of this studywas to determine the degree to which regional differences in brainvolumes correspond to the ability to detect sarcasm from dynamicvocal and facial paralinguistic stimuli.

Materials and methods

Subjects

Ninety subjects were studied, including 77 patients diagnosedwith one of six neurodegenerative diseases and 13 healthy older

Table 1General and neuropsychiatric characteristics of subject sample classified by diagnostic grou

bvFTD SemD PNFAM (SD) N=20 N=11 N=4

Age 60.0 (8.1) 63.0 (8.6) 66.3 (11.5)Education 16.7 (2.6) 16.6 (3.4) 16.3 (3.3)Sex (M/F) 15/5 5/6 0/4CDR 1.1 (0.6)a 0.7 (0.3)a 0.4 (0.3)CDR sum of boxes 6.6 (2.6)a 4.2 (2.4)a 1.5 (0.8)MMSE (max=30) 24.6 (5.7)a 22.0 (6.0)a 23.5 (5.3)Geriatric Depression Scale (max=30) 5.2 (5.9) 7.8 (4.5) 3.3 (2.9)Neuropsychiatric Inventory 37.8 (21.6)a 25.3 (19.4)a 10.8 (11.9)CATSEmotional Prosody Discrimination (max=22) 18.8 (4.5) 21.0 (0.6) 16.7 (5.8)Name Prosody Discrimination (max=22) 9.6 (3.5)a 9.3 (3.5) 11.5 (9.2)

TASITPart 1: Emotion Evaluation (max=14) 7.2 (3.1)a 6.5 (2.3)a 8.0 (2.9)Part 2: Sincere (max=20) 15.6 (3.1) 17.4 (2.0) 15.5 (4.7)Part 2: Simple Sarcasm (max=20) 13.1 (5.8)b 8.7 (6.1)a 13.0 (1.6)

For CATS and TASIT tests, the F-statistic and p-values are for overall diagnostic group differewere included in the overall diagnostic group difference analyses. Post hoc pairwise group diDunnett–Hsu test controlling for age, sex, and MMSE. bvFTD=behavioral variant frontotemAD=Alzheimer's disease, CBD=corticobasal degeneration, PSP=progressive supranuclear

a Differs from NC at pb0.05.b Non-significant trend towards difference from NC at pb0.10.

normal controls. Patients were recruited into the study from a demen-tia specialty clinic. These included 20 patients who met the Nearycriteria (Neary et al., 1998) for the frontotemporal dementia (bvFTD)variant of frontotemporal lobar degeneration (FTLD) (typicallycharacterized by bilateral frontal disease and a progressive behavioralsyndrome) (Snowden et al., 2007), 11 with the semantic dementia(SemD) variant of FTLD (typically characterized by left anteriortemporal and orbitofrontal atrophy along with profound semanticloss) (Hodges and Patterson, 2007), and four with the progressivenon-fluent aphasia (PNFA) variant of FTLD (typically characterized byleft inferior frontal atrophy and non-fluent speech) (Gorno-Tempiniet al., 2004). In addition to the FTLD patients, 27 subjects hadAlzheimer's disease (AD) (diagnosed by NINDS-ADRDA criteria(McKhann et al., 1984)), six had corticobasal degeneration (CBD)defined according to the criteria specified in Boxer et al. (2005), andnine had progressive supranuclear palsy (PSP), diagnosed by theLitvan criteria (Litvan et al., 1996). Patient diagnosis was derived by amultidisciplinary team consisting of neurologists, neuropsychologists,psychiatrists, and nurses, who performed extensive neurological,behavioral, neuropsychological, and neuroimaging assessments.Patients from diverse diagnostic groups with variable behavioral testscores and patterns of gray matter atrophy were included to providevariability in the sample and thus increase the power of the corre-lation analysis. Patients with a Clinical Dementia Rating (CDR) scoreN2 were excluded, as were subjects who were not fluent in English.

Healthy control subjects were recruited through advertisements inlocal newspapers and recruitment talks at local senior communitycenters, then underwent an extensive multidisciplinary clinicalevaluation. For inclusion as healthy controls for this study, subjectshad to have a normal neurologic exam, a Clinical Dementia RatingScale (CDR) score=0, MMSE score equal to or greater than 28/30, anddelayed memory performance equal to or greater than the 25thpercentile in both verbal and visuospatial domains.

All subjects, and where applicable their caregivers, signed aninstitutional review board-approved research consent form toparticipate in the study. Patients seen at the clinic represented abroad sample of the population in terms of ethnicity, sex, educationlevel, and socioeconomic status, and an attempt was made to recruitall available consecutive patients for this study. Subjects' demographiccharacteristics can be seen in Table 1. Subjects' mean age was 61.8(SD±8.3), and they averaged 16.1 (SD±3.0) years of education. Therewere 43 males and 47 females, and the mean CDR score for all non-normal subjects was 0.9 (SD±0.5). Statistically significant differences

p.

AD CBD PSP NC F-statistic (df) p-valueN=27 N=6 N=9 N=13

59.2 (7.0) 67.0 (6.2) 66.3 (10.3) 61.8 (10.3) 1.77 (83,6) n.s.15.3 (3.2) 14.5 (3.1) 16.3 (2.3) 17.3 (2.5) 1.22 (82,6) n.s.16/11 0/6 2/7 5/8 x2=19.35 0.00360.8 (0.4)a 0.8 (0.8)a 0.9 (0.5)a 0 (0) 8.04 (79,6) b0.00014.7 (2.1)a 3.6 (3.4) 5.6 (2.6) 0 (0) 10.78 (79,6) b0.0001

22.2 (5.0)a 24.3 (4.8) 27.7 (2.3) 29.3 (0.9) 4.46 (83,6) 0.00067.4 (5.4) 10.0 (8.7) 10.7 (5.6) 4.5 (5.1) 1.90 (73,6) n.s.11.1 (10.0) 14.2 (9.8) 16.0 (14.0) 0.1 (0.3) 18.44 (78,6) b0.0001

19.7 (2.6) 19.3 (1.5) 20.5 (1.2) 20.9 (1.0) 1.70 (63,9) n.s.7.4 (4.2)b 9.5 (3.4) 10.5 (2.9) 14.8 (3.9) 1.90 (42,9) n.s.

8.7 (2.4) 9.8 (2.8) 8.8 (2.6)a 11.9 (1.4) 4.05 (76,9) 0.001414.0 (3.1) 13.0 (5.6) 15.8 (3.4) 15.2 (4.0) 1.73 (80,9) n.s.15.1 (3.8) 17.0 (3.0) 13.4 (4.0) 16.8 (3.2) 4.36 (80,9) 0.0007

nces controlling for age, sex, and MMSE; for other measures, no confounding covariatesfferences were performed comparing each patient group with the control group using aporal dementia, SemD=semantic dementia, PNFA=progressive non-fluent aphasia,palsy, NC=older normal controls.

2007K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

were seen across groups in sex, but not age or education. Because sex,age, andMMSEwere included as potentially confounding covariates inthe imaging analyses, these were also included as covariates in allanalyses of test scores.

Assessment of sarcasm detection

Each subject performed the Social Inference — Minimal subtest ofThe Awareness of Social Inference Test (TASIT-2) (McDonald andRollins, 2002). This test is designed to assess subjects' ability tointerpret naturalistic social interactions in which the speaker utilizessincerity, sarcasm, or paradoxical sarcasm to communicate. Only theSincere (SIN) and Simple Sarcasm (SSR) subtests were analyzed forthis study. Subjects watch 10 brief (less than 1 min) video vignettes inwhich professional actors interact, and then answer four yes–noquestions about the actions, thoughts, words, and emotions of thecharacters. The vignettes for the Sincere and Simple Sarcasmconditions (5 each) are presented after an unscored sample video isused to instruct the subject about the task, and are mixed togetherthroughout the test, so subjects could not develop a “sincere” or“sarcastic” response set based on the order of the items.

In the Sincere and Simple Sarcasm conditions, the scripted verbalcontent is neutral and is interchangeable between the conditions (e.g.,“I'd be happy to do it. I've got plenty of time.”), so subjects mustobserve paralinguistic cues, including facial expression, voice prosody,hand and head gestures, and body posture, to determine the speaker'sintended meaning. In the Sincere condition, the speaker's non-verbalcues are consistent with the verbal content, thus no irony is implied,and the verbal content accurately signifies the speaker's intendedmeaning. If the example script above appeared in the Sincere condi-tion, the correct interpretationwould be that the speaker truly is eagerto help, and has enough spare time to do the work. In the SimpleSarcasm condition, the speaker uses exaggerated facial, vocal, andbody language indicating sarcasm, thus their intended meaning isironic and diverges from the manifest verbal content of their speech. Ifthe example script above was used in the Simple Sarcasm condition,the correct interpretation would be that the speaker doesn't want todo the task because she is too busy. Though the neutral content istheoretically interchangeable between conditions, no script was usedfor more than one single item during the test, so subjects were notgiven the opportunity to compare a sincere and a sarcastic reading ofthe same script. The four yes–no questions after each items requiredthe subject to correctly identify the speaker's meaning (e.g., “IsRuth trying to pressure Gary into helping her?” “Is she annoyed withhim?”).

Performance on the Sincere condition was used to gauge subjects'ability to perform the basic demands of the task, such as comprehendingthe actors' speech, following the flow of the social interaction,remembering the vignette long enough to answer yes–no questionsabout it, and comprehending the questions themselves. For bothconditions, approximately half of the questions were reversed, so apositive or negative response setwould indicate nonsensical respondingand would result in a failing (chance level) score on both conditions.

Testing was completed within 4 months before or after the MRIscan, and the average span of time between testing and scan was13 days (SD±26 days).

Assessment of emotion comprehension

Additionally, at the time of their sarcasm testing, subjects under-went testing of emotion recognition in facial, vocal, and combinedmodalities. Subjects were testedwith 2 subtests of the ComprehensiveAffect Testing System (CATS) (Froming et al., 2001), includingEmotional Prosody Discrimination (discriminating same or differentemotional voice prosody with neutral semantic content), and NameEmotional Prosody (multiple-choice naming emotional voice prosody

with the four emotions happy, sad, frightened, and angry). To assesstheir ability to identify emotions with more ecologically valid, dyna-mic, multimodal stimuli, subjects performed an abbreviated form ofthe Emotion Evaluation Subtest of the TASIT. For this test, subjectswatch brief (∼20 s) videos of actors performing semantically neutralscripts portraying one of the seven basic emotions (happy, surprised,neutral, sad, anxious, frightened, revolted), and must choose thecorrect emotion from a card on which the seven options are written.To reduce the effects of fatigue on our elderly, demented subjects, weadministered only items 1–14, for a maximum score of 14.

Neuropsychological/neuropsychiatric assessment

Each subject also underwent 2 h of cognitive testing, and ameasure of neuropsychiatric functioning was administered throughan informant interview. Standard neuropsychological measures oflanguage, visuospatial, memory, and executive functioning were used,and are detailed in Table 2. Subjects were also evaluated with theGeriatric Depression Scale (GDS), a 30-item self-report questionnaire(Yesavage et al., 1983). Behavior was measured using the Neuropsy-chiatric Inventory (NPI), a caregiver interview designed to assess thefrequency and severity of behaviors that commonly occur as a result ofa dementia syndrome (Cummings, 1997). Cognitive and neuropsy-chiatric assessment occurred within 3 months of sarcasm detectiontesting, and the average time between assessments was 20 days (SD±38 days).

Analysis of test performance

To determine whether subjects who performed poorly on theSimple Sarcasm task differed from the other subjects on any neuro-psychological or neuropsychiatric variables, patients were dividedinto two groups and directly compared. Patient scores on the SimpleSarcasm test were converted to z-scores using the healthy oldercontrol group as the standardization sample. The patients were thengrouped by whether they had passed or failed the Simple Sarcasmtask, with a cutoff for failure at zb−1.50 (i.e., patients wereconsidered to have failed the task if they performed at less than the7th percentile compared to healthy older normal controls, chosenbecause this level of performance signifies clinical impairment rela-tive to controls in standard neuropsychological assessment). t-tests,using age as a covariate, were used to compare the scores of the twogroups across all emotion, cognitive, and neuropsychiatric measures.

Structural MRI

MRI scans were obtained on a 1.5-T Magnetom VISION system(Siemens Inc., Iselin, N.J.) equipped with a standard quadrature headcoil. A volumetric magnetization prepared rapid gradient echo MRI(MPRAGE, TR/TE/TI=10/4/300 ms) was used to obtain T1-weightedimages of the entire brain, 15-degree flip angle, coronal orientationperpendicular to the double spin echo sequence, 1.0×1.0 mm2 in-plane resolution and 1.5 mm slab thickness.

Voxel-based morphometry

The voxel-based morphometry (VBM) technique utilizes an imagepre-processing step (spatial normalization, segmentation, modula-tion, and smoothing) followed by statistical analysis. Both stages wereperformed using the SPM5 software package (Wellcome Departmentof Cognitive Neurology, London; http://www.fil.ion.ucl.ac.uk/spm)running on Matlab 7.0.1 (MathWorks, Natick, MA). MRI images werepre-processed primarily using SPM5 default settings and tissueprobability maps, though light cleanup of partitions was performed.Spatially normalized, segmented, and modulated gray matter imageswere then smoothed with a 12 mm FWHM isotropic Gaussian Kernel.

2008 K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

VBM analyses of sarcasm processing

Covariates-only statistical models were used to show the relation-ship between TASIT scores and voxel-wise gray matter volume. Tocontrol for subjects' ability to comprehend the test, scores for bothconditions (Sincere and Simple Sarcasm) were entered into eachdesignmatrix, as were the confounding covariates age, sex, andMMSEscore (as a proxy for disease severity). Total intracranial volume (TIV)was used as a global covariate to correct for individual differences inhead size. Regionally specific differences in gray matter volumes ateach voxel were assessed using the general linear model, and thesignificance of each effect was determined using the theory ofGaussian fields (SPM5 defaults). Results were considered significantif they survived correction for family-wise error across the wholebrain (pFWEb0.05).

Main effect analyses: voxel-wise regression of gray matter on SSR scoreThe following contrasts were performed: 1) to look at the main

effect of paralinguistic sarcasm comprehension, controlling forperformance on the Sincere condition, a [0 1] t-contrast was used(with additional zeros for nuisance covariates), assuming that poorertest performance would be associated with decreased gray mattervolumes. 2) To determinewhether performance deficits on the Sincerecondition could be localized to a specific region, a [1 0] t-contrast wasused.

Table 2Neuropsychological and neuropsychiatric characteristics of non-normal subjects (N=77), s

Passed SSR task

(N=62)

GeneralAge 60.7 (7.5)Education 15.6 (2.8)Sex (M/F) 28/34Mini-mental State Examination 23.7 (5.5)

Neuropsychiatric/emotionGeriatric Depression Scale (max=30) 7.9 (5.9)Neuropsychiatric Inventory Total 19.0 (18.1)Comprehensive Affect Testing System (CATS)

Emotional Prosody Discrimination (max=22) 19.4 (3.2)Name Prosody Discrimination (max=22) 9.3 (4.0)

The Awareness of Social Inference Test (TASIT)Emotion recognition (max=15) 8.5 (2.8)Sincere (max=20) 14.6 (3.6)

LanguageBoston Naming Test 49.0 (14.0)Phonemic Fluency: FAS (SS) 6.4 (4.4)Semantic fluency (# animals/min) 11.3 (6.3)

VisuospatialModified Rey-O figure copy (max=17) 12.5 (5.0)Number location (max=10) 7.0 (2.7)WAIS-III block design (SS) 6.9 (4.0)

Verbal memoryCalifornia Verbal Learning Test — MS

Trials 1–4 total (max=36) 19.5 (8.5)30-second delay (max=9) 4.5 (2.7)10-min delay 3.8 (3.0)Recognition Hits 7.9 (1.4)

Non-verbal memoryModified Rey-O figure 10-min delay (max=17) 7.6 (5.0)Modified Rey-O figure recognition (yes/no) 37/24WMS-III visual reproductions: I (SS) 5.4 (4.2)WMS-III visual reproductions: II (SS) 7.2 (3.9)

Executive functioningWAIS-III digit span (SS) 8.2 (3.6)WAIS-III spatial span (SS) 6.3 (4.0)Modified trails speed (# lines/min) 11.0 (13.0)Design fluency (# designs/min) 5.9 (3.6)Stroop interference (# words/min) 23.3 (16.1)

F-statistics are derived from general linear models controlling for age, sex, and MMSE score

First co-atrophy error check: linear regression comparison of significantpeak voxels

Because regional atrophy is not randomly distributed across thissample, but is represented in patterns of atrophy that are similarwithin, and to some degree across, diagnostic categories, the maineffects analysis was expected to demonstrate some degree ofconfounding due to co-atrophy effects. This artifact, typical of VBMstudies in patients with neurodegenerative disease, occurs when brainregions are identified by VBM that are not directly related to (in thiscase) sarcasm score, but instead are the result of disease-specificpatterns of co-atrophy occurring along with regions truly related tosarcasm task performance. The superior quality of pre-processingafforded by the SPM5 software compared to SPM2 has the benefit ofincreasing the sensitivity of VBM analyses, but it simultaneouslyheightens the degree to which co-atrophy artifact appears, increasingthe importance of further analysis of main effect results.

To determine the relative contribution of the various regions foundto have an independent relationship to sarcasm in the massivelyunivariate main effects analysis, we performed a linear regressionanalysis of the voxel values at each peak coordinate using the SAS 9.1statistical program. Voxel probabilities were extracted from thesmoothed, warped, modulated, gray matter images of each subjectat each peak voxel that was significant in the main effects analysis.These voxel probability values were then analyzed together in linearregression analyses, including age, sex, MMSE, SIN, and TIV in each

tratified by performance on simple sarcasm recognition task (SSR). (SS=scaled score).

Failed SSR task

(N=15) F-value p-value

65.9 (8.8) 5.41 0.022717.3 (3.5) 3.82 n.s.10/5 χ2=2.23 n.s.23.4 (4.4) 0.05 n.s.

4.6 (4.4) 1.10 n.s.30.4 (21.2) 6.55 0.0128

20.1 (3.4) 0.35 n.s.8.2 (4.1) 0.15 n.s.

6.3 (2.0) 6.83 0.011017.1 (1.7) 6.13 0.0156

30.6 (22.8) 29.14 b0.00014.9 (2.2) 2.17 n.s.6.5 (4.3) 11.06 0.0014

14.6 (1.3) 1.92 n.s.9.2 (0.6) 10.00 0.00248.2 (3.8) 1.22 n.s.

17.1 (6.1) 1.50 n.s.3.6 (2.8) 1.87 n.s.2.9 (3.2) 1.90 n.s.6.7 (2.7) 8.23 0.0054

8.7 (4.6) 0.31 n.s.10/3 χ2=1.22 n.s.6.5 (4.4) 0.57 n.s.6.3 (2.8) 1.24 n.s.

11.1 (2.8) 11.07 0.00149.6 (3.8) 9.23 0.0034

15.7 (10.1) 2.82 n.s.7.9 (3.3) 3.07 n.s.

29.3 (15.3) 4.10 0.0471

.

Fig. 1. Bar graph representing the performance of each diagnostic group on both theSincere (SIN) condition and the Simple Sarcasm (SSR) condition of the TASIT. No groupdiffered significantly from healthy older control subjects on SIN, but the SemanticDementia group performed significantlyworse on SSR. Error bars represent±1 SD fromthe mean.

2009K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

model as potentially confounding covariates, and using SSR score asthe outcome variable. We used the modified Allen-Cady predictorselection technique specified in Vittinghoff et al. (2004), forcing age,sex, MMSE, SIN, and TIV into the model as covariates, and setting avery permissive inclusion threshold at pb0.20 to ensure that brainregions showing at least a modest independent relationship to SSRscore remained in the model.

Second co-atrophy error check — shared effects analysis: voxel-wiseregression of gray matter on SSR score controlling for diagnosticgroup membership

The linear regression analysis uses a data-driven approach toidentify and reject brain regions that appear in the VBM main effectresults simply because they are statistically more likely to co-atrophywith other brain regions directly related to sarcasm. However, thisanalysis does not rule out the possibility that significant findings holdtrue only in one diagnostic group and do not represent a generalizablebrain–behavior relationship. It is logically possible for this kind ofillusory correlation to occur in any VBM analysis using patients frommultiple neurodegenerative disease groups, because if disease groupmembership predicts a region of atrophy (G→A), and disease group

Table 3Voxel-basedmorphometry analyses of main effects: regionswhere simple sarcasm detectionlanguage comprehension (SIN), age, sex, MMSE score, and total intracranial volume.

Anatomic region BA Cluster size (mm3) x

Main effectsR superior temporal pole 38 22,240 46R parahippocampal gyrus 36 q 24R middle temporal gyrus 21 q 68L parahippocampal gyrus 35 17,472 −16q 30 q −22L superior temporal pole 38 q −36L inferior temporal pole 20 384 −46Caudate head – 680 0R superior frontal gyrus 10 88 28q 9 q 18

Shared effects (generalizing across diagnostic group)R superior temporal pole 38 1680 48R parahippocampal gyrus 30 2192 22L parahippocampal gyrus 35 728 −12R superior frontal gyrus 10 184 28q 9 120 22R middle temporal gyrus 21 72 72R caudate head – 80 6

Sharedeffects analysis also includes all 7 diagnostic groups, parameterized into binarycodes (yesdiagnostic group. All VBM results are corrected for family-wise error across thewhole brain at aability to predict SSR score in linear regressionmodeling, with estimated standardized regressio

membership also predicts poor performance on the behavior task(G→B), then that region of atrophy may appear to directly correlatewith the behavior (A↔B), when that correlation is actually spurious(A?B).

In order to perform a second error check for co-atrophy, weparameterized each diagnosis (0=no, 1=yes) and entered all 7 diag-nostic groups into the design matrix as confounding covariates (using 6dummy variables to represent the 7 groups). Then the relationshipbetween sarcasm score and atrophywas examined using a [0 10 0 0 0 00 0 0 0] contrast (see Fig. 2). The results of this analysis show regions ofatrophysignificantly related to Sarcasmscoreonly if theyappear inmorethan one diagnostic group. These results must be considered in light ofthe regression results, however, because this method will improperlyexclude any regions that are legitimately related to Sarcasm score, butwhich only atrophy in a single diagnostic group.

Hypotheses

Based on evidence from functional and lesion-based studies ofsocial and emotional functions, we hypothesized that deficits in theability to interpret paralinguistic cues as sarcastic would correspondwith gray matter atrophy in a right temporal–frontal network (Allisonet al., 2000; Gallagher and Frith, 2003; Perry et al., 2001; Rosen et al.,2002; Shamay-Tsoory, 2005a). Poor performance on the more generalSincere control condition was expected to occur because of deficits indifferent cognitive modalities, according to the functions particularlyaffected by any one of the multiple neurodegenerative diseasesrepresented in our sample (e.g., impaired verbal or visual memory,working memory, semantic loss, or syntax comprehension deficits).Because detection across the sample of the source of failure on thistask would not be isolated to a single anatomic network, since nosingle diagnostic group showed disproportionate deficits on theSincere task, we hypothesized that there would be no suprathresholdclusters for the Sincere condition main effect.

Results

Behavioral results

Sarcasm comprehensionAn omnibus analysis of variance using a general linear model,

controlling for sex, age, and MMSE, showed no significant diagnostic

(SSR) scores positively correlatedwith gray matter tissue density, controlling for sincere

y z t-score pFWE beta VIF

22 −24 6.31 0.000 – –

−14 −32 5.97 0.000 0.16 2.4−4 −20 5.77 0.001 – –

−16 −24 6.11 0.000 – –

−26 −16 5.79 0.001 0.26 2.426 −26 5.72 0.001 – –

12 −42 5.11 0.010 0.25 1.94 6 5.03 0.013 – –

64 16 4.87 0.022 0.28 1.952 26 4.62 0.049 – –

22 −22 5.21 0.009−22 −20 5.21 0.009−16 −26 5.10 0.01362 18 5.07 0.01552 32 4.88 0.028

−34 −6 4.91 0.0264 6 4.82 0.034

=1, no=0), as covariates in thedesignmatrix to remove results significantonly in a singlesignificance level of pb0.05.Multivariate regression analyses: areas demonstrating uniquen coefficients (beta-weights) and ameasure of collinearity (VIF: variance inflation factor).

2010 K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

group differences in how subjects performed on the Sincere condition,suggesting that all disease groups were able to adequately compre-hend the test despite their cognitive deficits. However, there weresignificant differences across groups on Simple Sarcasm score(pb0.0007) (Table 1, Fig. 1). SemD patients showed significantlylower Simple Sarcasm scores than controls (pb0.05 based on a post-hoc Dunnett–Hsu test controlling for sex, age, and MMSE). No otherdementia group (bvFTD, PNFA, AD, CBD, PSP) showed impairment on

Fig. 2.Designmatrices and transparent axial views of SPM5 “glass brain” representing resultsand shared effect analysis (additionally controlling for confounding effects of diagnostic grouvolume (results shown corrected at pFWEb0.05). Shadowed regions represent regions whe

either condition relative to normal controls. Sincere scores did notcorrelate significantly with Simple Sarcasm scores.

Neuropsychological/neuropsychiatric/emotion testingThe patients were then grouped by whether they had passed or

failed the Simple Sarcasm task, and the cognitive, emotional, andneuropsychiatric profiles of the two groups were compared (Table 2).Patients failing the Sarcasm task included 4 bvFTDs, 8 SemDs, 2 ADs,

of main effect analysis (controlling for Sincere condition score, MMSE, age, sex, and TIV)p) examining the relationship between Simple Sarcasm (SSR) condition and graymatterre atrophy significantly correlated with poorer performance on the SSR task.

2011K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

and 1 PSP patient. The “Fail” group performed significantly worse thanthe “Pass” group on tests of dynamic emotion recognition (TASIT EET),confrontation naming, semantic fluency, and verbal recognitionmemory, and they showed a significantly more impaired neuropsy-chiatric profile on the NPI. However, the “Fail” group performedsignificantly better than the “Pass” group on tests of visuospatialfunctioning, verbal and non-verbal working memory, and ability toinhibit an automatic verbal response. Also, patients in the “Fail” groupwere statistically more likely to give correct interpretations of videosfrom the Sincere condition of the TASIT. There were no differencesbetween the “Pass” and “Fail” groups on the simple emotional voiceprosody naming or recognition tasks (CATS), or on other neuropsy-chological tests.

Neuroimaging results

Main effects analysis resultsThemain effect of simple sarcasm comprehension (Simple Sarcasm

score controlling for Sincere score) included voxels at the bilateraltemporal poles, bilateral parahippocampal gyri, the right middletemporal gyrus, the right superior frontal gyrus, and the head of thecaudate (pb0.05, FWE) (Table 3 and Fig. 2). These results demon-strated a pattern very similar to the regions of the right and lefttemporal lobe frequently affected in SD, probably as a result of thepredicted co-atrophy artifact (see Materials and methods). Plots ofvoxel intensity at each of the ten significant peak voxels against totalSimple Sarcasm score showed no outliers on the independent variable,and sarcasm detection scores were widely distributed throughout therange of voxel intensities, demonstrating no outliers and suggestingthat there was no restriction of range. Analysis of the main effect ofperformance on the Sincere task showed no significant voxels.

First co-atrophy error check: results of linear regression comparing ofsignificant regions

Voxel probability scores were extracted from the smoothed,modulated, normalized gray matter images at each of the ten peak

Fig. 3. Results of main effects analysis (green) and shared effect analysis (yellow) of Simple−16), and axial (z=18, z=−22) slices of an averaged template brain (SPM5: t1.nii). Greperformance on the SSR task, controlling for sincere score, MMSE, age, sex, and TIV. Yellow cowas added into the design matrix, indicating regions that showed a significant relationshipcorrected level of significance (pFWEb0.05) using the t-score range shown at the bottom r

voxels identified in the main effect analysis. These included the rightsuperior temporal pole (RSTP), the right parahippocampal gyrus(RPHG), the right middle temporal gyrus (RMTG), two peaks withinthe left parahippocampal gyrus (LPHG1, LPHG2), the left superiortemporal pole (LSTP), and left inferior temporal pole (LITP), the headof the caudate (CH), and two peaks within the right superior frontalgyrus (RSFG1, RSFG2) (Table 3). When these regions and Sarcasmscore were entered into a partial correlation matrix, controlling forsex, age, MMSE, SIN, and TIV, all regions were significantly correlatedwith each other, and Sarcasm score, at pb0.01, with the strength ofcorrelations ranging from r=0.27 to r=0.81. A linear regression wasperformed inwhich variables with no discernable unique relationshipto Sarcasm score were removed (see Materials andmethods), as thesevariables may have been significant in the Main Effects result becauseof disease-specific co-atrophy patterns, rather than because of a directrelationship to sarcasm comprehension. The variables that remainedfor analysis included the RPHG (24, −14, −32), LPHG2 (−22, −26,−16), LITP (−46, 12, −42), and RSFG1 (28, 64, 16).

Second co-atrophy error check: shared effects analysis resultsPerforming the main effects analysis described above, but

controlling for diagnostic group membership, the left temporal loberegions dropped out of the analysis, and more focal correlationsbetween Sarcasm score and atrophy were seen in the right superiortemporal pole, right caudate, and bilateral parahippocampal gyri. Thepeaks in the right superior frontal gyrus and right posterior middletemporal gyrus increased in magnitude, as measured by their t-score(pb0.05, FWE) (see Table 3 and Figs. 2 and 3).

Results summary

The main effects analysis yielded significant results throughoutboth anterior temporal lobes that appeared similar to the atrophypattern seen in the SemD patients, who were most likely to fail thesarcasm task. However, the two co-atrophy error checks reduced theseareas to the regions most likely to have a significant relationship with

Sarcasm (SSR) condition score, superimposed on sagittal (x=28, x=46), coronal (y=en colored areas represent regions where atrophy significantly correlated with poorerlored areas represent regions remaining significant when diagnostic groupmembershipto sarcasm interpretation in more than one diagnostic group. Results are shown at a

ight.

2012 K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

sarcasm score, statistically independent of other significant brainregions and of diagnostic groupmembership. The areas surviving botherror checks included regions in the right and left parahippocampalgyri and the right superior frontal gyrus. The left temporal polesurvived the linear regression error check, but dropped out of theshared effects analysis, most likely because this region is atrophic inonly one diagnostic group, SemD. The right superior temporal polesurvived the shared effects error check, but did not survive the linearregression error check, potentially because it was highly collinear withthe homologous left temporal pole, decreasing the effectiveness of theregression analysis to segregate each region's independent contribu-tion. Thus, though the relative contribution of left versus righttemporal pole is unclear, there is a high likelihood that one or bothregions showed a significant independent relationship to sarcasmscore. Other regions (inferior temporal regions, caudate, posteriormiddle temporal gyrus) failed at least one error check, thus theevidence supporting their independent relationship to sarcasm scoreis weaker.

Discussion

VBM was used in patients with neurodegenerative disease andhealthy older adults to correlate MRI-derived brain volumes with ameasure of the ability to detect sarcasm based on paralinguistic cues.The primary finding was that lower scores on sarcasm recognitioncorresponded most significantly with atrophy to the temporal lobesbilaterally, particularly the parahippocampal gyri and the temporalpoles, as well as the right superior frontal gyrus. Subjects who failedthe sarcasm recognition task performed more poorly on realisticallydynamic, but not simple static emotion recognition tasks, and hadmore neuropsychiatric disturbances. However, they performed sig-nificantly better than other patients at correctly interpreting sincerecommunication, and had significantly better verbal and visuospatialworking memory. Previous studies have demonstrated a link betweenthe right hemisphere and the ability to comprehend paralinguisticprosody in patients with neurological disorders (Brown et al., 2005;Cutica et al., 2006; Pell, 2007). However, this is the first study to usequantitative image analysis to more precisely link paralinguisticcomprehension deficits with damage to specific brain structures.

In order for subjects to perform the sarcasm task used in this study,they were required to answer yes–no questions about the thoughts,words, feelings, and actions of the speakers. Subjects in all groupswere able to answer a high proportion of these questions correctlywhen the speakers' intentions were sincere, thus demonstratingadequate memory, working memory, syntax comprehension, andsemantic comprehension to perform the task. However, a subset ofpatients answered questions on the sarcastic items as if they believedthe speakers to be sincere. The breakdown of subjects' ability tocomprehend paralinguistic sarcasm may have occurred at differentlevels, including 1) initial failure to correctly process vocal and visualinformation so that the paralinguistic profile is recognized as atypical,2) failure to interpret the atypical paralinguistic profile as a cue thatthe social context has shifted and that the speaker's statements shouldno longer be interpreted literally, but require additional downstreamprocessing, or 3) failure to correctly infer the speaker's non-literalmeaning, including their thoughts, intentions, and emotions. Becausethis study used clinical patients and an atrophy model, rather than adynamic model of normal function using fMRI, the study design doesnot allow unequivocal identification of the level or levels at whichsubjects' failure occurred. However, examination of the knownfunctions of the regions found in this study suggests that differentsubsets of patients may have experienced breakdown primarily at thelatter two stages, involving more complex downstream interpretationof the speaker's meaning, while early, upstream processing of supra-segmental vocal cues may have remained largely intact. This inter-pretation is also supported by the finding that the patient group failing

the Sarcasm condition was no more likely than the passing group tohave difficulty with discriminating or naming emotional prosody onsimple auditory testing.

Recognizing social context from paralinguistic cues

Studies have established that the distinct auditory profile of sar-castic speech compared to sincere speech includes a higher funda-mental frequency (fo) with a greater range amplitude, higher energyvalues, shorter pauses, and lengthened syllables (Anolli et al., 2000).These studies suggest that sarcasm is primarily identified based on itstemporal and spectral vocal features, functions associated with thetemporal lobes (Beaucousin et al., 2007; Belin et al., 2000;Wildgruberet al., 2006). Our study found that sarcasm comprehension decreasedin conjunction with atrophy to specific regions of the temporal lobethat may be involved in social signal detection and higher-level con-ceptual processing.

Posterior parahippocampal cortexThough not commonly identified as a region involved in processing

social stimuli, volume in the parahippocampal cortex (PHc) showed astrong relationship with the ability to correctly interpret sarcasticcommunication. The PHc is a higher-order polymodal association areawith strong afferent and efferent connections to temporal, parietal,and frontal cortices. The more lateral region of the PHc (area TF)receives visuospatial information from parietal area 7 (in the dorsalstream or “where is it” pathway), along with strong unimodal inputsfrom visual areas V4, TEO, and TE, as well as projections from theagranular, dysgranular, and granular portions of the dorsal insula. Themore medial PHc (area TH) receives significant projections fromauditory association cortex in the superior temporal gyrus, as well asprojections from the parainsular cortex. TF and TH both receive subs-tantial projections from dorsal frontal areas (BA 46 and 9), the rostralportions of the anterior cingulate (BA 24 and 32) and the retrosple-nium (BA 30 and 29) (Suzuki and Amaral, 1994). The PHc also showsstrong reciprocal projections to all of these temporal, parietal, frontal,and insular regions (Lavenex et al., 2002). The PHc is involved inencoding and retrieving information about contexts, and its functionis distinct from that of anterior perirhinal cortex, which is involved inobject memory. A recent review of functional imaging studies exa-mining this area suggests that it is involved in encoding and retrievingcontextual information (Diana et al., 2007). For instance, the PHcactivates bilaterally when subjects view familiar objects in a novelvisuospatial arrangement, but not in response to novel objects if thespatial configuration remains the same (Pihlajamäki et al., 2004). Onestudy showed bilateral PHc activity when subjects were discriminat-ing correct and incorrect background features of auditory stimuli onretrieval (i.e., whether the speaker's voice was male or female at thetime words were encoded), but not of visually encoded information(i.e., whether the background texture to a picture was a lawn orclouds) (Peters et al., 2007). Patients with resections of either theright or left PHc do not recognize dissonant musical compositions asmore unpleasant than consonant music (Gosselin et al., 2006),suggesting that the PHc may be involved in deriving positive andnegative valence from complex auditory cues, and that PHc damagemay prevent listeners from recognizing that dissonance represents achange in the overall mood of the music. This finding has a clearanalogy to the interpretation of sarcastic vocal prosody.

We propose that the significant relationship between volume lossin the PHc and the inability to recognize sarcasm in this study maysuggest that many subjects primarily failed at the level of social signaldetection. If the paralinguistic input to the PHc from superior temporalauditory association cortex is typical, then the background commu-nicative context associated with the spoken words is assumed to besincere, and the speaker's thoughts and feelings are interpreted asmatching their words. However, if the paralinguistic input is recog-

2013K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

nized by the PHc as atypical, particularly if the vocal prosody adoptsthe dissonant and unpleasant cadences inherent to sarcasm, this alertsthe listener that the background communicative context has changed.Patients who do not recognize that the sarcastic speaker is signaling achange of interpretive context will continue in the default mode ofcommunication, incorrectly assuming that the speaker's words aresincere, and will thus fail to initiate the additional downstreamprocessing required to correctly interpret the speaker's paradoxicalstatements. The fact that subjects failing the sarcasm detection taskdid not perform worse than other subjects on simple prosody discri-mination tasks suggests that they may be able to hear the prosody ofthe sarcasm, but still fail to recognize that it has any social importance.Not only have vocal cues more consistently and strongly been asso-ciated with identification of sarcasm in normal subjects than visualfeatures (Rockwell, 2001, 2007), but volume loss to the most medialTH region of the PHc, an area much more extensively interconnectedwith auditory than visual cortex, correlated more strongly with failureto detect sarcasm in our study. The presence of extensive afferent andefferent connections between the PHc and both insular anddorsomedial frontal cortex suggests that the PHc may provide directbottom-up input to regions involved in higher social and emotionalprocessing, or these anterior regions may exert a top-down influenceon the PHc in reinforcing the social salience of the sarcastic para-linguistic profile. Clearly, however, this study was not designed toidentify the specific role of the PHc in sarcasm comprehension, only toestablish the correlation, thus these proposed functions are specu-lative and should be subject to further investigation in other researchmodalities.

Temporal polesThough both temporal poles were significantly related to sarcasm

processing in the main effects analysis, additional analysis was unableto clarify the relative contributions of the left versus right temporalpole. Like the frontal pole, the temporal poles are made up of tertiaryassociation cortex (Mesulam, 1998) and thus are liable to be involvedin downstream processing of sarcasm, most likely at the stage ofconceptual interpretation once the paralinguistic profile of sarcasmhas been detected. Both temporal poles seem to be involved in higher-level conceptual knowledge (Ralph et al., 2008), and the type ofinformation processed by either pole appears to be partly dependenton the modality of the input from that side of the brain (e.g., social-emotional vs. linguistic). The right temporal pole is associated withprocessing social and emotional information (Olson et al., 2007; Phanet al., 2002) and has been linked with the ability to generate anempathic response (Rankin et al., 2006). It is likely involved in high-level integration of social and emotional signals based on multipleexternal and internally generated sources of information, yieldinghigher-level social conceptual knowledge (Zahn et al., 2009). Patientswith damage to this areamay have beenmore likely to fail the sarcasmtask due to inability to correctly read the speaker's emotional and so-cial intent, even if they did recognize the paralinguistic speech profileas abnormal. The left temporal pole is involved in linguistic semanticnetworks, and damage to this area in patients with neurodegenerativedisease results in patients' loss of word and object knowledge (Murreet al., 2001). Patients demonstrated normal comprehension of thepost-video questions in the sincere condition, and our analyses con-trolled for performance on the Sincere task in order to removevariance associatedwith simple comprehension deficits. However, it ispossible that correct interpretation of the sarcastic videos may haverequired a higher-level of conceptual semantic information that didnot directly concern the meaning of words in the videos or post-videoquestions, which was accounted for by sincere control task perfor-mance. Functional imaging shows that knowledge of higher-ordersocial concepts is primarily associated with bilateral anterior temporalcortex (Zahn et al., 2007), and loss of social conceptual knowledge inbvFTD patients has been associated with damage to the right dorsal

anterior temporal lobe region also seen in our analysis (Zahn et al.,2009). Sarcasm is a learned socio-linguistic construct (Laval and Bert-Eboul, 2005), and perhaps loss of the acquired knowledge thatlanguage can be insincere limited patients' comprehension.

Interpreting paradoxical intentions in sarcasm

Right medial frontal poleOur study did find a small but significant cluster of atrophy in the

right superior frontal gyrus, corresponding to Brodmann's area 10,that correlated with sarcasm task performance. This area is signifi-cantly downstream from the initial steps of processing the para-linguistic features of the communication, and is more likely to beinvolved in recognizing that the sarcastic speaker intends to convey ameaning other than their words would suggest. In a recent review,Kreuger et al. (2009) suggest that the anterior medial prefrontalcortex is involved in interpreting low-frequency social scripts, andderiving intentions and event outcomes from goal-directed actionsequences. The frontal pole has repeatedly been linked with socialperspective taking, a skill that is likely involved in the ability tocorrectly interpret a sarcastic speaker's paradoxical intentions. In areview of evidence from clinical and neuroimaging studies, Decety(Decety and Jackson, 2004) suggests that dorsomedial frontal areasmay facilitate perspective taking by inhibiting the default self-perspective, in order to temporarily attend to and make inferencesabout the other's point of view. These areas may also be involved inperforming a “triadic attention” task, in which one's own perspective,the other's perspective, and reality must be held online and compared(Gallagher and Frith, 2003; Saxe, 2006; Zysset et al., 2002). Studies ofbrain injured patients also have documentedworse perspective takingand irony detection in patients with lesions to the dorsomedial andmedial frontopolar areas including BA 10 (Shamay-Tsoory et al., 2003,2005b, 2004; Stone et al., 1998; Stuss et al., 2001).

The fact that this study saw significant dorsomedial, but notdorsolateral, correlations in the frontal cortex is consistent with thebehavioral finding that executive impairment did not predict sarcasmcomprehension failure in this group of patients. In fact, patients whofailed the sarcasm task had significantly better working memory andresponse inhibition than patients who passed. If this study hadperformed a text-based task, or a task basing interpretation of sarcasmon other non-paralinguistic contextual cues, subjects' performancemight have correlated with executive deficits in areas such as workingmemory and complex non-verbal reasoning.

Clinical implications for neurodegenerative disease

These results raise the question of whether particular aspects ofthis pathway are differentially affected within specific disease groups.However, single diagnostic group analyses of behavioral correlatesusing VBM are methodologically unsound, because severe restrictionof range in both anatomic and behavioral diversity results in inade-quate power for unbiased whole brain analyses. As a result, this studycould not directly examine neural correlates of sarcasm within singlediagnostic groups.

Technical limitations notwithstanding, these data do imply thatcertain brain–behavior relationships exert a differential impact acrossdisease groups. Right temporal damage, particularly in the context ofneurodegenerative disease, has been associated with emotioncomprehension deficits (Rosen et al., 2004), disturbed emotionalexpression (Mendez et al., 2006; Miller et al., 1993), lack of empathy(Mendez and Perryman, 2003; Rankin et al., 2006, 2005), and generalloss of social sensitivity (Gorno-Tempini et al., 2004; Perry et al.,2001). In this study, SemD patients were more likely to fail thesarcasm recognition task than any other group, including bvFTDpatients, though our anatomic analysis implicated both left and righttemporal structures in this deficit. The semantic dementia subtype is

2014 K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

diagnosed primarily on the basis of left-temporally-mediated lan-guage symptoms such as loss of word knowledge. These languagecomprehension deficits should presumably have caused SemD pa-tients in this study to attendmore closely to the non-verbal features ofthe videos, but instead, their performance suggests that they took thespeakers' words at face value, ignoring the sarcastic paralinguisticprofile. Unexpectedly, these early SemD patients outperformednormal control subjects in their ability to correctly respond toquestions about characters' thoughts, feelings, words, and actions inthe sincere videos, suggesting more than adequate word comprehen-sion for the task. The association between sarcasm comprehensionand the posterior parahippocampal region suggests that some SemDpatients may have failed to recognize the paralinguistic profile ofsarcasm as a dissonant, unpleasant signal indicating that the socialcontext had changed and required more careful attention and inter-pretation. Anterior temporal lobe damage causing loss of the learnedsocial concept of “sarcasm” may also have biased SemD patients to-wards interpreting the communications as sincere.

While temporal lobe damage is ubiquitous in SemD, behavioralvariant FTD patients show a wide variety of neuroanatomic diseasepatterns, with only a subset of patients developing temporal lobeinvolvement. However, the right superior frontal gyrus is moreconsistently affected in bvFTD, and this structure was also found to beimportant for sarcasm comprehension. Considering the anatomicdiversity across bvFTD, patients with temporal damage may fail tocomprehend sarcasm because of upstream deficits in paralinguisticprocessing, or semantic loss of the social concept of “sarcasm”. On theother hand, patients without temporal damage may show downstreamfailure inferring intentions (“Theory of Mind”) due to dorsomedialfrontal damage reducing their ability to imagine the speaker'sperspective, and hold it online for analysis. Though 20% of the bvFTDpatients failed the sarcasm task, the bvFTD group as a whole did notshow statistically significant differences from healthy older controlsubjects with these stimuli, which primarily relied upon paralinguisticrather than contextual cues. In a recent study using stimuli thatcombined paralinguistic with contextual cues, bvFTD patients showedmuch greater impairment in sarcasm comprehension (Kipps et al.,2009). This suggests that if bvFTD patients were studied in a paradigmasking them to identify sarcasm based purely upon contextual cues, amore developmentally and executively complex skill (Laval and Bert-Eboul, 2005), they would be more likely to fail.

Conclusions

Using VBM with structural MRI images, the anatomic substrate ofthe ability to recognize the paralinguistic profile of sarcastic speechwas delineated in patients with neurodegenerative disease andhealthy older controls. This study provides lesion data suggestingthat the posterior parahippocampus may be involved in recognizing aparalinguistic speech profile as abnormal, which in turn activatesinterpretive processing by the temporal poles and right medial frontalpole that can access identify the social context as sarcastic, andrecognize the speaker's paradoxical intentions.

Acknowledgments

This research was supported in part by the National Institute onAging (NIA) grants 5-K23 AG021606, 5-R01 AG029577 and 5-P01AG19724, the State of California, Alzheimer's Disease Research Centerof California (ARCC) grant 03-75271, and the Larry L. HillblomFoundation, Inc., grant 2002/2J.

References

Allison, T., Puce, A., McCarthy, G., 2000. Social perception from visual cues: role of theSTS region. Trends Cogn. Sci. 4 (7), 267–278.

Anolli, L., Ciceri, R., Infantino, M.G., 2000. Irony as a game of implicitness: acousticprofiles of ironic communication. J. Psycholinguist. Res. 29 (3), 275–311.

Attardo, S., Eisterhold, J., Hay, J., Poggi, I., 2003. Multimodal markers of irony andsarcasm. Humor: Int. J. Humor Res. 16 (2), 243–260.

Bara, B.G., Bucciarelli, M., Geminiani, G.C., 2000. Development and decay of extra-linguistic communication. Brain Cogn. 43 (1), 21–27.

Beaucousin, V., Lacheret, A., Turbelin, M.R., Morel, M., Mazoyer, B., Tzourio-Mazoyer, N.,2007. FMRI study of emotional speech comprehension. Cereb. Cortex 17, 339–352.

Belin, P., Zatorre, R.J., Lafaille, P., Ahad, P., Pike, B., 2000. Voice-selective areas in humanauditory cortex. Nature 403, 309–312.

Boxer, A.L., Geschwind, M.D., Belfor, N., Gorno-Tempini, M.L., Schauer, G.F., Miller, B.,2005. Patterns of brain atrophy differentiate corticobasal degeneration fromprogressive supranuclear palsy. Arch. Neurol.

Brown, W.S., Symington, M., Van Lancker-Sidtis, D., Dietrich, R., Paul, L.K., 2005. Para-linguistic processing in children with callosal agenesis: emergence of neurolin-guistic deficits. Brain Lang. 93, 135–139.

Champagne, M., Virbel, J., Nespoulos, J.-L., Joanette, Y., 2003. Impact of right hemisphericdamage on a hierarchy of complexity evidenced inyoung normal subjects. Brain Cogn.53, 152–157.

Channon, S., Pellijeff, A., Rule, A., 2005. Social cognition after head injury: sarcasm andtheory of mind. Brain Lang. 93, 123–134.

Channon, S., Rule, A., Maudgil, D., Martinos, M., Pellijeff, A., Frankl, J., et al., 2007.Interpretation of mentalistic actions and sarcastic remarks: effects of frontal andposterior lesions on mentalizing. Neuropsychologia 45, 1725–1734.

Cummings, J.L., 1997. The Neuropsychiatric Inventory: assessing psychopathology indementia patients. Neurology 48 (5 Suppl. 6), S10–16.

Cutica, I., Bucciarelli, M., Bara, B.G., 2006. Neuropragmatics: extralinguistic pragmaticability is better preserved in left-hemisphere-damaged patients than in right-hemisphere-damaged patients. Brain Lang. 98, 12–25.

Decety, J., Jackson, P.L., 2004. The functional architecture of human empathy. Behav.Cogn. Neurosci. Rev. 3 (2), 71–100.

Dennis, M., Purvis, K., Barnes, M.A., Wilkinson, M., Winner, E., 2001. Understanding ofliteral truth, ironic criticism, and deceptive praise following childhood head injury.Brain Lang. 78 (1), 1–16.

Dews, S., Winner, E., 1995. Muting the meaning: a social function of irony. MetaphorSymb. Act. 10 (1), 3–19.

Diana, R.A., P.Yonelinas, A., Ranganath, C., 2007. Imaging recollection and familiarity inthe medial temporal lobe: a three-component model. Trends Cogn. Sci. 11 (9),379–386.

Eviatar, Z., Just, M.A., 2006. Brain correlates of discourse processing: an fMRI investi-gation of irony and conventional metaphor comprehension. Neuropsychologia 44,2348–2359.

Froming, K.B., Ekman, P., Levy, M., 2001. Comprehensive Affect Testing System.Gainsville, FL, Psychology Software, Inc.

Gallagher, H.L., Frith, C.D., 2003. Functional imaging of ‘theory of mind’. Trends Cogn.Sci. 7 (2), 77–83.

Gorno-Tempini, M.L., Dronkers, N.F., Rankin, K.P., Ogar, J.M., Phengrasamy, L., Rosen, H.J.,et al., 2004. Cognitive and anatomy in three variants of primary progressive aphasia.Ann. Neurol. 55, 335–346.

Gorno-Tempini, M.L., Rankin, K.P., Woolley, J.D., Rosen, H., Phengrasamy, L., Miller, B.,2004. Cognitive and behavioral profile in a case of right anterior temporal lobeneurodegeneration. Cortex 40, 631–644.

Gosselin, N., Samson, S. v., Adolphs, R., Noulhiane, M., Roy, M., Hasboun, D., et al., 2006.Emotional responses to unpleasant music correlates with damage to the parahi-ppocampal cortex. Brain 129 (10), 2585–2592.

Hodges, J.R., Patterson, K., 2007. Semantic dementia: a unique clinicopathologicalsyndrome. Lancet Neurol. 6 (11), 1004–1014.

Jorgensen, J., 1996. The functions of sarcastic irony in speech. J. Pragmat. 26 (5),613–634.

Kipps, C.M., Nestor, P.J., Acosta-Cabronero, J., Arnold, R., Hodges, J.R., 2009. Understandingsocial dysfunction in the behavioral variant of frontotemporal dementia: the role ofemotion and sarcasm processing. Brain 132 (3), 592–603.

Kreuger, F., Barbey, A.K., Grafman, J., 2009. The medial prefrontal cortex mediates socialevent knowledge. Trends Cogn. Sci. 13 (3), 103–109.

Laval, V., Bert-Eboul, A., 2005. French-speaking children's understanding of sarcasm:the role of intonation and context. J. Speech, Lang. Hear. Res. 48, 610–620.

Lavenex, P., Suzuki, W.A., Amaral, D.G., 2002. Perirhinal and parahippocampal corticesof the macaque monkey: projections to the neocortex. J. Comp. Neurol. 447 (4),394–420.

Leitman, D.I., Ziwich, R., Pasternak, R., Javitt, D.C., 2006. Theory of Mind (ToM) andcounterfactuality deficits in schizophrenia: misperception or misinterpretation?Psychol. Med. 36 (8), 1075–1083.

Litvan, I., Agid, Y., Calne, D., Campbell, G., Dubois, B., Duvoisin, R.C., et al., 1996. Clinicalresearch criteria for the diagnosis of progressive supranuclear palsy (Steele–Richardson–Olszewski syndrome): report of the NINDS-SPSP international work-shop. Neurology 47 (1), 1–9.

Martin, I., McDonald, S., 2004. An exploration of causes of non-literal languageproblems in individuals with Asperger syndrome. J. Autism Dev. Disord. 34 (3),311–328.

McDonald, S., 1996. Clinical insights into pragmatic theory: frontal lobe deficits andsarcasm. Brain Lang. 53, 81–104.

McDonald, S., 1999. Exploring the process of inference generation in sarcasm: a reviewof normal and clinical studies. Brain Lang. 68, 486–506.

McDonald, S., Flanagan, S., Rollins, J., Kinch, J., 2003. TASIT: a new clinical tool forassessing social perception after traumatic brain injury. J. Head Trauma Rehabil. 18(3), 219–238.

2015K.P. Rankin et al. / NeuroImage 47 (2009) 2005–2015

McDonald, S., Bornhofen, C., Shum, D., Long, E., Saunders, C., Neulinger, K., 2006.Reliability and validity of The Awareness of Social Inference Test (TASIT): a clinicaltest of social perception. Disabil. Rehabil. 28 (24), 1529–1542.

McDonald, S.F.S., Rollins, J., 2002. Awareness of Social Inference Test, the (TASIT).Harcourt Assessment.

McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., Stadlan, E.M., 1984.Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDAWork Groupunder the auspices of Department of Health and Human Services Task Force onAlzheimer's disease. Neurology 34 (7), 939–944.

Mendez, M.F., Perryman, K.M., 2003. Disrupted facial empathy in drawings from artistswith frontotemporal dementia. Neurocase 9 (1), 44–50.

Mendez, M.F., McMurtray, A., Chen, A.K., Shapira, J.S., Mishkin, F., Miller, B.L., 2006.Functional neuroimaging and presenting psychiatric features in frontotemporaldementia. J. Neurol. Neurosurg. Psychiatry 77 (1), 4–7.

Mesulam, M., 1998. From sensation to cognition. Brain 121, 1013–1052.Miller, B.L., Chang, L., Mena, I., Boone, K., Lesser, I.M., 1993. Progressive right fronto-

temporal degeneration: clinical, neuropsychological and SPECT characteristics.Dementia 4 (3–4), 204–213.

Murre, J.M., Graham, K.S., Hodges, J.R., 2001. Semantic dementia: relevance toconnectionist models of long-term memory. Brain 124 (Pt. 4), 647–675.

Neary, D., Snowden, J.S., Gustafson, L., Passant, U., Stuss, D., Black, S., et al., 1998. Fronto-temporal lobar degeneration: a consensus on clinical diagnostic criteria. Neurology 51(6), 1546–1554.

Olson, I.R., Plotzker, A., Ezzyat, Y., 2007. The enigmatic temporal pole: a review offindings on social and emotional processing. Brain 130 (7), 1718–1731.

Pell, M.D., 2007. Reduced sensitivity to prosodic attitudes in adults with focal righthemisphere brain damage. Brain Lang. 101, 64–79.

Perry, R.J., Rosen, H.R., Kramer, J.H., Beer, J.S., Levenson, R.L., Miller, B.L., 2001. Hemisphericdominance for emotions, empathy and social behaviour: evidence from right and lefthanders with frontotemporal dementia. Neurocase 7 (2), 145–160.

Peters, J., Suchan, B., Köster, O., Daum, R., 2007. Domain-specific retrieval of sourceinformation in the medial temporal lobe. Eur. J. Neurosci. 26 (5), 1333–1343.

Phan, K., Wager, T., Taylor, S., Liberzon, I., 2002. Functional neuroanatomy of emotion: ameta-analysis of emotion activation studies in PET and fMRI. NeuroImage 16,331–348.

Pihlajamäki, M., Tanila, H., Könönen, M., Hänninen, T., Hämäläinen, A., Soininen, H.,et al., 2004. Visual presentation of novel objects and new spatial arrangements ofobjects differentially activates the medial temporal lobe subareas in humans. Eur. J.Neurosci. 19 (7), 1939–1949.

Rajendran, G., Mitchell, P., Rickards, H., 2005. How do individuals with Aspergersyndrome respond to nonliteral language and inappropriate requests in computer-mediated communication? J. Autism Dev. Disord. 35 (4), 429–443.

Ralph, M.A., Pobric, G., Jefferies, E., 2008. Conceptual knowledge is underpinned by thetemporal pole bilaterally: convergent evidence from rTMS. Cereb. Cortex 19 (4),832–838.

Rankin, K.P., Kramer, J.H., Miller, B.L., 2005. Patterns of cognitive and emotional empathyin frontotemporal lobar degeneration. Cogn. Behav. Neurol. 18 (1), 28–36.

Rankin, K.P., Gorno-Tempini, M.L., Allison, S.C., Stanley, C.M., Glenn, S., Weiner, M.W.,

et al., 2006. Structural anatomy of empathy in neurodegenerative disease. Brain129 (11), 2945–2956.

Rockwell, P., 2001. Facial expression and sarcasm. Percept. Mot. Skills 93, 47–50.Rockwell, P., 2007. Vocal features of conversational sarcasm: a comparison of methods.

J. Psycholinguist. Res. 36 (5), 361–369.Rosen, H.J., Perry, R.J., Murphy, J., Kramer, J.H., Mychack, P., Schuff, N., et al., 2002.

Emotion comprehension in the temporal variant of frontotemporal dementia. Brain125, 2286–2295.

Rosen, H.J., Pace-Savitsky, C., Perry, R.J., Kramer, J.H., Miller, B., Levinson, R.W., 2004.Recognition of emotion in the frontal and temporal variants of frontotemporaldementia. Dement. Geriatr. Cogn. Disord. 17 (4), 277–281.

Saxe, R., 2006. Uniquely human social cognition. Curr. Opin. Neurobiol. 16, 1–5.Shamay-Tsoory, S.G., Tomer, R., Berger, B.D., Aharon-Peretz, J., 2003. Characterization of

empathy deficits following prefrontal brain damage: the role of the right ventro-medial prefrontal cortex. J. Cogn. Neurosci. 15 (3), 324–337.

Shamay-Tsoory, S.G., Tomer, R., Goldsher, D., Berger, B.D., Aharon-Peretz, J., 2004.Impairment in cognitive and affective empathy in patients with brain lesions:anatomical and cognitive correlates. J. Clin. Exp. Neuropsychol. 26 (8), 1113–1127.

Shamay-Tsoory, S.G., Tomer, R., Aharon-Peretz, J., 2005a. The neuroanatomical basisof understanding sarcasm and its relationship to social cognition. Neuropsychology19 (3), 288–300.

Shamay-Tsoory, S.G., Tomer, R., Berger, B.D., Goldsher, D., Aharon-Peretz, J., 2005b.Impaired “affective theory of mind” is associated with right ventromedialprefrontal damage. Cogn. Behav. Neurol. 18 (1), 55–67.

Snowden, J., Neary, D., Mann, D., 2007. Frontotemporal lobar degeneration: clinical andpathological relationships. Acta Neuropathol. 114 (1), 31–38.

Stone, V.E., Baron-Cohen, S., Knight, R.T., 1998. Frontal lobe contributions to theory ofmind. J. Cogn. Neurosci. 10 (5), 640–656.

Stuss, D., Gallup, G.G., Alexander, M.P., 2001. The frontal lobes are necessary for ‘theoryof mind’. Brain 124, 279–286.

Suzuki, W. a., Amaral, D. g., 1994. Perirhinal and parahippocampal cortices of themacaque monkey: cortical afferents. J. Comp. Neurol. 350 (4), 497–533.

Vittinghoff, E., Glidden, d. V., Shiboski, S.C., McCulloch, C.E., 2004. Regression Methodsin Biostatistics, 1st ed. Springer.

Wildgruber, D., Ackermann, H., Kreifelts, B., Ethofer, T., 2006. Cerebral processing oflinguistic and emotional prosody: fMRI studies. In: Anders Ende, Junghofer Kissler,Wildgruber (Eds.), Progress in Brain Research, Vol. 156, pp. 249–268.

Yesavage, J.A., Brink, T.L., Rolse, T.L., Lum, O., Huang, V., Adey, M., et al., 1983.Development and validity of a Geriatric Depression Scale: a preliminary report.J. Psychiatr. Res. 17, 37–49.

Zahn, R., Moll, J., Krueger, F., Huey, E.D., Garrido, G., Grafman, J., 2007. Social concepts arerepresented in the superior anterior temporal cortex. Proc. Natl. Acad. Sci. U. S. A.104 (15), 6430–6435.

Zahn, R., Moll, J., Iyengar, V., Huey, E.D., Tierney, M.C., Kreuger, F., et al., 2009. Socialconceptual impairments in frontotemporal lobar degeneration with right anteriortemporal hypometabolism. Brain 132 (3), 604–616.

Zysset, S., Huber, O., Ferstl, E., von Cramon, D.Y., 2002. The anterior frontomedian cortexand evaluative judgment: an fMRI study. NeuroImage 15 (4), 983–991.