NeuroImage 49 (2010) 3295–3307

Contents lists available at ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r.com/ locate /yn img

The amodal system for conscious word and picture identification in the absence of asemantic task

Leen Van Doren a, Patrick Dupont a, Sophie De Grauwe a, Ronald Peeters b, Rik Vandenberghe a,c,⁎a Cognitive Neurology Laboratory, Experimental Neurology Section, K.U. Leuven, Belgiumb Radiology Department, University Hospital Gasthuisberg, Belgiumc Neurology Department, University Hospitals Leuven, Herestraat 49, 3000 Leuven, Belgium

⁎ Corresponding author. Neurology Department, Univstraat 49, 3000 Leuven, Belgium. Fax: +32 (0)16 34442

E-mail address: rik.vandenberghe@uz.kuleuven.ac.be

1053-8119/$ – see front matter © 2009 Elsevier Inc. Adoi:10.1016/j.neuroimage.2009.12.005

a b s t r a c t

a r t i c l e i n f o

Article history:Received 13 August 2009Revised 19 November 2009Accepted 1 December 2009Available online 29 December 2009

Keywords:SemanticfMRIReadingNamingShort-term memoryAmodal

Previous studies using explicit semantic tasks, such as category or similarity judgments, have revealedconsiderable neuroanatomical overlap between processing of the meaning of words and pictures. This resultmay have been influenced by the semantic executive control required by such tasks. We examined thedegree of overlap while minimizing semantic executive demands. In a first fMRI experiment (n=28), wetitrated word (35.3 ms, SD=9.6) and picture presentation duration (50.7 ms, SD=15.8) such that consciousstimulus identification became a stochastic process, with a 50% chance of success. Subjects had to indicate bykey press whether or not they had been able to identify the stimulus. In a second fMRI experiment (n=19),the identification runs were followed by a surprise forced-choice recognition task and events were sorted onthe basis of subsequent memory retrieval success rather than a subjective consciousness report. For bothwords and pictures, when stimulus processing exceeded the conscious identification threshold, the leftoccipitotemporal sulcus (OTS), intraparietal sulcus, inferior frontal junction, and middle third of the inferiorfrontal sulcus (IFS) were more active than when subjects had been unable to identify the stimulus. For bothwords and pictures, activity in two of these regions, IFS and OTS, predicted subsequent memory retrievalsuccess. A Bayesian comparison revealed that the effective connectivity between IFS and the word- orpicture-specific systems was mainly mediated via its connections with OTS. The amodal nature of left OTSand IFS involvement in word and picture processing extends to tasks with minimal semantic executivedemands.

© 2009 Elsevier Inc. All rights reserved.

Introduction

The substrate for shared versus input modality-specific processingof meaning of words versus pictures has received a great deal ofattention, both in patient lesion studies (Rapp et al., 1993; Chertkow etal., 1997; Rogers and McClelland, 2004) and in functional imaging ofthe intact brain (Vandenberghe et al., 1996; Wagner et al., 1997;Buckner et al., 2000; Thierry and Price, 2006; Vandenbulcke et al., 2006,2007). Candidate regions involved in amodal processing, i.e., proces-sing of meaning of concrete words as well as pictures, are the ventraloccipitotemporal cortex (Buckner et al., 2000), the posterior third ofthe left middle temporal gyrus (Chertkow et al., 1997; Vandenbulcke etal., 2007; Kircher et al., 2009), the anterior temporal pole (Hodges et al.,1992; Vandenberghe et al., 1996; Rogers and McClelland, 2004), theleft inferior frontal sulcus (Wagner et al., 1997), the anterior inferiorfrontal gyrus (Goldberg et al., 2007), and the left middle frontal gyrus(Demb et al., 1995; Vandenberghe et al., 1996).

ersity Hospitals Leuven, Here-85.(R. Vandenberghe).

ll rights reserved.

Many tasks that have been used to study commonality betweencognitive processing of words and pictures have required at leastsome degree of explicit task-related retrieval of the associations andmeaning of the referent. This is true for relatively demanding taskswith a high decision weighting component such as the Pyramids andPalm Trees test (Howard and Patterson, 1992; Vandenberghe et al.,1996) but also for relatively easy tasks that require subjects toexplicitly retrieve well-known and typical features of the referentsuch as living–nonliving judgments (Wagner et al., 1997), sound, orcolour matching (Garrard and Carroll, 2006), etc. Task-relateddeployment of strategies for retrieving, comparing, or decidingabout semantic properties and associations may partly account forthe commonality in activity pattern between words and pictures inthese studies, in particular in the prefrontal cortex.

Other neuroimaging studies of amodal processing have made useof automatic semantic priming (Rissman et al., 2003; Sachs et al.,2008; Kircher et al., 2009; Sass et al., 2009) and combined words andpictures cross-modally (Kircher et al., 2009; Sass et al., 2009). Duringspeeded lexical decision, the left posterior middle and superiortemporal cortex shows a semantic relatedness effect during automaticpriming (Copland et al., 2003; Gold et al., 2006) for word–word pairs

3296 L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

and for picture–word pairs (Sachs et al., 2008; Kircher et al., 2009;Sass et al., 2009). Subliminal (masked) priming removes the strategicsemantic effects altogether (Forster, 1998; Holcomb and Grainger,2006; Diaz and McCarthy, 2007; Grainger and Holcomb, 2009).Neuroimaging studies of subliminal (masked) priming have reporteddifferential effects in left fusiform cortex for word–word pairscomparing same versus different words (Dehaene et al., 2001; Qiaoet al., 2010) and in the left posterior fusiform cortex for picture–picture pairs comparing repeated versus unrepeated pictures (Eddy etal., 2007). Cross-modal effects or within-study overlap betweenword–word and picture–picture priming effects have not beenreported in neuroimaging studies of subliminal priming until nowto the best of our knowledge. Prefrontal supraliminal priming effects(suppression or enhancement) have been reported at variouslocations distributed over the lateral convexity (Wagner et al., 1997;Copland et al., 2003; Rissman et al., 2003; Raposo et al., 2006; Race etal., 2009). This variability may relate to differences in the balancebetween automatic versus strategic component processes (Copland etal., 2003; Gold et al., 2006), but it could also be a consequence of therelatively small effect sizes and the inherent inability of fMRI tocapture the time dependency of priming effects.

For the first time, we applied a method to the study of amodalprocessing that stems from the consciousness research field (Kanwisher,2001; Bar et al., 2001; Marois et al., 2004; Carmel et al., 2006; Wilenius-Emet et al., 2004). At brief stimulus durations, conscious stimulusidentification is a probabilistic process. For each individual and eachinput modality, we selected a duration of word or picture presentationso that subjects reported conscious identification of the stimulus onlywith a probability of 50% across items (Marois et al., 2004; Carmel et al.,2006; Wilenius-Emet et al., 2004) and also within items across subjects.In this way, sensory input was matched between events that wereassociated with conscious processing and events in which processingremained subliminal. We tried to detect zones of amodal cognitiveprocessing during supra- versus subliminal word and picture identifi-cation. In a previous study of speeded picture identification (Bar et al.,2001; Bar et al., 2006), activity in mid and anterior fusiform cortexcorrelated with the subjects' level of confidence that they had been ableto identify the pictures correctly, ranging from zero (no stimulusperceived) to five (fully confident about stimulus identity), with alsoorbitofrontal involvement (Bar et al., 2006). In that study, confidencewas manipulated by repeating the same stimuli up to 6 times. Onlypictures were used (Bar et al., 2001; Bar et al., 2006). Our cross-modaldesign allowed us to examine whether these fusiform and other regionsare shared between words and pictures during supra- versus sub-threshold processing while sensory characteristics and prior exposurewere strictly matched between supra- and subliminally processedstimuli.

How does conscious processing of a word or picture differ fromsubliminal processing? According to the Theory of Visual Attention(TVA) (Bundesen, 1990; Bundesen et al., 2005; Bundesen andHabekost, 2008), conscious processing takes place when a perceptualentity is selected among other competing entities and gains access tovisual short-term memory (VSTM) according to a winner-takes-allprinciple. From VSTM, the consciously perceived unit is madeavailable (“broadcasted”) to other cognitive brain systems, such asdeclarative memory (Baars, 1988); 2002) or phonological retrieval(Jackendoff, 2007) systems. When we consciously identify a stimulus,the surface features of that stimulus and its visual form becomeavailable for subsequent explicit retrieval, as well as the identity of thereferent. The central question in this study is whether similar brainnetworks are involved for word and picture input when stimuli gainaccess to consciousness. Conscious stimulus identification may in itsturn trigger explicit lexical– or associative–semantic processes but theshort stimulus durations and the task instruction in our study directedattentional resources towards perceptual identification and awayfrom downstream processes of explicit semantic elaboration.

We determined which regions became activated when perceptualprocessing exceeded the threshold of conscious identification on thebasis of a subjective consciousness report (Frith et al., 1999; Beck etal., 2001; Bar et al., 2001; Haynes et al., 2005). We dealt with theinherent subjectivity of this measure (Frith et al., 1999) in two ways:first, we also included foil stimuli consisting of nonexisting chimeras.Subjects were instructed to respond negatively to the foil stimuli, i.e.,in the same way as to stimuli that they had not been able to identify.When the false-positive response rate to foil stimuli was too high, weexcluded the run. In case of pictures this task is very similar to theobject decision test (Riddoch and Humphreys, 1993), a classicalneuropsychological task that probes the structural description level ofprocessing. According to the Hierarchical Interactive Theory (HIT)(Humphreys and Forde, 2001), the activation of a structuraldescription of the object is necessary for object identification anddistinct from the associative–semantic level (Humphreys and Forde,2001). At the structural description level, visual percepts are matchedwith mnemonic presentations of real-life entities in a way that isinvariant for viewpoint, size, orientation, etc., and generalizes acrossdifferent exemplars of a same entity (Humphreys and Forde, 2001).

As a second measure to circumvent the subjectivity of aconsciousness report, we only retained regions that also fulfilled asecond, more objective criterion: following runs of subjectiveconsciousness report, we conducted a surprise forced-choice yes/norecognition task and sorted events on the basis of subsequentmemoryretrieval success rather than subjective consciousness report. Onlyregions that stringently fulfilled both criteria, an association with apositive consciousness report and with successful encoding, conjoint-ly for words and for pictures, were retained as amodal zones of supra-versus subliminal processing. To further verify the level and accuracyof stimulus identification and also to evaluate how the extent of theamodal activations obtained in the first and second experimentrelated to those obtained inmore classical tasks, we conducted a third,overt naming/reading fMRI experiment.

Materials and methods

Subjects

Twenty-eight healthy native Dutch speakers (15 women and 13men, between 19 and 29 years of age) participated in the first fMRIexperiment, which was based on the subjective consciousness report,and 8 additional subjects (3 women and 5 men, age range 19–25 years) in a control experiment. Nineteen other subjects (12womenand 7 men, between 19 and 35 years of age) participated in a thirdfMRI experiment, the subsequent memory retrieval experiment. Fourother subjects (1 man and 3 women, between 21 and 23 years of age)participated in a fourth, overt naming/reading fMRI experiment. Allparticipants were strictly right-handed (Oldfield, 1971), free ofpsychotropic and vasoactive medication, and without neurologicalor psychiatric history. They all gave written informed consent inaccordance with the Declaration of Helsinki. The experiments wereapproved by the Ethics Committee, University Hospitals Leuven.

Stimuli and tasks

First experimentVisual stimuli were presented foveally and projected from a Barco

6400i LCD projector (1024×768 pixels) at a refresh rate of 75 Hz ontoa screen 36 cm in front of the subject's eyes. Subjects viewed thescreen using a mirror attached to the head coil.

Each trial consisted of a forwardmask (duration 200ms), a writtenword (duration xword ms) or a picture (duration xpicture ms) and abackward mask (1700 ms −xword or xpicture, respectively), followedby a fixation point (duration 350 ms) (Fig. 1A). Subjects had toindicate by key presswhether or not they had been able to identify the

Fig. 1. Stimuli and tasks. (A) First experiment and subsequent memory retrieval experiment. For each subject, run, and modality, the variable x is titrated so as to reach a proportionof approximately 50% positive responses. (B) Overt naming/reading experiment with silent gap acquisition. The subsequent response window was 1000 ms during which scanningwas interrupted (silent gap) and the fixation point was turned green. During this silent gap, the subjects had to read or name the presented stimulus aloud. The variable x wastitrated so as to reach 50% correct naming/reading responses. Abbreviations: ITI: intertrial interval; TA: acquisition time.

3297L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

stimulus. The variables xword and xpicture were titrated for eachindividual, each modality (word versus picture) and each run so as toreach a proportion of approximately 50% affirmative responses perindividual and modality. The intertrial interval was 2250 ms.

For each individual, the starting values for the word and picturedurations were determined on the basis of the percentage positiveresponses during a prior test run on the scanner table using a trainingset of stimuli that was not used for fMRI. After each of the fMRI runs,we again checked the percentage positive reports per modality. If itwas below 37.5% or above 62.5%, the presentation duration for thatmodality was lengthened or shortened, respectively, with 13.3 ms(screen refresh rate was 75 Hz) in the next run.

Each run contained 8 catch trials (4 foil word and 4 foil picturetrials). During these trials, we presented chimeric stimuli that did notrefer to a real-world entity. They were constructed by dividing a wordor a picture in halves and by combining the word or picture halvesinto chimeric word or chimeric picture stimuli, preserving wordlength and continuity of picture outline, respectively. Since these foilstimuli did not belong to the real world, subjects had to respondnegatively to the foils (similarly to the object decision task (Riddochand Humphreys, 1993)). Subjects therefore had to respond negativelywhen they had not been able to identify the stimulus and also whenthey had identified the stimulus as an entity that did not belong to thereal world (in case of the chimeric foils). Runs in which subjects gave

an affirmative response to more than 3 out of 8 chimeric stimuli werenot included in the primary analysis because this false-positiveresponse ratemay indicate that subjects responded affirmatively evenin the absence of basic-level stimulus identification.

Stimuli were pseudorandomly chosen for each run and eachsubject out of a pool of 240 concrete entities (108 biological, 132nonbiological). Each entity was used during one run only perindividual. In each run, it was used once as a word and once as apicture. Half of the entities were shown first as a word and in one ofthe later trials as a picture and half vice versa. For each entity, themodality of first presentation was counterbalanced across subjects.Word length was two to six letters (mean=4.5, SD=1.0) and 1 to 3syllables (mean=1.4, SD=0.5), word size 0.79 to 3.17 visualdegrees. The base-10 logarithms of the word frequencies (permillion) ranged from 0.0 to 3.0 (mean=1.2, SD=0.7) (Baayen etal., 1993). Picture size was 0.95 to 3.97 visual degrees. The meanluminance of the stimuli was 101 cd/m2. Pictures were taken fromthe Snodgrass–Vanderwart set (Snodgrass and Vanderwart, 1980)but only if their names did not exceed an upper limit of 6 letters inlength. The pictures were supplemented by outlines of concreteentities that were closely similar in appearance to the Snodgrass–Vanderwart set. The forward and backward masks were chosenpseudorandomly out of a pool of 40 different masks constructed byscrambling the pictures.

3298 L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

In addition, each run contained 40 null events and 40 mask-onlyevents per run. Together with the word (n=40) and the picture trials(n=40) as well as the catch trials (n=8), this resulted in a total of168 trials per run. Subjects were instructed to respond negatively tothe mask-only trials. A negative key press therefore was requiredwhen subjects had been unable to identify the word or picture, whenthe stimulus consisted of a chimera, and in mask-only trials.

Each run of the first experiment consisted of 195 volumes. Twentysubjects underwent 6 runs, 4 subjects underwent 5 runs, and 4subjects underwent 4 runs.

One week after the fMRI session, 27 of the subjects werebehaviorally re-tested (one dropout for study-unrelated reasons).We presented the same runs as during the fMRI experiment withexactly the same stimulus durations (Fig. 1A). The subjects had toselect a button depending on whether or not they were able toidentify the stimulus, as in the fMRI experiment. Following the keypress, they had to pronounce the target word aloud. This provided anadditional estimate of the degree to which the stimulus durationsemployed during fMRI were compatible with accurate stimulusidentification.

In a control experiment in 8 additional subjects, the picturesconsisted of color photographs of single concrete entities against anempty background instead of outlines. The entities were partlydifferent from those used in the main experiment. This experimentalso contained a manipulation of stimulus familiarity (not reportedhere). Masks were constructed by scrambling the pixels of thephotographs. Each subject underwent 3 fMRI runs. The experimentwas otherwise identical to the above experiment.

Subsequent memory retrieval experimentIn the second experiment we sorted trials on the basis of

subsequent memory retrieval success rather than subjective con-sciousness report. Nineteen naive subjects underwent 3 fMRI runsidentical in design to those of the first experiment. These runs werefollowed by a surprise subsequent episodic memory retrieval task. Afoveal word or picture was presented for 500 ms and subjects had toindicate by key press whether the stimulus had been presented inthe preceding fMRI runs. Half of the stimuli were from the precedingfMRI runs (“old”) and half were novel. Stimuli in this experimentwere taken from the same pool as was used in the first experimentand were pseudorandomly allocated to be used either during thefMRI runs and as old stimuli during the subsequent memoryretrieval phase or as novel stimuli during the episodic memoryretrieval phase only. For each entity this was counterbalanced acrosssubjects.

Overt naming/reading experimentSubjects were instructed to read the word or name the picture

aloud during a silent gap of 1000 ms during which the fixation pointturned green (Fig. 1B). In case they had not been able to identify thestimulus, they had to pronounce the word “blanco.” The setup wasotherwise identical to that of the first experiment. As in the firstexperiment, stimulus durations xword and xpicture were titrated perindividual from run to run aiming for 50% correct naming/readingresponses. A jittered interval between 125 and 965 ms (with steps of120 ms) preceded the presentation of a forward mask. The fixationperiod that followed the backward mask was jittered between 1075and 235 ms with steps of 120 ms so that the sum of the first and thesecond period was always 1200 ms. All subjects underwent 6 runs,consisting of 174 volumes each.

Image acquisition

All fMRI experiments were conducted on a 3-T Philips Achievasystem (Best, The Netherlands) equipped with an 8-channel headvolume coil. Functional images were obtained using T2⁎ echo-planar

images (EPI) with blood oxygenation level-dependent (BOLD) contrastusing parallel imaging (GRAPPA). EPIs (TR/TE=2000/30 ms) com-prised 36 axial slices acquired continuously in ascending order coveringthe entire cerebrum (voxel size=2.75×2.75×3.75 mm3). For eachexperiment, the first 6 volumes of each run were discarded to allow theMRI signal to reach steady state. Structural images were acquiredduring the same session using a high-resolution T1-weighted sequence(TR/TE=9.6/4.6 ms, voxel size=0.98×0.98×1.2 mm3). Foam pillowswere used to minimize head movements.

In the overt naming/reading experiment, we used a clusteredvolume acquisition technique (Edmister et al., 1999). This acquisitioncreates a silent gap in between two consecutive scans during whichthe subjects responded overtly (Fig. 1B). In this event-related fMRIscheme, repetition time (TR) was 3000 ms and acquisition time (TA)was 2000 ms, leaving a brief period (1000 ms) without imageacquisition in between two consecutive scans.

Data analysis

Behaviorally, the reaction times of the identification responses topictures or words were analysed using repeated-measures analysis ofvariance (ANOVA) with input modality (words or pictures) andresponse (affirmative or negative) as within-subject factors. We alsoconducted a separate ANOVA with order of presentation of the entity(first versus second presentation of the entity) and input modality(words or pictures) as factors. In the forced-choice recognition phaseof the subsequent memory retrieval experiment, the true-positiveresponse rate as well as d′ (MacMillan and Creelman, 1990) werecompared between old items that subjects had identified during thepreceding identification runs and old items that they had failed toidentify using a Student's t test. We also compared the true-positiveresponse rate to old items that subjects had failed to identify duringthe preceding identification runs with the false-positive rate to novelitems (Student's t test).

The imaging data were analyzed with Statistical ParametricMapping SPM2 (Wellcome Trust Centre for Neuroimaging, UniversityCollege London, UK). After correction for differences in sliceacquisition time, the EPI volumes were realigned and resliced andan average functional image was created. The structural image wasco-registered with that mean volume and subsequently normalised tothe Montreal Neurological Institute (MNI) T1 template (Friston et al.,1995). The same normalisation matrix was applied to the functionalimages, which were then spatially smoothed using a Gaussian filterwith a kernel size of 5×5×7mm3 full width of half maximum. For thestatistical analysis of the time series, we applied a high-pass filter of(1/128) Hz. The event-related response was modeled using acanonical hemodynamic response function consisting of a mixtureof two gamma functions (Friston et al., 1999). Statistical inferencewascorrected for intrinsic autocorrelations. A statistical parametric map ofthe t statistic for the parameter estimates was generated andsubsequently transformed to a Z map.

First experimentIn the first experiment, we restricted the analysis of the fMRI

data to those entities to which approximately half of the subjects(range, 37% to 63%) responded affirmatively in both the word and thepicture format. In this way, the stimuli associated with a positiveconsciousness report and those associated with a negative responsewere matched over subjects. The remaining trials were modeledseparately but were not included in the main analysis. For everyindividual, we calculated the following key contrasts:

1. conjunction analysis (Nichols et al., 2005): (positive−negativeconsciousness report for words) and (positive−negative con-sciousness report for pictures)

2. main effect of input modality: words−pictures and inversely

3299L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

3. interaction effect: (positive−negative report for words)−(positive−negative report for pictures) and inversely

4. conjunction analysis (Nichols et al., 2005): (negative consciousnessreport for words−mask-only trials) and (negative consciousnessreport for pictures−mask-only trials).

The individual contrast images were weighted for the totalnumber of runs per subject and were then entered into a second-level, random-effects analysis. At the second level, we examined foreach of the a priori comparisons whether, on average, the contrastimages revealed significant differences (one-sample t test). Thesignificance level for whole-brain analyses was set at a voxel-levelinference threshold of uncorrected Pb0.001 combined with a cluster-level inference of Pb0.05 corrected for the whole brain volume(Poline et al., 1997). In a secondary analysis, we included all trials andall runs.

The control experimentwith colored photographswas analysed bymeans of a fixed-effects analysis. Our main contrast was equivalent tocontrast 1. The analysis was restricted to volumes of interest whichcorresponded to the significant clusters obtained in contrast 1 of thefirst experiment. The significance threshold was set at Pb0.05corrected for these volumes of interest (small volume correction).

Subsequent memory retrieval experimentIn the subsequent memory retrieval experiment, events from the

identification runs were sorted on the basis of subsequent memoryretrieval success. The main contrast was:

5. conjunction analysis (Nichols et al., 2005): (words that subjectssubsequently retrieved successfully−words that subjects failed tosubsequently retrieve) and (pictures that subjects subsequentlyretrieved successfully−pictures that subjects failed to subse-quently retrieve)

The individual contrast images were weighted for the totalnumber of runs per subject and were then entered into a second-level, random-effects analysis. At the second level, we examinedwhether, on average, the contrast images revealed significantdifferences (one-sample t test). Since we were interested in amodalregions that fulfilled a dual criterion (associated with a positiveconsciousness report as well as predictive of subsequent memoryretrieval success), the analysis of contrast 5 was restricted to thesignificant clusters obtained in contrast 1 of the first experiment. Wetested for a significant effect of subsequent memory retrieval in thesevolumes using a significance threshold of Pb0.05 corrected for thesevolumes of interest (small volume correction).

Naming/reading experimentThe overt naming/reading experiment was analysed bymeans of a

fixed-effects analysis. We calculated the following contrasts:

6. (accurately named pictures+accurately read words)−(pictures+words to which the subject responded “blanco”)

7. main effect of input modality: words−pictures and inversely8. interaction effect: (accurately read words−words to which the

subjects responded “blanco”)− (accurately named pictures−pictures to which the subject responded “blanco”) and inversely

Dynamic causal modellingTo test how the amodal system was connected with the input

modality-specific systems, we formally compared different dynamiccausal models (Friston et al., 2003). Amodal nodes were includedwhen they were significantly activated in contrast 1 (conjoint activityduring positive versus negative consciousness report for both wordsand pictures) as well as in contrast 5 (conjoint activity duringsubsequent memory retrieval success for both words and pictures).Modality-specific nodes were included when they were differentiallyactive for words compared to pictures (contrast 2 and its inverse, first

experiment), excluding activations that were due to a differentialdecrease. Each node contained the same voxels for all subjects.

For a detailed representation of the models that we compared, werefer to Fig. 2. The models we compared differed in how the amodalsystem was connected with the modality-specific systems. For eachamodal node, we constructed a model in which that node only wasconnected with the modality-specific systems (Figs. 2A and B). Wealso included a model where all amodal nodes were connectedequally to the input modality-specific systems (Fig. 2C). The numberof models therefore that we compared equalled the number of amodalnodes plus one. All other parameters were matched between themodels (Fig. 2). Input modality-specific nodes that were located inoccipital cortex received input from their specific modality, amodalnodes that were located in occipital cortex received input from bothmodalities (Fig. 2).

The time series that were used for the DCM were extracted fromthe pooled data from the first experiment as well as the identificationruns of the second experiment (n=47).

At the group level, the models were formally compared by meansof the group Bayes factor (which is the product of the Bayes factors ofeach subject), which is based on the Akaike (AIC) and the Bayesianinformation criterion (BIC) (Penny et al., 2004). The group Bayesfactor is a ratio of model evidences based on the data and is used tocompare models. It takes into account the complexity of the models.To evaluate between-subject consistency, we also calculated in howmany individuals DCM provided positive evidence (defined as anindividual Bayes factorN3) for a specific model compared to the othermodels (Stephan et al., 2007).

Results

Behavioral data

First experimentIn the first experiment, our titration procedure yielded an average

word presentation duration across runs and subjects of 35.3 ms(SD=9.6) and an average picture presentation duration of 50.7 ms(SD=15.8). Thirty-four out of 156 runs were excluded becausesubjects responded false-positively to more than 3 out of 8 catch trialsin these runs, indicating that in these runs subjects produced apositive key press even when they had failed to identify the stimuli atthe basic level. Subjects responded false-positively to 0.6% (SD=0.5)of all mask-only events.

Seventy-two out of 240 real-world entities were identified byapproximately half of the subjects in both the word and the pictureformat. We restricted the analysis to these entities so that the contrastbetween positive and negative report was matched for stimuli oversubjects.

When subjects gave a positive consciousness report, reaction timeswere significantly shorter compared to when they gave a negativereport (F(1,27)=14.1, P=0.001) (Table 1). Subjects responded fasterto words than to pictures (F(1,27)=14.5, P=0.001). There were nointeraction effects (F(1,27)=0.06, P=0.816) (Table 1).

The positive response rate did not differ significantly between firstand second presentations of an entity (F(1,27)=3.32, P=0.08) orbetween words and pictures (F(1,27)=2.11, P=0.16). There was nointeraction between repetition of an entity and input modality(F(1,27)=0.28, P=0.60) (first presentation of an entity, wordmodality: positive response rate 51.7% (SD=6.4), first presentationof an entity, picture modality: 54.5% (SD=8.1), second presentationof an entity, word modality: 50.6% (SD=6.8); second presentation ofan entity, picture modality: 52.1% (SD=8.4)).

In the behavioral retest study, inwhich the same stimulus durationswere used as in the fMRI study, a positive key press in response to aword trial was associated with 90.6% (SD=8.9) correct reading and anegative key press with 2.6% (SD=8.2) correct reading. A positive key

Fig. 2. Three DCM models tested. (A) First model: the connections between the amodal and the modality-specific system go exclusively via IFS. (B) Second model: the connectionsbetween the amodal and the modality-specific system go exclusively via OTS. (C) Third model: each of the two amodal nodes are connected with the two modality-specific systems.Legend: Lines marked with ID: connections that are modulated by consciousness report (positive versus negative) according to the model tested. Lines marked with P: connectionsthat are modulated picture modality according to the model tested. Lines marked with W: connections that are modulated by word modality. Dashed lines: connections that aremodulated by word input modality (W) and consciousness report (ID). Dash-dotted lines: connections that are modulated by picture input modality (W) and consciousness report(ID). OTS: left occipitotemporal sulcus; IFS: left inferior frontal sulcus; STS: left superior temporal sulcus; FO: left frontal operculum; mog: middle occipital gyrus; coll: collateralsulcus.

Table 1First, control, and second experiment: reaction times (in ms) (mean and SD).

Firstexperiment

Controlexperiment

Secondexperiment

Mean SD Mean SD Mean SD

Positive report for words 752.9 134.7 744.9 121.0 804.6 105.8Negative report for words 811.8 128.3 740.5 104.4 866.9 126.1Positive report for pictures 782.3 122.8 782.2 134.1 806.0 124.7Negative report for pictures 838.7 119.5 815.0 111.4 912.8 120.5Mask events (word duration) 638.5 86.2 640.4 74.1 703.5 90.1Mask events (picture duration) 638.8 89.5 642.2 73.3 715.3 96.0

3300 L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

press to a picture trial was associated with 81.8% (SD=14.1) correctnaming and a negative key press with 2.0% correct naming (SD=5.1).

In the color photograph experiment, behavioral findings were verysimilar to those of the main experiment except for the fact thatreaction times did not differ for words between positive and negativereports (Table 1). This finding is difficult to interpret and may be afalse-negative due to the small sample size (n=8).

Subsequent memory retrieval experimentIn the identification runs of the subsequent memory retrieval

experiment, our titration procedure yielded an average word

3301L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

presentation duration of 30.6 ms (SD=9.2) and an average picturepresentation duration of 49.7 ms (SD=11.1) across runs and subjects.Reaction times were significantly faster when subjects respondedaffirmatively compared to negatively (F(1,18)=13.0, P=0.002)(Table 1). Subjects responded faster to words than to pictures(F(1,18)=7.6, P=0.013), without interaction effects.

In the subsequent memory retrieval phase, the true-positiveresponse rate differed significantly between old stimuli to whichsubjects had responded positively in the preceding fMRI runs (54.3%,SD=15.2) and old stimuli to which subjects had respondednegatively (23.1%, SD=13.4) (Student's t(18)=9.0, Pb0.001). Thiswas confirmed by analysis of the sensitivity measure d′ (MacMillanand Creelman, 1990) (positive response: d′=1.11, SD=0.68;negative response: d′=0.21, SD=0.36, Student's t(18)=9.40,Pb0.001). We also determined whether old stimuli to which subjectshad given a negative consciousness report were recognized differentlyfrom novel items. The true-positive response rate for old stimuli towhich subjects had given a negative consciousness report (23.1%,SD=13.4, see above) was significantly higher than the false-positiveresponse rate for novel stimuli (18.8%, SD=14.3) (t(18)=2.23,P=0.039).

Overall, the true-positive recognition rate, i.e., the proportion ofold stimuli that subjects recognized correctly, was 38.7% (SD=13.9)for words and 38.6% (SD 12.9) for pictures. The false-positiverecognition rate, i.e., the proportion of novel stimuli to which subjectsresponded as if they had been presented in the previous fMRI runs,was 18.0% (SD 16.4) for words and 19.7% (SD 13.0) for pictures.

Overt naming/reading experimentIn the overt naming/reading experiment, we titrated stimulus

duration so as to reach 50% accurate naming or reading. Across runsand subjects, the average word presentation duration was 45.5 ms(SD 9.9) and the average picture presentation duration was 77.0 ms(SD 12.2). Subjects read the words correctly in 53.6% (SD 19.2) of allword trials, read the words incorrectly in 17.0% (SD 10.2), andresponded “blanco” in 28.8% (SD 20.9) of all word trials, with responseomissions in the remainder of the word trials. Fifty-eight percent oferrors consisted of substitution, addition, or omission of one to threeletters, most often in the middle of the word (45.3%) and lessfrequently at the end (30.5%) or the beginning of the word (24.2%).About 9.8% of errors were semantic paralexias. The remainder of worderrors did not bear any apparent relationship with the target word.Pictures were named correctly in 49.5% (SD 9.7) of all trials. Subjectsnamed the pictures incorrectly in 31.5% (SD 14.1) and responded“blanco” in 18.5% (SD 15.2) of all trials. About 45.4% of picture namingerrors were based on perceptual similarity without semanticrelatedness, 28.5% of errors could be explained by both perceptualsimilarity and semantic relatedness, 4.3% of picture naming errorswere semantically related to the target word, without perceptualsimilarity. In 6.3% of the picture naming errors, subjects provided aname at the superordinate level. The remainder of picture errors didnot bear any apparent relationship with the target picture.

Imaging results

First experiment

Regardless of input modality (conjunction contrast 1), words orpictures, a positive consciousness report was associated withsignificantly higher activity in the middle third of the left inferiorfrontal sulcus (IFS) (Figs. 3A and C), the left inferior frontal junction(IFJ) (Figs. 3B and C), the border between the posterior and themiddlethird of the left occipitotemporal sulcus (OTS) (Figs. 3C and D) and themiddle third of the left intraparietal sulcus (IPS) (Figs. 3C and E)(Table 2). A secondary analysis in which we included all events and allruns strictly confirmed these findings.

As for the input modality-specific effects (contrasts 2 and inverse),in comparison to pictures, words activated the posterior third of theleft superior temporal sulcus (STS) and the left frontal operculum(FO) (Table 2; Fig. 4). The FO activation (Fig. 5, yellow) lay clearlyinferior to the IFS and IFJ areas reported above (Fig. 5, green). Severalother regions such as left and right inferior parietal lobule (Fig. 5,yellow) responded more strongly to words than to pictures, but thesedifferences were purely due to a differential decrease compared tobaseline. Pictures activated the middle occipital gyrus (mog)bilaterally as well as the left collateral sulcus (Table 2).

In a whole-brain analysis, we did not obtain a significantinteraction between input modality and conscious stimulus identifi-cation (contrasts 3 and inverse). In left FO (interaction effect−57, 12,15, Z=2.66, uncorrected (uncorr.) voxel-level Pb0.005) and left STS(interaction effect −60, −51, 6, Z=2.84, uncorr. voxel-levelPb0.005) word-specific responses tended to be larger when subjectswere able to consciously identify the stimuli (Fig. 4). The same wastrue for picture-specific responses in the right mog (interaction effect42, −81, 12, Z=3.96, uncorr. voxel-level Pb0.001).

Compared to mask-only trials, when subjects gave a negativeconsciousness report to a word or a picture trial, left OTS and left IFJwere significantly activated forwords and for pictures (Figs. 3B andD),as well as the anterior cingulate sulcus and the anterior insulabilaterally (Table 2). To examine IFS with higher sensitivity, weconducted a small volume correction using the IFS cluster that wasobtained from contrast 1 as volume of interest. Activity in IFS wassignificantly higher during a negative consciousness report comparedto mask-only trials both for words and for pictures (−45, 33, 21,Z=2.98, corr. P=0.024).

In the control experiment with colored photographs, the contrastbetween a positive and a negative consciousness report yielded verysimilar results, apart from IPS. When subjects gave a positive insteadof a negative response, the left OTS (−45, −45, −18, Z=3.18, ext. 8,corr. Pb0.05), IFS (−45, 33, 15, Z=3.35, ext. 9, corr. Pb0.01), and IFJ(−42, 12, 30, Z=3.84, ext. 16, corr. Pb0.01) were significantlyactivated for both words and pictures.

Subsequent memory retrieval experimentActivity in two clusters that were associated with a positive

consciousness report for both words and pictures (derived fromconjunction contrast 1) predicted subsequent memory retrievalsuccess in an amodal manner, conjointly for both words and pictures(conjunction contrast 5): OTS (−54, −57, −12, Z=2.98, ext. 5,corrected Pb0.05) and IFS (−42, 36, 18, Z=2.67, ext. 15, correctedP=0.05) (Fig. 5, orange outline). IPS and IFJ activity did not predictsubsequent memory retrieval success, even if we lowered thethreshold to an uncorrected Pb0.05.

Overt naming/reading experimentWhen subjects responded correctly to the words or the pictures

(contrast 6), left IFS, IFJ, andOTSwere all significantlymore active thanwhen they responded “blanco” (IFS: −48, 39, 9, Z=5.16, ext.=179,corr. Pb0.001, IFJ: −48, 9, 27, Z=4.19; OTS: −54, −54, −12,Z=6.15, ext.=197, corr. Pb0.001), together with several otheractivations (Fig. 5, blue outline).

In contrast with the first experiment (contrast 2 and inverse, Figs 4and 5, yellow), the overt naming/reading experiment did not reveal amain effect of input modality (contrast 7 and inverse) in left FO andthe posterior third of left STS, even when we lowered the threshold touncorrected Pb0.001. This was due to the fact that these regions wereactivated both during word and during picture trials compared tomask-only events (conjunction analysis (Nichols et al., 2005):−48, 9,21, Z=6.33, corr. voxel-level Pb0.001 and −57, −39, 3, Z=6.44,corr. voxel-level Pb0.001, respectively).

We did not find any significant interactions between inputmodality and naming/reading success.

3302 L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

Dynamic causal modellingThe outcome of the comparison between the 3 models (Fig. 2) was

highly in favor of model 2 (Fig. 2B) above model 1 or 3: the groupBayes factor for model 2 compared to the model 1 or 3 was more than10768 and 10522, respectively (Penny et al., 2004). The proportion of

subjects with positive evidence in favor of model 2 versus subjectswith positive evidence in favor of one of the other models was 45:2and 47:0, respectively (Stephan et al., 2007). It was therefore muchmore likely that the effective connectivity between the amodal andthe modality-specific regions went principally via OTS. In the winning

Table 2Details of the significant activity foci obtained in the first experiment.

Region x y z Z Extent Corr. P

Conjunction contrast 1: (positive report−negative report) (W & and P)L IFS −42 36 15 4.09 23 b0.05L IFJ −45 9 33 3.97 22 b0.05L OTS −51 −57 −15 4.29 33 b0.01L IPS −39 −51 57 3.78 27 b0.01

Contrast 2: W−PL STS −63 −51 9 3.85 32 b0.005L FO −51 12 18 4.26 25 b0.05L IPL −60 −42 42 4.01 47 b0.001R IPL 51 −39 51 4.33 126 b0.001

Contrast 2, inverse: P−WL mog −45 −81 15 4.72 194 b0.001R mog 51 −69 −6 6.02 795 b0.001L coll. s. −27 −48 −15 4.68 84 b0.001

Conjunction contrast 4: (negative report−mask-only) (W & and P)L IFJ −45 6 33 4.50 40 b0.001L OTS −45 −60 −9 3.94 40 b0.005Anterior cingulate 3 18 48 4.13 77 b0.001L anterior insula −33 21 9 4.41 37 b0.005R anterior insula 42 24 3 4.84 66 b0.001

Abbreviations: IPL: inferior parietal lobule; coll. s.: collateral sulcus; L: left; R: right; W:words; P: pictures; extent: unit=3×3×3 mm3 voxels; corr. P first expt: cluster-levelinference corrected for whole brain search volume.

3303L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

model, there were no direct connections between IFS and themodality-specific nodes. There was a significant modulatory effect(strength=0.05) of positive consciousness report on the forwardconnection from OTS to IFS and only a weak modulatory effect on thefeedback connection (strength=0.01).

Discussion

Two regions, the middle third of the left IFS (Figs. 3A and C) andthe left occipitotemporal sulcus (Figs. 3C and D), were activated inan amodal manner when a stimulus, word or picture, gained accessto consciousness. In left OTS, and to a lesser degree also in IFS, thiseffect consisted of an enhancement of a response that was alsopresent for subliminal stimuli. Activation in these two regionspredicted subsequent memory retrieval success for both words andpictures (Fig. 5, orange outline). With regards to the initial researchquestions, our results imply that the functional contribution of themiddle third of left IFS to amodal processing is not limited toconditions that require explicit semantic strategic retrieval. Withregards to left OTS, subliminal effects (Dehaene et al., 2001; Eddy etal., 2007) and effects of conscious recognition (Bar et al., 2001)occurred in a same region independently of input modality, words,or pictures.

The subjectivity of the response decision (Frith et al., 1999) in thefirst experiment cannot account for our findings. We monitored theresponse bias by means of the false-positive response rate to foilstimuli and rejected runs when this rate exceeded a preset threshold.Furthermore, in the subsequent memory retrieval experiment, eventswere sorted on the basis of subsequent memory retrieval rather thansubjective consciousness report and this more objective sortingcriterion yielded essentially the same results in IFS and OTS.

Fig. 3. First experiment: conjunction contrast: (positive−negative consciousness report forlevel uncorrected Pb 0.001 (green). (A) IFS: Zmap and time–activity curve. Time–activity cuvoxels belonging to this cluster. (B) IFJ: Zmap and time–activity curves for each of the conditithe conjunction projected on the Population-Average, Landmark and Surface based (PALSToolkit (side view and ventral view) (CARET, Washington University School of Medicine, Deand time–activity curves averaged for each of the conditions over all 28 subjects and all 27 voof the conditions over all 28 subjects and all 33 voxels belonging to this cluster. Legend: Darkwords. Red: Positive consciousness report for pictures. Magenta: Negative consciousness rethat of the word events; dotted: duration of interval between masks equal to that of the pi

Neither can differences in difficulty between trials with versuswithout conscious identification account for the effects. We includedonly those items in our primary analysis that were identified byapproximately half of the participants. In this way, we were able toexclude item-related differences (such as stimulus familiarity orvisual complexity) between trials that did and trials that did not leadto conscious identification across subjects. Furthermore, if anything,trials in which subjects failed to identify the stimulus could beconsidered more difficult than those in which subjects succeeded, yetactivity was higher in the latter type (Fig. 3).

Could subvocal phonological retrieval during picture processinghave given rise to apparently amodal effects in left IFS duringconsciousness report? This can be ruled out for two reasons: in thefirst experiment, words but not pictures caused activation of the leftposterior superior temporal sulcus (STS) and of the frontal operculum(FO) compared to baseline. Even when we lowered the significancethreshold to an uncorrected Pb0.05 in the first experiment, left STSand FO were not activated during picture events compared to mask-only trials (Fig. 4). Had the subjects named the pictures silently, leftSTS and FO would be activated also during picture processing, as wasthe case during the overt naming and reading experiment. Further-more, many previous neuroimaging studies of phonological proces-sing as well as a lesion mapping study (Fiez et al., 2006) implicate theleft FO rather than the IFS in phonological output retrieval (Paulesu etal., 1993; Fiez and Petersen, 1998; Price, 1998; Poldrack et al., 1999;Nixon et al., 2004). The absence of any silent naming/readingresponse in STS or FO can probably be accounted for by the highperceptual demands and the fast presentation rate which may havedirected attentional resources during the picture trials towardsperceptual processing rather than lexical retrieval.

The OTS focus (Figs. 3C and D and 6) is near or identical to thevisual word form area (VWFA) (Cohen et al., 2000; Dehaene et al.,2001; Cohen et al., 2002; Dehaene et al., 2002; Cohen et al., 2003;Gaillard et al., 2006; Vinckier et al., 2007). The anatomical relationshipbetween the OTS cluster and the VWFA activity peaks from theliterature is depicted in Fig. 6. While there is a certain degree ofvariability in the exact location of the activity peaks from theliterature, the original focus reported by Cohen et al. (2000) fallswell within our OTS cluster along the x, y, and z dimensions. In leftOTS, activity is robustly enhanced during conscious identificationcompared to subliminal processing. In a previous study (Dehaene etal., 2001), left OTS showed an approximately 90% higher responsewhen words were unmasked compared to subliminal (masked)words. Our findings also confirm previous evidence that the role ofVWFA is not limited to word processing exclusively and that it isactivated to a same degree for pictures as for words (Price and Devlin,2004; Ben-Shachar et al., 2007). At a neuronal level, similarcomputations may be performed for the two types of stimuli or theapparent amodality may be a consequence of the relatively lowresolution of fMRI: neurons with preference for word or for picturestimuli may be intermixed in this region creating an apparentlyamodal effect on fMRI responses.

According to TVA, a visuoperceptual unit is consciously identifiedand selected for more elaborate processing when it wins thecompetition for entry into VSTM (Bundesen, 1990; Bundesen et al.,2005; Desimone and Duncan, 1995). Access to VSTM is required for

words) and (positive−negative consciousness report for pictures). Threshold: voxel-rves show responses in each of the 6 conditions averaged over all 28 subjects and all 23ons averaged over all 28 subjects and all 22 voxels belonging to this cluster. (C) Zmap of) (Van Essen, 2005) atlas using Computerized Anatomical Reconstruction and Editingpartment of Anatomy and Neurobiology, http://brainmap.wustl.edu). (D) OTS: Z mapxels belonging to this cluster. (E) IPS: Zmap and time–activity curves averaged for eachblue: Positive consciousness report for words. Cyan: Negative consciousness report for

port for pictures. Black: Mask-only (solid: duration of interval between masks equal tocture events).

Fig. 4. First experiment. Words minus pictures thresholded at uncorr. voxel-level Pb0.001. (A) Activity map projected on the PALS brain using CARET. (B) Left frontal operculum: Zmap and time–activity curves for each of the 6 conditions averaged over all 28 subjects and all 25 voxels belonging to this cluster. (C) Left posterior STS: Z map and time-activitycurves for each of the 6 conditions averaged over all 28 subjects and all 32 voxels belonging to this cluster. Same conventions as in Fig. 3.

3304 L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

subjects to be aware of object identity and also enhances episodicmemory encoding, a combined effect that we observed in our study inthe middle third of IFS and in OTS. IFS and OTS may mediate access toshort-term memory enabling a more advanced level of processingassociated with a higher likelihood of conscious processing andsubsequent memory retrieval. Single neuron recording studiesprovide evidence that both inferior temporal cortex and prefrontalcortex are involved in short-term memory but with some fundamen-tal distinctions. For instance, neuronal activity in inferior temporalcortex is more sensitive to intervening stimuli than prefrontal activity(Miller, 2000). Alternatively, the IFS activation may relate to cognitive

Fig. 5. Summary of the main results superimposed on the PALS atlas using CARET.Green: conjunction contrast: positive minus negative consciousness report for bothwords and pictures conjointly. Thresholded at uncorrected Pb0.001 (voxel-levelinference). Orange outline: conjunction contrast: voxels within these clusters that arepredictive of subsequent memory retrieval success for both words and pictures.Thresholded at Pb0.05 corrected for the volume of interest. Yellow: Contrast 2: wordsminus pictures. Thresholded at uncorrected Pb0.001 (voxel-level inference). Dark blueoutline: Contrast 6: (accurately named pictures+accurately read words)−(pictures+words to which the subject responded “blanco”). Thresholded at uncorrectedPb0.001 (voxel-level inference).

control induced by perceptual difficulty (Duncan and Owen, 2000).IFS activation may reflect successful cognitive control allowingperceptual identification as opposed to a failure of cognitive controlleading to lack of perceptual identification. Exactly how IFS exertscognitive control to enhance success of perceptual identificationremains to be specified in more detail, e.g., at the neuronal level. Thisexplanation is akin to that proposed by Bar et al. (2001, 2003, 2006)regarding the interaction between orbitofrontal and ventral occipito-temporal cortex in stimulus identification. As a third possibleexplanation, IFS activity may reflect implicit semantic processing,i.e., a spread of activation to nearby semantic nodes whenever theentity is evoked regardless of task demands (Collins and Loftus, 1975;Neely, 1977; Ruff et al., 2008).

According to Gabrieli et al. (1998), the middle third of IFS isinvolved in semantic working memory at the “crossroad” betweensemantic knowledge and episodic memory processing and mediatesaccess to preexisting knowledge, integrating perception and inter-pretation of the current stimulus with other knowledge (Wagner etal., 1999). Repetition priming effects in this region are influenced bythe task performed during the priming phase, semantic versusnonsemantic (Demb et al., 1995; Wagner et al., 2000). Our resultsdemonstrate that IFS is also involved when semantic executivedemands are minimal in the absence of a semantic task. The term“semantic working memory” may not be entirely appropriate for thetype of processing subjects had to do in our experiment.

The amodal areas that we obtained during conscious identificationand subsequent memory retrieval overlapped with the activitypattern seen during classical picture naming and word reading butthe activity pattern in our study was muchmore restricted. It was also

Fig. 6. Neuroanatomical relationship between OTS cluster obtained in the current experiments and visual word form area coordinates from the literature. (A) Sagittal section (x=−41 mm). This section provides a see-through projection of all activity peaks from x=−33 to −48 mm. This panel provides an accurate view of how the OTS cluster relates toactivity peaks from the literature along the y and z dimensions. (B) Coronal section (y=−59 mm). This section provides a see-through projection of all activity peaks from y=−48to −69 mm. It provides an accurate view of how the OTS cluster relates to the activity peaks along the x and z dimensions. (C) Axial section (z=−7 mm). This section provides asee-through projection from z=−22 to 8 mm. It provides an accurate view of how the cluster relates to the activity peaks along the x and y dimensions. Legend: Green area: OTScluster obtained from the conjunction contrast 1 thresholded at an uncorr. Pb0.001; orange area: voxels within the OTS cluster where activity predicts subsequent memory retrievalsuccess (conjunction contrast 5); cyan circle (−42, −57, −6): hemifield-independent response to words minus rest (Cohen et al., 2000) and visually presented words minusnonword stimuli (Gaillard et al., 2006); green circle (−48, −60, −12) and triangle (−44, −52, −16): masked words minus baseline (Dehaene et al., 2001); yellow circle (−39,−57, −9): (words and consonant letter strings) minus checkerboards (Cohen et al., 2002); yellow triangle (−33, −69, −6): words minus consonant letter strings (Cohen et al.,2002); magenta (−41, −61, 8): visually presented words minus rest (Dehaene et al., 2002); red circle (−42, −63, −15): (words and consonant letter strings) minus fixation orcheckerboards (Cohen et al., 2003); dark blue (−42, −51, −3): letter strings minus checkerboards (Gaillard et al., 2006); black (−40, −48, −22): increase with increasingproximity to words (Vinckier et al., 2007).

3305L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

more restricted than the activity pattern obtained in studies of amodalprocessing that used tasks with higher semantic executive demands(Vandenberghe et al., 1996; Vandenbulcke et al., 2007). Prefrontalactivations in particular were far less extensive than in previousexperiments that made use of explicit semantic tasks. This indirectlysuggests that some of the amodal prefrontal activations, e.g., in themiddle frontal gyrus or in anterior inferior frontal gyrus, as seen inprevious studies, may have been contingent on the level of semanticexecutive control required. As our experiments, however, demon-strate, this is not the case for the middle third of the left IFS. Theabsence of another region, the left anterior temporal pole, may be dueto lack of sensitivity of fMRI in this region (Devlin et al., 2000) or itmay point to a functional distinction between IFS and the anteriortemporal pole. The anterior temporal pole is principally found to beactivated whenmultiple representations must be combined in a novelway, for instance, during semantic association tasks with multipleitems (Vandenberghe et al., 1996), comprehension of sentences(Vandenberghe et al., 2002), or stories (Fletcher et al., 1995). The leftanterior temporal pole may play a role as an associative structure,binding distributed semantic representations in a combinatorialfashion (Vandenbulcke et al., 2005). The absence of anterior temporalcortex in our study may be due to the fact that we presented singlestimuli and that subjects did not have to actively retrieve associations.Another region that has been typically found in studies of amodalprocessing but is missing in the current study is the posterior third ofthe left middle temporal gyrus (GTm) (Vandenberghe et al., 1996;Vandenbulcke et al., 2007). A possible explanation is that this area isinvolved in retrieval of associative–semantic knowledge beyond themere identification of words or the pictures. If the task demandsexplicit naming, category or similarity judgments of words orpictures, its activity level may be higher than when simpleidentification is required (Ruff et al., 2008).

To define the position of the amodal regions vis-à-vis themodality-specific systems, we explicitly modeled the effectiveconnectivity between the amodal zones (IFS and OTS) and themodality-specific systems and compared different models. Directfeedforward connections from the two input modality-specificsystems towards IFS would be in agreement with a role of IFS as aconvergence zone onto which input from the input modality-specificsystems converges. This convergence onto IFS might be associatedwith a higher probability of conscious processing (Baars, 1988;Damasio, 1989; Baars, 2002). According to this hypothesis, IFS serves

as a hub receiving afferents from the input modality-specific systemsand sending efferents to other cognitive systems, such as the episodicmemory system. As a second possibility, IFS may influence modality-specific systems through its interactions with OTS rather than viadirect effects on the modality-specific systems. A Bayesian compar-ison between different dynamic causal models provided strongarguments in favor of the latter model.

To conclude, left IFS and OTS are involved in cognitive processingof words and pictures in an amodal manner, independent of semanticexecutive demands. We postulate that these amodal regions mediateaccess to short-term memory, enhancing cognitive control, detailedperceptual analysis, and episodic memory encoding.

Acknowledgments

This work was supported by Research Foundation Flanders (FWO)[G.0076.02 and G0668.07 to R.V.], K.U.Leuven [OT/04/41, OT/08/056,EF/05/014 to R.V.], and FederaalWetenschapsbeleid belspo [IUAP P6/29].

References

Baars, B., 1988. A cognitive theory of consciousness. Cambridge University Press.Cambridge, UK.

Baars, B., 2002. The conscious access hypothesis: origins and recent evidence. TrendsCogn. Sci. 6, 47–52.

Baayen, H., Piepenbrock, R., Rijn, H.V., 1993. The CELEX lexical database (CD-ROM).Linguistic Data Consortium. PA, Philadelphia.

Bar, M., 2003. A cortical mechanism for triggering top-down facilitation in visual objectrecognition. J. Cogn. Neurosci. 15, 600–609.

Bar, M., Kassam, K., Ghuman, A., Boshyan, J., Schmid, A., Dale, A., Hamalainen, M.,Marinkovic, K., Schacter, D., Rosen, B., Halgren, E., 2006. Top-down facilitation ofvisual recognition. Proc. Nat. Acad. Sci. 103, 449–454.

Bar, M., Tootell, R., Shacter, D., Greve, D., Fischl, B., Mendola, J., Rosen, B., Dale, A., 2001.Cortical mechanisms specific to explicit visual object recognition. Neuron 29,529–535.

Beck, D., Rees, G., Frith, C., Lavie, N., 2001. Neural correlates of change detection andchange blindness. Nat. Neurosci. 4, 645–650.

Ben-Shachar, M., Dougherry, R., Deutsch, G., Wandell, B., 2007. Differential sensitivity towords and shapes in ventral occipito-temporal cortex. Cereb. Cortex 17,1604–1611.

Buckner, R., Koutstaal, W., Shacter, D., Rosen, B., 2000. Functional MRI evidence for arole of frontal and inferior temporal cortex in amodal components of priming. Brain123, 620–640.

Bundesen, C., 1990. A theory of visual attention. Psychol. Rev. 97, 523–547.Bundesen, C., Habekost, T., 2008. Principles of visual attention. Linking Mind and Brain.

Oxford University Press, Oxford, UK.

3306 L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

Bundesen, C., Habekost, T., Kyllingsbaek, S., 2005. A neural theory of visual attention:Bridging cognition and neurophysiology. Psychol. Rev. 112, 291–328.

Carmel, D., Lavie, N., Rees, G., 2006. Conscious awareness of flicker in humans involvesfrontal and parietal cortex. Curr. Biol. 16, 907–911.

Chertkow, H., Bub, D., Deaudon, C., Whitehead, V., 1997. On the status of object conceptsin aphasia. Brain Lang. 58, 203–232.

Cohen, L., Dehaene, S., Naccache, L., Lehericy, S., Dehaene-Lambertz, G., Henaff, M.,Michel, F., 2000. The visual word form area: spatial and temporal characterizationof an initial stage of reading in normal subjects and posterior split brain patients.Brain 123, 291–307.

Cohen, L., Lehericy, S., Chochon, F., Lemer, C., Rivaud, S., Dehaene, S., 2002. Language-specific tuning of visual cortex? Functional properties of the visual word form area.Brain 125, 1054–1069.

Cohen, L., Martinaud, O., Lemer, C., Lehéricy, S., Samson, Y., Obadia, M., Slachevsky, A.,Dehaene, S., 2003. Visual word recognition in the left and right hemispheres:anatomical and functional correlates of peripheral alexias. Cereb. Cortex 13,1313–1333.

Collins, A., Loftus, E., 1975. A spreading–activation theory of semantic processing.Psychol. Rev. 82, 407–428.

Copland, D., deZubicaray, G., McMahon, K., Wilson, S., Eastburn, M., Chenery, H., 2003.Brain activity during automatic semantic priming revealed by event-relatedfunctional magnetic resonance imaging. NeuroImage 20, 302–310.

Damasio, A., 1989. Time-locked multiregional retroactivation: a systems-level proposalfor the neural substrates of recall and recognition. Cognition 33, 25–50.

Dehaene, S., Leclech, G., Poline, J., Bihan, D.L., Cohen, L., 2002. The visual word form area:a prelexical representation of visual words in the fusiform gyrus. Neuroreport 13,321–325.

Dehaene, S., Naccache, L., Cohen, L., LeBihan, D., Mangin, J., Poline, J., Rivere, D., 2001.Cerebral mechanisms of word masking and unconscious repetition priming. Nat.Neurosci. 4, 752–758.

Demb, J., Desmond, J., Wagner, A., Vaidya, C., Glover, G., Gabrieli, J., 1995. Semanticencoding and retrieval in the left inferior prefrontal cortex: a functional MRI studyof task difficulty and process specificity. J. Neurosci. 15, 5870–5878.

Desimone, R., Duncan, J., 1995. Neural mechanisms of selective visual attention. Ann.Rev. Neurosci. 18, 193–222.

Devlin, J., Russell, R., Davis, M., Price, C., Wilson, J., Moss, H., Matthews, P., Tyler, L., 2000.Susceptibility-induced loss of signal: comparing PET and fMRI on a semantic task.Neuroimage 11, 589–600.

Diaz, M., McCarthy, G., 2007. Unconscious word processing engages a distributednetwork of brain regions. J. Cogn. Neurosci. 19, 1768–1775.

Duncan, J., Owen, A., 2000. Common regions of the human frontal lobe recruited bydiverse cognitive demands. Trends Neurosci. 23, 475–483.

Eddy, M., Schnyer, D., Schmid, A., Holcomb, P., 2007. Spatial dynamics of masked picturerepetition effects. Neuroimage 34, 1723–1732.

Edmister, W., Talavage, T., Ledden, P., Weisskoff, R., 1999. Improved auditory corteximaging using clustered volume acquisitions. Hum. Brain Mapp. 7, 89–97.

Fiez, J., Petersen, S., 1998. Neuroimaging studies of word reading. Proc. Natl. Acad. Sci.U.S.A. 95, 914–921.

Fiez, J., Tranel, D., Seager-Frerichs, D., Damasio, H., 2006. Specific reading andphonological processing deficits are associated with damage to the left frontaloperculum. Cortex 42, 624–643.

Fletcher, P., Happe, F., Frith, U., Baker, S., Dolan, R., Frackowiak, R., Frith, C., 1995. Otherminds in the brain: a functional imaging study of “theory of mind” in storycomprehension. Cognition 57, 109–128.

Forster, K., 1998. The pros and cons of masked priming. J. Psycholinguist. Res. 27,203–232.

Friston, K., Harrison, L., Penny, W., 2003. Dynamic causal modelling. NeuroImage 19,1273–1302.

Friston, K., Holmes, A., Worsley, K., Poline, J., Frith, C., Heather, J., Frackowiak, R., 1995.Statistical parametric maps in functional imaging: a general approach. Hum. BrainMapp. 2, 189–210.

Friston, K., Zarahn, E., Josephs, O., Henson, R., Dale, A., 1999. Stochastic designs in event-related fMRI. NeuroImage 10, 607–619.

Frith, C., Perry, R., Lumer, E., 1999. The neural correlates of conscious experience: anexperimental framework. Trends Cogn. Sci. 3, 105–114.

Gabrieli, J., Poldrack, R., Desmond, J., 1998. The role of left prefrontal cortex in languageand memory. Proc. Natl. Acad. Sci. U.S.A. 95, 906–913.

Gaillard, R., Naccache, L., Pinel, P., Clémenceau, S., Volle, E., Hasboun, D., Dupont, S.,Baulac, M., Dehaene, S., Adam, C., Cohen, L., 2006. Direct intracranial, fMRI andlesion evidence for the causal role of left inferotemporal cortex in reading. Neuron50, 191–204.

Garrard, P., Carroll, E., 2006. Lost in semantic space: a multi-modal, nonverbalassessment of feature knowledge in semantic dementia. Brain 129, 1152–1163.

Gold, B., Balota, D., Jones, S., Powell, D., Smith, C., Andersen, A., 2006. Dissociation ofautomatic and strategic lexical–semantics: functional magnetic resonance imagingevidence for differing roles of multiple frontotemporal regions. J. Neurosci. 26,6523–6532.

Goldberg, R., Perfetti, C., Fiez, J., Schneider, W., 2007. Selective retrieval of abstractsemantic knowledge in left prefrontal cortex. J. Neurosci. 27, 3790–3798.

Grainger, J., Holcomb, P., 2009. Watching the word go by: on the time-course ofcomponent processes of visual word recognition. Lang. Linguist. Compass 1,128–156.

Haynes, J., Driver, J., Rees, G., 2005. Visibility reflects dynamic changes of effectiveconnectivity between V1 and fusiform cortex. Neuron 46, 811–821.

Hodges, J., Patterson, K., Oxbury, S., Funnell, E., 1992. Semantic dementia: progressivefluent aphasia with temporal lobe atrophy. Brain 115, 1783–1806.

Holcomb, P., Grainger, J., 2006. On the time course of visual word recognition: an event-related potential investigation using masked repetition priming. J. Cogn. Neurosci.18, 1631–1643.

Howard, D., Patterson, K., 1992. Pyramids and Palm Trees: A Test of Semantic Accessfrom Pictures and Words. Thames Valley Publishing.

Humphreys, G., Forde, E., 2001. Hierarchies, similarity and interactivity in objectrecognition. Behav. Brain Sci. 24, 453–476.

Jackendoff, R., 2007. Language, Consciousness, Culture. Essays on Mental Structure. MITPress, Cambridge, MA.

Kanwisher, N., 2001. Neural events and perceptual awareness. In: Dehaene, S. (Ed.), TheCognitive Neuroscience of Consciousness. Elsevier Science publishers, Amsterdam,pp. 89–113.

Kircher, T., Sass, K., Sachs, O., Krach, S., 2009. Priming words with pictures: neuralcorrelates of semantic associations in a cross-modal priming task using fMRI. Hum.Brain Mapp. 30, 4116–4128.

MacMillan, N., Creelman, C., 1990. Response bias: characteristics of detection theory,threshold theory and “non-parametric” indexes. Psychol. Bull. 107, 403–413.

Marois, R., Yi, D., Chun, M., 2004. The neural fate of consciously perceived and missedevents in the attentional blink. Neuron 41, 465–472.

Miller, E., 2000. The prefrontal cortex and cognitive control. Nat. Rev. Neurosci. 1,59–65.

Neely, J., 1977. Semantic priming and retrieval from lexical memory: roles ofinhibitionless spreading activation and limited-capacity attention. J. Exp. Psychol.Gen. 106, 226–254.

Nichols, T., Brett, M., Andersson, J., Wager, T., Poline, J., 2005. Valid conjunctioninference with the minimum statistic. Neuroimage 25, 653–660.

Nixon, P., Lazarova, J., Hodinott-Hill, I., Gough, P., Passingham, R., 2004. The inferiorfrontal gyrus and phonological processing: an investigation using rTMS. J. Cogn.Neurosci. 16 (2), 289–300.

Oldfield, R., 1971. The assessment and analysis of handedness: the EdinburghInventory. Neuropsychologia 9, 97–113.

Paulesu, E., Frith, C., Frackowiak, R., 1993. The neural correlates of the verbal componentof working memory. Nature 362, 342–345.

Penny, W., Stephan, K., Mechelli, A., Friston, K., 2004. Comparing dynamic causalmodels. NeuroImage 22, 1157–1172.

Poldrack, R., Wagner, A., Prull, M., Desmond, J., Glover, G., Gabrieli, J., 1999. Functionalspecialization for semantic and phonological processing in the left inferiorprefrontal cortex. NeuroImage 10, 15–35.

Poline, J., Worsley, K., Evans, A., Friston, K., 1997. Combining spatial extent and peakintensity to test for activations in functional imaging. NeuroImage 5, 83–96.

Price, C., 1998. The functional anatomy of word comprehension and production. TrendsCogn. Sci. 2, 281–288.

Price, C., Devlin, J., 2004. The pro and cons of labelling a left occipitotemporal region‘visual word form area’. Neuroimage 22, 477–479.

Qiao, E., Vinckier, F., Szwed, M., Naccache, L., Valabrégue, R., Dehaene, S., Cohen, L.,2010. Unconsciously deciphering handwriting: subliminal invariance for hand-written words in the visual word form area. Neurimage 49, 1786–1799.

Race, E., Shanker, S., Wagner, A., 2009. Neural priming in human frontal cortex: multipleforms of learning reduce demands on the prefrontal executive system. J. Cogn.Neurosci. 21, 1766–1781.

Raposo, A., Moss, H., Stamatakis, E., Tyler, L., 2006. Repetition suppression andsemantic enhancement: an investigation of the neural correlates of priming.Neuropsychologia 44, 2284–2295.

Rapp, B., Hillis, A., Caramazza, A., 1993. The role of representations in cognitivetheory: more on multiple semantics and the agnosias. Cogn. Neuropsychol. 10,235–249.

Riddoch, M., Humphreys, G., 1993. Birmingham Object Recognition Battery. LawrenceErlbaum Associates Ltd, Hove, UK.

Rissman, J., Eliassen, J., Blumstein, S.E., 2003. An event-related fMRI investigation ofimplicit semantic priming. J. Cogn. Neurosci. 15, 1160–1175.

Rogers, T., McClelland, J., 2004. Semantic Cognition. A Parallel Distributed Approach.MIT Press, Cambridge, MA.

Ruff, I., Blumstein, S., Myers, E., Hutchison, E., 2008. Recruitment of anterior andposterior structures in lexical–semantic processing: an fMRI study comparingimplicit and explicit tasks. Brain Lang. 105, 41–49.

Sachs, O., Weis, S., Zellagui, N., Huber, W., Zvyagintsev, M., Mathiak, K., Kircher, T.,2008. Automatic processing of semantic relations in fMRI: neural activationduring semantic priming of taxonomic and thematic categories. Brain Res. 1218,194–205.

Sass, K., Krach, S., Sachs, O., Kircher, T., 2009. Lion–tiger–stripes: neural correlates ofindirect semantic priming across processing modalities. Neuroimage 45, 224–236.

Snodgrass, J., Vanderwart, M., 1980. A standardized set of 260 pictures: norms for nameagreement, image agreement, familiarity and visual complexity. J. Exp. Psychol.Hum. Learn. Mem. 6, 174–215.

Stephan, K., Marshall, J., Penny, W., Friston, K., Fink, G., 2007. Inter-hemisphericintegration of visual processing during task-driven lateralization. J. Neurosci. 28,3512–3522.

Thierry, G., Price, C., 2006. Dissociating verbal and nonverbal conceptual processing inthe human brain. J. Cogn. Neurosci. 18, 1018–1028.

Vandenberghe, R., Nobre, A., Price, C., 2002. The response of left temporal cortex tosentences. J. Cogn. Neurosci. 14, 550–560.

Vandenberghe, R., Price, C.,Wise, R., Josephs, O., Frackowiak, R., 1996. Functional anatomyof a common semantic system for words and pictures. Nature 383, 254–256.

Vandenbulcke, M., Peeters, R., Dupont, P., Van Hecke, P., Vandenberghe, R., 2007. Wordreading and posterior temporal dysfunction in amnestic mild cognitive impair-ment. Cereb. Cortex 17, 542–551.

3307L. Van Doren et al. / NeuroImage 49 (2010) 3295–3307

Vandenbulcke, M., Peeters, R., Fannes, K., Vandenberghe, R., 2006. Knowledge of visualattributes in the right hemisphere. Nat. Neurosci. 9, 964–970.

Vandenbulcke, M., Peeters, R., Hecke, P.V., Vandenberghe, R., 2005. Anterior temporallaterality in primary progressive aphasia shifts to the right. Ann. Neurol. 58, 362–370.

Van Essen, D.C., 2005. A population-average, landmark- and surface-based atlas ofhuman cerebral cortex. Neuroimage 28, 635–662.

Vinckier, F., Dehaene, S., Jobert, A., Dubus, J., Sigman, M., Cohen, L., 2007. Hierarchicalcoding of letter strings in the ventral stream: dissecting the inner organization ofthe visual word-form system. Neuron 55, 143–156.

Wagner, A., Desmond, J., Demb, J., Glover, G., Gabrieli, J., 1997. Semantic repetitionpriming for verbal and pictorial knowledge: a functional MRI study of left inferiorprefrontal cortex. J. Cogn. Neurosci. 9, 714–726.

Wagner, A., Koutstaal, W., Maril, A., Shacter, D., Buckner, R., 2000. Task-specificrepetition priming in left inferior prefrontal cortex. Cereb. Cortex 10, 1176–1184.

Wagner, A., Koutstaal,W., Shacter, D., 1999.When encoding yields remembering: insightsfrom event-related neuroimaging. Phil. Trans. R. Soc. Lond. B 354, 1307–1324.

Wilenius-Emet, M., Revonsuo, A., Ojanen, V., 2004. An electrophysiological correlate ofhuman visual awareness. Neurosci. Lett. 354, 38–41.