Multisensory stimulation with or without saccades: fMRI evidence for crossmodal effects on...

www.elsevier.com/locate/ynimg

NeuroImage 26 (2005) 414–425

Multisensory stimulation with or without saccades: fMRI evidence for

crossmodal effects on sensory-specific cortices that reflect

multisensory location-congruence rather than task-relevance

E. Macaluso,a,T C.D. Frith,b and J. Driverb,c

aFondazione Santa Lucia, Istituto di Ricovero e Cura a Carattere Scientifico, Via Ardeatina, 306-00179 Rome, ItalybWellcome Department of Imaging Neuroscience, Institute of Neurology, London, UKcDepartment of Psychology, Institute of Cognitive Neuroscience, University College London, London, UK

Received 11 June 2004; revised 29 November 2004; accepted 8 February 2005

Available online 29 March 2005

During covert attention to peripheral visual targets, presenting a

concurrent tactile stimulus at the same location as a visual target can

boost neural responses to it, even in sensory-specific occipital areas.

Here, we examined any such crossmodal spatial-congruence effects in

the context of overt spatial orienting, when saccadic eye-movements

were directed to each peripheral target or central fixation maintained.

In addition, we tested whether crossmodal spatial-congruence effects

depend on the task-relevance of visual or tactile stimuli. On each trial,

subjects received spatially congruent (same location) or incongruent

(opposite hemifields) visuo-tactile stimulation. In different blocks, they

made saccades either to the location of each visual stimulus, or to the

location of each tactile stimulus; or else passively received the

multisensory stimulation. Activity in visual extrastriate areas and in

somatosensory parietal operculum was modulated by spatial congru-

ence of the multisensory stimulation, with stronger activations when

concurrent visual and tactile stimuli were both delivered at the same

contralateral location. Critically, lateral occipital cortex and parietal

operculum showed such crossmodal spatial effects irrespective of which

modality was task relevant; and also of whether the stimuli were used

to guide eye-movements or were just passively received. These results

reveal crossmodal spatial-congruence effects upon visual and somato-

sensory sensory-specific areas that are relatively dautomaticT, deter-mined by the spatial relation of multisensory input rather than by its

task-relevance.

D 2005 Elsevier Inc. All rights reserved.

Keywords: Space; Multisensory; Vision; Touch; Saccades; fMRI

Introduction

Events and objects in the external world can produce multi-

sensory signals that the brain registers via several distinct sensory

1053-8119/$ - see front matter D 2005 Elsevier Inc. All rights reserved.

doi:10.1016/j.neuroimage.2005.02.002

T Corresponding author. Fax: +39 06 5150 1213.

E-mail address: [email protected] (E. Macaluso).

Available online on ScienceDirect (www.sciencedirect.com).

modalities. The signals for each sensory modality will initially be

processed in anatomically distant cortical brain areas. But to

achieve optimal behavior and produce appropriate responses,

signals in different modalities that relate to a single event or

object in the external world will often have to be integrated (Stein

and Meredith, 1993). Many different factors are known to play a

role in multisensory integration, such as the location of the sources

(Meredith and Stein, 1996) and the relative timings of multisensory

signals (Meredith et al., 1987). On the spatial aspects, many

behavioral studies have now demonstrated that the relative location

of two stimuli in different sensory modalities can affect perform-

ance. For example, in the case of crossmodal spatial effects

between vision and touch, Spence et al. (1998) showed that tactile

stimulation on one hand can improve judgement of visual targets

presented near to the stimulated hand, compared to visual targets

presented near to the opposite hand (see also Driver and Spence,

1998; McDonald et al., 2000). Electro-physiological studies in

animals have demonstrated the existence of multisensory neurons

that can respond to stimuli in more than one modality (Bruce et al.,

1981; Duhamel et al., 1998; Graziano and Gross, 1995). Critically,

the activity of some of these neurons appears to reflect temporal

and spatial relations between multisensory stimuli in the external

world, for example, showing modulation of responses according to

relative position of the unimodal sources in space (e.g., under- or

over-additive responses to multisensory versus unimodal inputs;

Meredith and Stein, 1986a,b; Stein and Meredith, 1993).

Some human neuroimaging studies have sought to identify

candidate multisensory regions in the human brain. One approach

has been to stimulate one or other modality at a time, and analyse

for areas that respond not just to one modality but to several or to

all. This approach has revealed multimodal responses in several

brain areas, including intraparietal sulcus, inferior parietal cortex,

superior temporal sulcus and premotor regions (Bremmer et al.,

2001; Macaluso and Driver, 2001), in agreement with the single

cell literature reporting multimodal neurons in these regions (Bruce

et al., 1981; Duhamel et al., 1998; Graziano and Gross, 1995). To

E. Macaluso et al. / NeuroImage 26 (2005) 414–425 415

date, relatively few human neuroimaging studies with combined

multisensory stimulation have shown supra-additive responses

(i.e., greater than the combination of the responses to each

modality alone in such regions; though see Calvert et al., 2000,

for one example).

Several studies have manipulated the spatial congruence of

concurrent bimodal stimuli (i.e., unimodal sources at the same

versus different locations), but instead of observing crossmodal

effects primarily in heteromodal association cortices, have instead

reported crossmodal modulations for spatially congruent multi-

sensory stimulation arising within what would traditionally be

regarded as unimodal, sensory-specific cortices (Macaluso et al.,

2000b, 2002a; see also Misaki et al., 2002). This implies that

spatial multisensory interactions may not only involve brain areas

traditionally considered to be heteromodal, but may also affect

sensory-specific cortex (see Spence and Driver, 2004, for reviews

and discussion).

Macaluso et al. (2000b) presented subjects with left or right

visual targets for covert attention and detection (i.e., without any

overt orienting) during fMRI. Unpredictably, on some trials, a task-

irrelevant tactile stimulation was delivered either at the position of

the visual target or in the opposite hemifield. The results showed

that combining visual and tactile stimuli at the same location

(bimodal stimulation that was dcongruentT in this way) resulted in

increased activity in occipital cortex contralateral to the stimulated

side. When the visual target and the task-irrelevant tactile

stimulation were delivered in opposite hemifields, this amplifica-

tion did not occur, demonstrating the spatial nature of this

crossmodal effect upon unimodal visual cortex (see also Macaluso

et al., 2002a).

Crossmodal spatial congruence not only affects perceptual

judgments in covert spatial attention tasks (e.g., McDonald et al.,

2000; Spence et al., 1998), but can also influence overt spatial

orienting, such as saccades. For example, Diederich et al. (2003)

measured the effect of task-irrelevant tactile stimuli on saccadic

reaction times (RT) to visual targets. Tactile stimulation to a hand

placed in the same hemifield as the visual target resulted in faster

saccadic RTs, compared to tactile stimuli to the hand placed in the

opposite visual hemifield (see also Amlot et al., 2003). Rorden

et al. (2002) provided further behavioral evidence suggesting

possible links between crossmodal effects observed during covert

orienting tasks and those reported during overt saccadic tasks. In

their study, participants performed leftward or rightward saccades

depending on the position of a peripheral visual cue. After the

visual onset, but before initiation of the saccade, a task-relevant

tactile target was presented either at the location of the visual target

(congruent spatial configuration) or in the opposite hemifield

(incongruent configuration). The task of the subject was to saccade

toward the visual cue, and then perform a perceptual discrimination

regarding the tactile target (up/down judgement, thus judging a

property orthogonal to target or saccadic side; see Spence and

Driver, 1997). Again, the results showed that the spatially

congruent situation (touch at the same location of the impending

saccade) yielded better performance for the tactile judgement. This

indicates a possible relationship between perceptual crossmodal

enhancements typically observed in covert attention tasks, and

facilitatory effects observed during overt saccadic tasks.

The aim of the present fMRI study was to investigate neural

crossmodal spatial effects between vision and touch, now during

presence (or absence) of overt saccadic spatial orienting, and also to

determine the extent to which these may depend on the task-

relevance of either modality. On each trial here, subjects received

concurrent bimodal visuo-tactile stimulation that was either

spatially congruent (i.e., vision and touch at the same location) or

spatially incongruent (with concurrent visual and tactile stimuli

presented in opposite hemifields). In different fMRI scanning

sessions, subjects were instructed either to saccade to the position of

the visual stimulus (vision relevant, ignoring any tactile stimulus);

or to saccade to the position of the tactile stimulus instead (touch

relevant, now ignoring any visual stimulus); or to maintain central

fixation and receive the stimuli passively (peripheral stimuli now

task-irrelevant in both modalities). Comparing brain activity for

spatially congruent versus spatially incongruent stimulation should

reveal brain regions affected by crossmodal processes that depends

on the relative position of concurrent multisensory stimuli (as found

by Macaluso et al., 2000b, 2002a, for regions of visual cortex in a

covert-attention visual detection task). Critically, the inclusion here

of tasks requiring overt saccadic orienting (to visual or tactile

stimuli), plus a passive control task that did not require any spatial

orienting, should allow us to separate crossmodal spatial effects that

depend solely on the spatial stimulus configuration, versus those

that depend on the task-relevance of sensory information for guiding

overt spatial orienting. Note that in the current paradigm the relevant

modality (i.e., saccade to vision or saccade to touch) was blocked,

and served only as a context to study spatial interactions between

vision and touch. Thus, the brain activations of main interest

reported here will reflect sensory interactions between vision and

touch, rather than sensory-motor congruency effects per se.

Methods

Subjects

Eleven volunteers participated (7 males and 4 females). All but

one were right-handed, with mean age of 23 years (range 18–32).

After receiving an explanation of the procedures, subjects gave

written informed consent, in a protocol approved by the Joint

Ethics Committee of the Institute of Neurology and the National

Hospital for Neurology and Neurosurgery.

Paradigm

Functional MRI data were acquired during presentation of four

event types, under 3 types of instruction (i.e., 3 blocked task

conditions). The four event-types were bimodal visuo-tactile

stimulation organised according to a 2 � 2 factorial design, with

the side of touch (left or right hand) and the side of vision (left or

right visual hemifield) as crossed independent factors. Hence, for

two event-types, the bimodal visuo-tactile stimulation was spatially

congruent in location (touch and vision on the same side, either

both left or both right), and for the other two types, the simulation

was spatially incongruent in location (with stimuli in the two

modalities located in opposite hemifields; visual on left and tactile

on right, or vice-versa). The order of these four event-types was

randomised and unpredictable.

These four events were presented under 3 types of instructions

(blocked tasks): saccade to the location of the tactile stimulus;

saccade to the location of the visual stimulus; or maintain central

fixation (i.e., do not respond to any of the stimuli). The three

different tasks were presented in separate fMRI scanning sessions,

with the instruction regarding the current task given verbally before

E. Macaluso et al. / NeuroImage 26 (2005) 414–425416

the start of each session, and eye-tracking implemented (see below)

to confirm adherence to the task.

Stimuli and task

Subjects lay in the scanner with each hand resting on a plastic

support placed on top of the RF-coil. Each hand rested on the

corresponding side and on each side there was an LED to present

visual stimuli, and a piezoelectric component (T220-H3BS-304,

Piezo Systems Inc., Cambridge, USA) to deliver unseen and

inaudible tactile stimulation to the index finger. To avoid the

transmission of any vibration from the piezoelectric device to its

support or the RF-coil, four MR-compatible springs were placed

between each tactile device and the coil. This ensured that the

activation of the vibrators did not result in any acoustical

stimulation, which could otherwise have arisen because of

vibrations being transmitted between various parts of the appara-

tus. Between this apparatus and the subject’s eyes there was an

opaque screen so that subjects could not see either their hands or

the LEDs when unilluminated (the latter became visible only when

illuminated). The LEDs were placed at approximately 248 visual

angle to left or right of the central midline and were visible with

both eyes. The tactile stimulators and index fingers were located

immediately behind the LEDs. This allowed us to deliver visual

and tactile stimuli in close spatial correspondence when both

stimuli were on the same side (dcongruentT). A cross drawn on the

opaque screen served as a central fixation point. In addition, a

mirror was also placed on top of the RF-coil, to allow monitoring

of eye-position throughout the experiment, with a remote optics

eye-tracker (see below).

On each trial, concurrent bimodal visuo-tactile stimulation was

presented for 50 ms. This stimulation could be either spatially

congruent (both stimuli at the same location) or incongruent (again

concurrent stimulation of vision and touch, but now in the two

opposite hemifields). According to the instruction (current task,

blocked), the subject either performed a saccade to the position of

the unseen tactile target (ignoring any visual stimulus), or

performed a saccade to the position of the visual target (ignoring

any tactile stimulus) or simply maintained central fixation (thus not

responding to either type of stimulus). Because the stimulus

sequence was randomised, on each trial, the saccade direction was

unpredictable until presentation of the stimulation for that trial.

Moreover, the short time of target presentation (50 ms) and the fact

that the hands and unilluminated LEDs were not visible to the

subject meant that by the time the saccade was initiated, the target

position for the saccade was no longer marked in any way (hence,

there was no trial-type-specific stimulation still visible to undergo a

shift in retinal position due to the saccade). After the saccade to the

target location, subjects made an eye-movement back to central

fixation, so the current design cannot discriminate between brain

activity for leftward versus rightward saccades. However, this

would be beyond the aim of the present study, which sought

instead to measure brain activity for sensory stimulation (vision

and touch) at the same versus different locations, under different

task conditions.

The mean inter-trial interval was 4 s (range 3–5 s, with a

uniform distribution). During each session, there were 60 trials,

with 15 repetitions for each of the four trial-types (i.e., spatially

congruent visual–tactile stimulation on the left, spatially congruent

on the right, touch on the left plus vision on the right, and touch

on the right plus vision on the left). Each subject underwent six

separate scanning sessions (each lasting approximately 4.5 min),

thus repeating each of the three tasks twice (saccade to touch,

saccade to vision, or passive central fixation). All three tasks were

presented once in the first 3 fMRI-sessions, and once in the

second three fMRI-sessions, but in a different order. Across

subjects, all six permutations of the three tasks were used, thus

counterbalancing the order of task presentation within and across

subjects.

Image acquisition

Functional images were acquired with a 1.5-T SONATA MRI

scanner (Siemens, Erlangen, Germany). BOLD (blood oxygenation

level dependent) contrast was obtained using echo-planar T2*

weighted imaging (EPI). The acquisition of 32 transverse slices

gave coverage of the whole cerebral cortex. Repetition time was

2.88 s. The in-plane resolution was 3 � 3 mm.

Data analysis

Data were analysed with SPM2 (http://www.fil.ion.ucl.ac.uk).

For each subject, the 546 volumes were realigned with the first

volume, and acquisition timing was corrected using the middle

slice as reference (Henson et al., 1999). To allow inter-subject

analysis, images were normalised to the Montreal Neurological

Institute (MNI) standard space (Collins et al., 1994), using the

mean of the 546 functional images. All images were smoothed

using an 8 mm isotropic Gaussian kernel.

Statistical inference was based on a random effects approach

(Holmes and Friston, 1998). This comprised two steps. First, for

each subject, the data were best-fitted at every voxel using a linear

combination of effects of interest, plus confounds. The effects of

interest were the timing of the four event-types (given by crossing

of the 2 stimulus factors: side of touch and side of vision), in each

of the six sessions (two sessions for each of the three tasks). Trials

containing any incorrect saccadic behaviour were modeled as

confounds (see below). All event-types were convolved with the

SPM2 standard haemodynamic response function. Linear com-

pounds (contrasts) were then used to determine the main effect of

position of the tactile stimulus and the main effect of position of

the visual stimulus separately for each of the three tasks. This led to

the creation of six contrast-images for each subject. Note that in the

present design, these main effects of stimulus side for each

modality are mathematically equivalent to the interactions between

stimulus side of one modality and the spatial congruence of the

multimodal stimulation (see below). Furthermore, these contrasts

reflect differential effects (e.g., touch on the left versus touch on

the right, during central fixation), and therefore any effect specific

to just subject or session will automatically be removed from

further analyses. These contrast-images then underwent the second

step, which comprised an ANOVA that modeled the mean of each

of the six differential effects (see below for further details). Finally,

linear compounds were used to compare these effects, now using

between-subjects variance (rather than between scans). Correction

for non-sphericity (Friston et al., 2002) was used to account for

possible differences in error variance across conditions and any

non-independent error terms for the repeated measures.

Our main analyses then aimed to identify brain regions affected

by the relative position of the tactile and visual stimuli (congruent:

same location, versus incongruent: opposite side), and to assess

whether any such crossmodal spatial effect was common to all

http:\\www.fil.ion.ucl.ac.uk

Table 1

Effect of the side of the sensory stimulation for touch and vision

Anatomical area Co-ordinates Z values Cluster

size

P values

LEFT minus right SOMATOSENSORY stimulation

Right post-central

gyrus

44 �30 62 5.4 246 0.002

Right parietal

operculum

42 �24 4 4.4 179 0.009

RIGHT minus left SOMATOSENSORY stimulation

Left post-central

gyrus

�48 �22 56 4.9 177 0.010

Left parietal

operculum

�64 �22 18 4.5 509 b0.001

LEFT minus right VISUAL stimulation

Right middle

occipital gyrus

46 �68 8 5.8 2649 b0.001

Right superior

occipital gyrus

22 �86 36 6.2

Right lingual/

fusiform gyrus

22 �68 14 5.6

Right striate cortex 12 �78 6 4.7 195 0.006

RIGHT minus left VISUAL stimulation

Left middle

occipital gyrus

�50 �74 0 3.7 82 0.184

Left superior

occipital gyrus

�14 �84 24 4.0 73 0.248

Left lingual/

fusiform gyrus

�34 �62 �10 4.0 43 0.625

Left intraparietal

sulcus

�24 �36 50 4.5 102 0.096

Left inferior

premotor cortex

�50 8 28 3.7 57 0.415

Anatomic areas, Talairach co-ordinates of the maxima within each cluster,

Z values, cluster sizes and corrected P values of the regions that showed a

main effect of the side of the sensory stimulation. For each modality, we

directly compared stimulation of one versus the other hemifield, irrespec-

tive of spatial congruence and current task. For the main effect of right

minus left visual stimulation, none of the clusters survived correction for

multiple comparisons, but the pattern of activation appeared convincingly

lateralised to the contralateral occipital cortex. Coordinates in millimeters:

x, distance to right (+) or left (�) of the mid-sagittal plane; y, distance

anterior (+) or posterior (�) to vertical plane through anterior commissure;

z, distance above (+) or below (�) intercommissural (AC–PC) line.


three types of task or was instead specific to just some or one of

them. To reveal any modulation of brain responses when touch and

vision were presented at the same contralateral location (as

previously reported in Macaluso et al., 2000b, 2002a), we tested

for a conjoint main effect of side of touch and main effect of side of

vision (Friston et al., 1999; Price and Friston, 1997). Note that for

this purpose, our basic 2 � 2 stimulus design (with side of touch

and side of vision as independent factors) can be redefined as a 2 �2 design with side of touch (or equivalently, side of vision) and

spatial congruence of the bimodal visuo-tactile stimulation as

independent factors. Accordingly, the test conducted for conjoint

main effects of side for the two modalities is mathematically

identical to testing for the effect of side for one modality (touch or

vision) in presence of an interaction between side of that modality

and spatial congruence of the bimodal stimulation. Therefore, this

will specifically highlight brain areas where the effect of side for

one modality was larger when the other modality was on the same

side, compared with the other modality being on the opposite side,

as can be predicted given the prior results of Macaluso et al.

(2000b, 2002a).

To assess whether any such crossmodal–spatial modulation

was common to all three tasks, we again used conjunction

analyses (Friston et al., 1999; Price and Friston, 1997) that

included the main effects of side of touch and main effect of side

of vision for all three tasks. These conjunctions between all six

effects modeled at the second-level (random effects) analysis will

reveal brain regions that show specifically crossmodal spatial

effects of the predicted type (larger effect of stimulus side for

crossmodally congruent stimulation), irrespectively of the current

task. Conversely, to test for crossmodal spatial modulations that

were specific for one (or two) of the three tasks, we tested for

the conjunction of the two main effects of side within one (or

two) task(s) only, with the additional constraint that this conjoint

effect had to be larger during the task(s) of interest compared

with the other task(s). For this additional constraint (which can

only make our analyses more conservative, since it is additional

to the main contrast), a threshold of P-uncorr. = 0.01 was

adopted.

For all comparisons corrected P values were assessed using a

small volume correction procedure (Worsley et al., 1996). Given

our specific interest in any crossmodal spatial-congruence effects

upon sensory-specific cortex (in line with the previous results of

Macaluso et al., 2000a, 2002b), the search volumes consisted of

somatosensory and visual areas contralateral to the location of the

critical spatially-congruent bimodal stimulation. These regions

were highlighted using the main effects of side (left or right) for

one or the other modality across all three tasks. As an initial

definition of such regions, correction at the cluster-level was used

(P-corr = 0.05, cluster size set by thresholding the SPM-maps at

P-uncorrected = 0.001 for the voxel level). For the particular main

effect of right visual stimulation, no cluster survived this cluster-

level correction procedure (P-corr = 0.05), so for completeness,

we dropped the constraint concerning correction for multiple

comparisons, thus considering only the voxel-level threshold

(P-uncorrected = 0.001 as before). Note that for this particular

case of seeking left occipital regions responding to right visual

stimulation, the issue of correction for multiple comparison is

moot because of the innumerable prior studies showing contrala-

teral occipital responses for lateralised visual stimulation. The

location and the extent of the volumes of interest are reported in

Table 1 and Fig. 2.

Eye tracking

Eye-position was monitored using an ASL Eye-Tracking

System (Applied Science Laboratories, Bedford, USA), with

remote optics (Model 504, sampling rate = 60 Hz) that was

custom-adapted for use in the scanner. Eye-position data were

analysed for 10 out of 11 subjects, for whom reliable eye-position

was available throughout all imaging sessions. Eye-position traces

were examined in a 2100-ms window, beginning 100 ms prior to

stimulus onset. For trials requiring central fixation (central fixation

task), losses of fixation were identified using the derivative of the

horizontal eye-position trace (i.e., saccade velocity). When this

exceeded 508/s, the trial was considered a non-fixation trial and

was modeled separately in the fMRI analysis as an erroneous

saccade (exclusion rate: 20.8%). Note that the use of a velocity

criterion, rather than eye-position, should make our exclusion


procedure sensitive also to small amplitude eye-movements (even

micro-saccades), and was thus conservative. For trials that did

require shifts of gaze direction (i.e., saccades to tactile or saccades

to visual targets), we identified trials where either the subject made

a saccade to the wrong side (e.g., saccade to the visual target

during a spatially-incongruent trial under instructions to saccade to

touch) or did not perform any saccade at all. Again, these trials

were modeled separately in the fMRI analysis. Their rates were:

5.8% wrong saccade direction during tactile task, 1% wrong

saccade direction during the visual task and overall 0.9% of

no-response trials across the two saccadic tasks. For trials requiring

saccadic responses, eye-velocity was used to compute saccadic

reaction time, here defined as the time between target onset and

eye-velocity first exceeding 508/s. Note that saccade-error trials

during spatially incongruent stimulation (5.8% during saccade to

touch), might in principle be associated with interesting brain

processes (e.g., failure to suppress some stimulus-driven saccadic

mechanisms). However, we could not assess this in our imaging

data because there were too few of these error-trials. Future studies

may use weaker stimuli in the relevant modality and more salient

distracters in order to produce more saccades to the wrong position,

thus allowing for the analysis of brain activity associated with this

potentially interesting type of error trial (Amador et al., 2004;

Curtis and D’Esposito, 2003).

Results

Behavioral performance

Fig. 1 shows horizontal eye-position traces for the 10/11 subjects

for whom reliable eye-tracking recording was available. The traces

are divided according to the current task (saccade to touch, saccade

to vision, or central fixation) and the spatial congruence of the

bimodal stimulation (left congruent trials in red, right congruent

trials in green and spatially incongruent trials in black). Generally

subjects performed well, with only 5.8% wrong saccade direction

during the tactile task, and 1% wrong saccade direction during the

Fig. 1. Eye-position for 10 out of 11 participating subjects for whom this was avail

the task performed (left panel: saccade to touch; central panel: saccade to vision; ri

are in red (both vision and touch on the left side) and in green (both vision and tou

these all show correct saccadic directions: for example, traces deflecting toward th

vision was on the left. Eye-position traces are time-locked to the saccade-onset f

stimulus-onset for the central fixation trials (rightmost plot). For each subject, as exp

(left and central panels), but no systematic change for trials requiring central fixatio

were removed (see Methods). All traces were adjusted using 100 ms pre-stimulu

stimulation was approximately 248.

visual task. Analysis of saccadic reaction times revealed: (a) a main

effect of spatial congruency (F(9,1) = 17.03, P = 0.003), with faster

saccades for spatially congruent trials compared to incongruent trials

(mean RT: congruent = 325 ms, incongruent = 361 ms), consistent

with the prior behavioral studies (Amlot et al., 2003; Diederich et al.,

2003); (b) a main effect of task (F(9,1) = 26.69, P = 0.001), with

saccades to visual targets faster than saccades to tactile targets (mean

RT: vision = 311 ms, touch = 374 ms), again consistent with prior

behavioral studies; (c) interaction between spatial congruency and

task (F(9,1) = 11.34, P = 0.008), with larger congruency effects for

the tactile task (congruent minus incongruent: touch = 63 ms,

vision = 10 ms); and (d) an overall effect of the position of the tactile

stimulus (F(9,1) = 11.71, P = 0.008), with overall saccadic RTs

faster when touch was on the right hand. This marginally interacted

with the current task also (F(1,9) = 4.03, P = 0.076), indicating that it

may have been driven primarily by faster RTs for saccades to the

right hand compared to the left hand, when touch was relevant.

Overall, saccadic RT were slower than those reported in previous

behavioral studies performed outside the scanner (e.g., Diederich et

al., 2003). One possible explanation for this relates to the demanding

environment in which subjects had to perform the task. Unlike

behavioral studies, during fMRI, the subject had to lay still in a dark

and noisy surrounding, often finding the MR-scanning session

rather strenuous. However, we must note that there is no reason to

believe that the fMRI-environment should have any differential

effect for the different conditions. Moreover, the RTs reported here

are consistent with the saccadic RTs we previously measured with a

similar experimental set-up (Macaluso et al., 2003). In summary, the

behavioral data showed the expected effect of spatial congruency,

with faster saccades for spatially congruent bimodal stimulations,

and also showed that saccades to visual targets were generally faster

than saccades to tactile targets, as would be expected.

Imaging data

The analysis of the fMRI data aimed to highlight brain areas

where activity was higher during spatially congruent bimodal

stimulation (i.e., vision and touch at the same location) compared to

able. Horizontal eye-position traces for each subject are plotted according to

ght panel: central fixation). In each panel traces for spatially congruent trials

ch on the right side). Traces in black relate to spatially incongruent trials and

e right during the tactile task refer to trials when touch was on the right and

or the saccade to touch and saccade to vision tasks, and to the time of the

ected the plots shows a sharp change in eye-position during the saccade tasks

n (rightmost panel), once the few trials containing detected losses of fixation

s baseline and no further filtering was used. The position of the peripheral


spatially incongruent trials (vision and touch in opposite hemi-

fields). Critically, we further assessed this in relation to whether the

visual or the tactile stimuli served as task-relevant targets for

saccades, or only passive central fixation was required. Given our

previous results (Macaluso et al., 2000b, 2002a), we were

particularly interested in spatially-specific effects whereby multi-

modal spatial congruence affects spatial representations in con-

tralateral sensory-specific cortices, as we had previously shown for

visual occipital cortex in a covert-attention visual detection task.

Therefore, our analyses first identified brain regions that show

differential activity depending on stimulus position. Table 1 and

Fig. 2 report these spatially-specific effects for tactile and visual

stimulation. Lateralised tactile stimulation to the left or to the right

hand (irrespective of location of the visual stimulus and the current

task) revealed, as expected, activation in the contralateral post-

central gyrus, plus a region comprising the parietal operculum and

the posterior part of the insulae (see Table 1 and Fig. 2A). In the

left hemisphere, the opercular cluster extended dorsally to include

a region in the inferior part of the post-central gyrus (see Fig. 2A

left panel). For visual stimulation, contralateral effects were

observed in ventral, lateral and dorsal occipital cortex (see Table

1 and Fig. 2B). While in the right hemisphere, these effects were

statistically robust for all three regions, in the left hemisphere, none

Fig. 2. Main effect of stimulus position for touch and vision. (A) Somatosenso

stimulation, irrespective of side of vision and current task. (B) Effect of side of the

rendered on the surface of the MNI brain template. These comparisons reveale

stimulated side and they were successively used as volumes of interest to assess

visual or tactile cortex (see Figs. 3 and 4). SPM thresholds are set to P-corr. =

stimulation for which a cluster-level correction procedure had to be dropped to

uncorrected threshold, a left frontal region also appeared to be active (see leftm

activation was not predicted on the basis of previous experiments (unlike the con

of the clusters survived full correction for multiple comparisons.

However, dropping the constraint regarding cluster-level correction

for multiple comparisons, while still maintaining the same voxel-

level threshold (i.e., P-uncorr. = 0.001), revealed a specific pattern

of primarily left occipital–parietal activation with all activations

contralateral to the stimulated side (see Fig. 2B leftmost image).

Thus, overall the lateralised tactile or visual stimulation showed the

expected contralateral activations of sensory-specific cortices in

postcentral and occipital regions, respectively (see Fig. 2).

Subsequent analyses tested whether these spatially specific effects

(i.e., higher activation for contralateral versus ipsilateral stimula-

tion) were modulated by the spatial congruency of the bimodal

stimulation (touch and vision stimulated in either the same or

opposite hemifields).

First, we tested for any such spatial congruency effects that

were common to all three types of tasks (saccade to touch, saccade

to vision, and central fixation). Importantly, this showed that

activity in both extrastriate visual cortex (lateral occipital) and

somatosensory cortex in the parietal operculum were affected by

the spatial congruence of the bimodal stimulation, irrespective of

current task (see Table 2). Fig. 3 shows the anatomical location and

the pattern of activation for these regions (see also Fig. 5). The

signal plots show that the effect of stimulus location (with higher

ry responses for left minus right (red) and right minus left (green) tactile

visual stimuli (red: left minus right; green: right minus left). All clusters are

d the expected activation of sensory-specific cortices contralateral to the

any effects of crossmodal spatial congruence in contralaterally-responsive

0.05 at cluster level, except for the main effect of right versus left visual

reveal the expected activation in left occipital cortex (in green). At this

ost panel). However, this is reported for completeness only because this

tralateral responses in left occipital cortex).

Table 2

Crossmodal spatial effects

Anatomical area Co-ordinates Z values P values

A. Crossmodal effects INDEPENDENT of current task

Right parietal operculum 46 �18 10 3.6 0.036

Left parietal operculum �40 �16 16 2.6 0.552

Right middle occipital gyrus 40 �60 16 3.9 0.088

Left middle occipital gyrus �46 �62 �4 3.6 0.051

Right superior occipital gyrus 24 �82 38 4.3 0.025

B. Crossmodal effects observed only during the FIXATION task

Right lingual/fusiform gyrus 22 �54 �10 3.1 0.385

Left lingual/fusiform gyrus �32 �64 �12 4.7 b0.001

Anatomic areas, Talairach co-ordinates, Z values and corrected P values for

the regions that showed crossmodal spatial effects. (A) Areas showing

crossmodal effects during all three types of task (saccade to touch, saccade

to vision and central fixation). (B) Areas showing crossmodal spatial effects

only during the fixation task. All effects were contralateral to the location of

the spatially congruent bimodal stimulation (touch and vision on the same

side). Coordinates in millimeters: x, distance to right (+) or left (�) of the

mid-sagittal plane; y, distance anterior (+) or posterior (�) to vertical plane

through anterior commissure; z , distance above (+) or below (�)

intercommissural (AC–PC) line.


activity for stimulation of the contralateral side) was significantly

larger when the concurrent multisensory stimuli were at the same-

contralateral-location compared to when one stimulus was con-

tralateral and the other ipsilateral. In all plots, the effect of stimulus

side specific to congruent multimodal stimulation is represented by

the difference between the first two bars (shown in red and green),

which is significantly reduced for the other two bars (gray). For

clusters in the right hemisphere, activity was higher during left-

hemifield congruent stimulation (red bars) compared to right-

hemifield (green bars). The reverse applied for clusters in the left

hemisphere, now with bar 2 (both stimuli on the right, in green)

larger than bar 1 (both stimuli on the left, in red).

The critical multimodal spatial effect is represented by the fact

that these differences (bars 1 and 2, red and green) cannot be

explained by any difference observed during the spatially incon-

gruent stimulation (bars 3 and 4, gray). For example, the signal plot

for right occipital cortex during the saccade task (central plot in the

first row of Fig. 3A) shows as expected that this region responded

more to left than right visual stimuli. While this can be observed to

some extent when touch was in a spatially incongruent confi-

guration (compare bar 4 versus bar 3, for this plot), this difference

was larger when touch was also on the left side (bar 1 minus bar 2;

congruent trials). Thus, the lateralised visual responses in this

region were modulated by the position of the tactile stimulation.

Importantly, this effect was observed irrespective of task, that is

both when vision was relevant (saccade to vision, central plot in

top row of Fig. 3A), but also when vision was irrelevant (saccade

to touch, plot on the left of Fig. 3A top row); and also when

subjects simply maintained central fixation (plot on the right). This

indicates that crossmodal-congruency effects in this region of

occipital visual cortex are not dependent on the behavioral

relevance of the contralateral visual location, nor on that location

being a target for a saccadic eye-movement.

Analogous patterns of activation were found in a medial region

of the parietal operculum (see Fig. 3B). This region showed a

larger effect related to the position of the tactile stimulus (higher

activity for touch at the contralateral versus ipsilateral side) when

the visual stimulus was also at the contralateral side. For the right

hemisphere, the critical crossmodal spatial modulation can be seen

by comparing the difference between bar 1 (in red) minus bar 2 (in

green) to that for bar 3 minus bar 4. Again, the larger difference is

found for the first two trial-types, indicating that effect of

contralateral left somatosensory stimulation was larger when the

concurrent visual stimulus was also on the left (spatially congruent

condition), than when the visual stimulus was on the right. An

analogous pattern of activation was observed in the left parietal

operculum, but now with higher activity for spatially congruent

bimodal stimulation on the right side, that is, activation again

contralateral to the position of the spatially-congruent bimodal

stimulation (green bars for the signal plots in the second row of

Fig. 3B). These somatosensory regions were likewise modulated

according to the relative position of tactile and visual stimuli

irrespective of whether the visual or the tactile position was task

relevant; and even when the stimuli were presented passively

(central fixation condition, rightmost plots).

These results are summarised in Fig. 5, where the sizes of the

modulatory effect of spatial congruence (vision and touch at the

same location versus opposite sides) on contralateral responses are

shown for all four regions (lateral occipital cortex and parietal

operculum, in the two hemispheres). The modulatory effects are

plotted separately for the three tasks (saccade to touch, saccade to

vision and fixation) showing that for these four regions, the effect

of spatial congruence was observed irrespective of task (the twelve

leftmost bar-plots all have positive values). Thus, these data

demonstrate that some visual regions (lateral occipital cortex) and

also some somatosensory regions (parietal operculum) are modu-

lated by the relative position of bimodal visuo-tactile stimuli (larger

responses when both stimuli were at the contralateral location

together, in a spatially-congruent arrangement); and that these

crossmodal spatial effects do not simply reflect the behavioral

relevance of one or the other modality, nor the stimulated location

being the target for a saccade.

Further analyses assessed whether any region showed cross-

modal spatial effects selectively during some tasks, but not during

the others. The only area that showed a consistent pattern of

activation in both hemispheres was the ventral occipital cortex,

which showed crossmodal spatial effects only during central

fixation (see Table 2). Anatomical location and signal plots for

this region are shown in Fig. 4 (see also Fig. 5). The ventral

occipital cortex showed higher activity for visual stimulation of the

contralateral hemifield compared with ipsilateral visual stimula-

tion, and this effect was selectively modulated by the position of

the tactile stimulation only during central fixation. This can be seen

in the rightmost plots of Fig. 4, where the difference between bar

1 and 2 (congruent conditions) cannot be explained by any diffe-

rence between bar 3 and 4 (incongruent conditions). This was not

the case during the saccade to touch tasks (leftmost plots in Fig. 4)

nor during the saccade to vision task (central plots in Fig. 4), when

the difference between left and right visual stimulation was similar

in the crossmodally congruent and incongruent conditions. These

effects are also summarised in Fig. 5 (last six bar-plots), showing

that the spatial congruence of vision and touch resulted in higher

contralateral responses (displayed as positive values in Fig. 5) only

during fixation (white bar-plots). Thus, unlike the lateral occipital

cortex (and parietal operculum), crossmodal effects in ventral

occipital cortex were observed only when no overt spatial orienting

took place, with the eyes maintained at fixation (see Fig. 5).

For completeness, we also tested for brain activation associated

with spatially incongruent minus congruent trials. In particular, it

Fig. 3. Crossmodal spatial effects independent of current task. (A) Modulation of visual responses for bimodal visuo-tactile stimulation with both modalities at

the same location on the contralateral side. In lateral occipital cortex, the effect of side in vision was larger when touch was at the same location as the visual

stimulus (bar 1 and bar 2) than when touch was on the opposite side (bar 3 and bar 4). (B) Analogously, in the parietal operculum (a somatosensory region), the

responses to contralateral tactile stimulation was larger when vision was also at the same-contralateral-location (compare bar 1 versus 2, and bar 3 versus bar 4,

for all signal plots). All these effects were contralateral to the position of the spatially congruent multimodal stimulation and they were all observed

irrespective of current task. The bars in color indicate the critical crossmodal spatial effect for spatially congruent trials. Note that for illustrative purposes here

each plot shows the level of activity for each of the four trial-types, while statistical inference was based on a model focusing on the appropriate differences

between conditions (see Methods). Note also that because all analyses considered bdifferences of differencesQ (i.e., interactions, or modulations), the activity

plotted in this figure and Fig. 4 are mean-adjusted to have a sum of zero, and thus, the absolute level of activity for each condition is arbitrary. The critical

multimodal spatial effect is represented by the fact that the activation related to the stimulus position during congruent stimulation (e.g., vision left minus vision

right, bar 1 minus bar 2) is consistently larger than the same subtraction for spatially incongruent stimulation (e.g., bar 4 minus 3). All coronal sections are

taken through the maxima and the effect sizes are expressed in standard error units (SE). For display purpose, SPM thresholds are set to P-uncorr = 0.01.

(TL: touch left; TR: touch right; VL: vision left; VR: vision right).


Fig. 4. Task-specific crossmodal spatial effects. This figure shows anatomical location and signal plots for the ventral occipital cortex, the only region where

spatial crossmodal effects were observed only during the central fixation task. This can be observed in the rightmost plots of the two rows, where the difference

between left and right visual stimulation was larger when touch and vision were at the same contralateral side (see red bar 1 and green bar 2), compared with

trials when the two modalities were presented in opposite hemifields (bars 3 and 4). This modulatory effect of multimodal spatial congruence was not observed

when subjects used the stimuli to direct saccadic eye-movements (see leftmost and central plots on both rows). All coronal sections are taken through the maxima

and the effect sizes are expressed in standard error units (SE). For display purpose, SPM thresholds are set to P-uncorr = 0.01. (TL: touch left; TR: touch right;

VL: vision left; VR: vision right).


might be suggested that potential conflict-monitoring areas (e.g.,

the anterior cingulate cortex; e.g., Van Veen et al., 2001) or areas

involved in anti-saccades (e.g., supplementary frontal eye-field;

Everling et al., 1998) might activate when subjects performed

Fig. 5. Summary of all crossmodal enhancements for spatially congruent

trials. This figure shows the size of the interaction between stimulus

position (left versus right hemifield) and spatial congruence of the bimodal

stimulation (same versus opposite side) for the areas also displayed in Figs.

3 and 4. Data are divided according to the task (saccade to touch, saccade to

vision or fixation). Positive values indicate that the difference of brain

activity for contralateral minus ipsilateral stimulation was larger when the

two modalities were spatially congruent (both at same location) than when

they were spatially incongruent (opposite sides), see also Figs. 3 and 4.

Lateral occipital cortex and parietal operculum showed crossmodal spatial

enhancements irrespective of task (all bar-plots show positive values), while

ventral occipital cortex showed this modulation only during the fixation

task (see white bar-plots for this region). (Occ.: Occipital cortex; Lat.:

Lateral; Vent.: Ventral; Operc.: Parietal Operculum).

either of our saccade-tasks with spatially incongruent stimuli.

Thus, we examined the pattern of activation for incongruent minus

congruent trials in the two saccade tasks. This did not reveal any

significant activation, when cluster-level correction for multiple

comparison was used. When this constraint was dropped (while

maintaining the voxel-level threshold at P-uncorr. = 0.001), several

voxels in frontal areas showed some activation, but none of these

passed even this less conservative threshold for both tasks. The

lack of activation specific to incongruent trials might be due to

relatively little conflict being produced by them, and/or the fact

that the task (saccade to vision or saccade to touch) was blocked in

different sessions here. Accordingly, the selection of one or the

other modality as the target for motor responses, possibly in frontal

cortex, might have entail sustained activity during the whole fMRI

session, which would not be detected with the current design.

Discussion

The present study investigated crossmodal spatial-congruence

effects (with visual and tactile stimuli at the same location or

opposite sides concurrently). It did so in the presence or absence of

overt spatial saccadic orienting, and assessed the role of modality

task-relevance (for saccadic targeting) on any such crossmodal

effects. The fMRI results showed that activity in visual extrastriate

areas and the parietal operculum was modulated by the spatial

congruence of the bimodal stimulation, with larger responses when

both the visual and tactile stimuli were delivered together at the

same contralateral location (spatially congruent conditions). These

crossmodal spatial effects were found in lateral occipital and

parietal operculum regions irrespective of which modality was


currently task-relevant, and of whether the stimuli were used to

guide eye-movements or were received passively. This suggests

that crossmodal spatial effects in these sensory-specific (i.e., visual

or tactile) areas relate to an automatic mechanism that depends

predominantly on the spatial configuration of the multisensory

stimuli, rather than on endogenous task-relevant factors (such as

which modality had to be saccaded to; or indeed whether any

saccade had to be initiated at all).

The present study used variations on a prototypical paradigm

for studying crossmodal spatial interactions, where bimodal

stimulation (here in vision and touch) is presented either in

spatially congruent configurations (both stimuli at the same

location) or in spatially incongruent configurations (here with

vision and touch delivered in opposite hemifields). In agreement

with our previous fMRI work (e.g., Macaluso et al., 2000b, 2002a),

boosting of sensory responses was observed in occipital cortex

contralateral to the hemifield where spatially congruent bimodal

stimulation was presented (see Fig. 3A). This activation was

located in the lateral occipital cortex here (see Fig. 3A). Previous

imaging studies of visual-tactile interactions in a visual detection

task without saccades (Macaluso et al., 2000b, 2002a) did not

detect crossmodal effects at fully corrected significance in this

specific occipital area, but in fact corresponding trends were

observed just below statistical threshold in those studies for this

region. Moreover, the same region did previously show spatially-

specific crossmodal visual–tactile effects in tasks concerning

endogenous covert spatial attention (Macaluso et al., 2000a,

2002b).

Here, we show for the first time that spatially specific

crossmodal spatial-congruence effects (of multisensory stimulation

at the same rather than opposite locations) can be observed also in

the context of saccade tasks, when stimuli are used to guide

spatially directed overt responses. These findings may accord with

the notion that overt and covert spatial orienting might overlap to

some extent, both at the level of brain activations (e.g., see

Corbetta et al., 1998) and at the level of behavioral effects related

to spatial orienting to multimodal stimuli (c.f. Diederich et al.,

2003; Spence et al., 1998). The present paradigm further allowed

us to investigate crossmodal spatial interactions when either touch

was task-relevant (i.e., tactile position used to direct saccades,

while ignoring any visual stimulus), or vision was relevant (i.e.,

saccade to vision, ignoring any tactile stimulus). The results

showed that spatially-specific crossmodal modulation of responses

in visual cortex did not depend on vision being behaviorally

relevant, thus highlighting the automatic, stimulus-driven nature of

these crossmodal effects. Such findings are in agreement with

suggestions that crossmodal influences upon unimodal visual

cortex elicited with this prototypical paradigm might be related

to mechanisms of integration of spatial representations (see

Macaluso and Driver, 2001; and also Technical Comments by

McDonald et al., 2001; and Technical Response by Macaluso et al.,

2001; on this issue). Macaluso et al. (2002b) also found that lateral

visual cortex can be influenced by the currently attended location

in a tactile task (when vision was task-irrelevant), while several

ERP studies have demonstrated modulation of early, sensory-

specific visual components according to the direction of tactile

spatial attention (Eimer and Driver, 2000; Kennett et al., 2001).

Thus, it appears that, regardless of the current task (e.g.,

endogenous or exogenous covert spatial attention, or with vision

task-relevant or not in overt saccadic spatial orienting as here), the

level of activity in occipital visual cortex can reflect not only task-

relevant visual processing but also spatial aspects involving other

modalities, here touch.

Unlike previous studies that used similar paradigms (Macaluso

et al., 2000b, 2002a), the present study was able to demonstrate

crossmodal spatial effects also in somatosensory cortex for the first

time. Analogously to the effects in the lateral occipital cortex,

crossmodal amplification was again observed in the hemisphere

contralateral to the hemifield where spatially congruent visual and

tactile stimuli were presented together, in the parietal operculum.

This may correspond to secondary somatosensory cortex (Burton

et al., 1993). Note that while some studies have demonstrated

somatotopic organisation of the secondary somatosensory cortex

within the parietal operculum (e.g., Ruben et al., 2001), here we

did not compare stimulation of two different body-parts (e.g.,

hallux verus index-finger) in single-subject analyses, and thus our

vibro-tactile stimulation produced activation of the contralateral

operculum extending from the lip of the lateral sulcus to the

insulae. A possible role of secondary somatosensory cortex in

spatial processing of multi-sensory stimuli is further supported by

some electro-physiological reports of crossmodal effects in single

neurons of awake monkeys (Burton et al., 1997).

One possible reason why previous imaging studies that used

similar paradigms (e.g., Macaluso et al., 2000b, 2002a) did not find

crossmodal spatial effects in these somatosensory regions might be

that here, for the first time, tactile stimulations were delivered

directly in front of the subject, albeit unseen, without any reflection

of seen hands due to the use of mirrors (cf. Macaluso et al., 2000b,

see also Misaki et al., 2002). Thus, the felt position of the hands

corresponded directly with the location (in external space) where

visual stimuli were also presented, resulting in a correct alignment

of visual, proprioceptive and tactile signals. As with the cross-

modal modulation in lateral occipital cortex, the effects in the

parietal operculum were obtained here regardless of the currently

relevant modality for the saccade task, and indeed regardless of

whether the stimulated locations were saccade targets or not.

Accordingly, both visual and somatosensory cortices appear to be

influenced by spatially congruence of a bimodal visual–tactile

event in the contralateral hemifield, regardless of the task. These

patterns of activation are consistent with the proposal that

integration of spatial representations between senses involves

coordination of activity in anatomically distant areas, responsible

for the processing of stimuli in different modalities but originating

from the same position in external space (Macaluso and Driver,

2001).

The only area that showed crossmodal spatial-congruence

effects specific to one of the three tasks was ventral occipital

cortex. In this region crossmodal effects, characterised by larger

brain activity for spatially congruent bimodal stimulation in the

contralateral hemifield, were observed exclusively when subjects

maintained central fixation and did not use the stimuli to guide any

eye-movements (see Fig. 4). This confirms that our analysis

approach is appropriate for revealing any crossmodal effects that

are specific for only one task (hence the task-independent effects

reported above are not merely due to some bias in our statistical

approach). The activation of this ventral area specifically in task

conditions requiring central fixation accords with some of our

previous results (Macaluso et al., 2000b, 2002a) that showed

crossmodal spatial visual–tactile effects in ventral occipital cortex

during covert orienting (i.e., again with central fixation and no

saccades). Here, we observe that such effects in this ventral region

were abolished when the sensory signals (vision or touch) were


used to guide overt responses. This might reflect the well-known

specialisation of ventral regions for stimulus identification and

analysis, with a lesser involvement when the sensory input is used

to guide direct overt spatial responses (Goodale and Milner, 1992).

All effects of spatial congruency between vision and touch were

observed in regions of the brain primarily concerned with sensory

processing (i.e., visual areas in occipital cortex, and somatosensory

regions in the parietal operculum), that here also showed a main

effect of stimulated hemifield for one or the other modality (see

Fig. 2). A different outcome might be expected in designs

manipulating the spatial relation between the position of the target

stimuli and the direction of motor responses (e.g., employing anti-

versus pro-saccade tasks), rather than the spatial congruence of

stimuli in different sensory modalities, as done here. For example,

frontal areas involved in saccadic control (e.g., supplementary

frontal eye-fields) might show interesting effects related to

voluntary overt orienting versus more reflexive behaviour using

stimuli in a different modality than vision (e.g., Curtis and

D’Esposito, 2003).

In conclusion, the present study demonstrated that particular

visual (lateral occipital) and somatosensory (parietal operculum)

areas can both be affected by the spatial congruence of multi-

sensory visual–tactile stimulation. These effects were spatially-

specific, with boosting of sensory responses observed in brain

regions contralateral to the position of spatially congruent bimodal

stimulation. The effects in visual cortex accord with some of our

prior work (e.g., Macaluso et al., 2000b; 2002a), while we show

such an influence on somatosensory cortex (parietal operculum)

also, for the first time. Critically, here we also show for the first

time that these effects do not depend on the behavioral relevance of

one or the other sensory modality, nor on their serving as saccade

targets. Instead, they appear to reflect an automatic, stimulus-

driven mechanism. These observations support a recent proposal

(see Driver and Spence, 1998; Macaluso and Driver, 2001) that

integration of spatial representations between sensory modalities

does not rely solely on sensory convergence to multimodal areas,

but can also involve crossmodal influences upon sensory-specific

cortices, providing a distributed – but integrated – system for

representing space across sensory modalities.

References

Amlot, R., Walker, R., Driver, J., Spence, C., 2003. Multimodal visual-

somatosensory integration in saccade generation. Neuropsychology 41,

1–15.

Amador, N., Schlag-Rey, M., Schlag, J., 2004. Primate antisaccade. II.

Supplementary eye field neuronal activity predicts correct performance.

J. Neurophysiol. 91, 1672–1689.

Bremmer, F., Schlack, A., Shah, N.J., Zafiris, O., Kubischik, M.,

Hoffmann, K., Zilles, K., Fink, G.R., 2001. Polymodal motion

processing in posterior parietal and premotor cortex: a human fMRI

study strongly implies equivalencies between humans and monkeys.

Neuron 29, 287–296.

Bruce, C., Desimone, R., Gross, C.G., 1981. Visual properties of

neurons in a polysensory area in superior temporal sulcus of the

macaque. J. Neurophys. 46, 369–384.

Burton, H., Videen, T.O., Raichle, M.E., 1993. Tactile-vibration-activated

foci in insular and parietal–opercular cortex studied with positron

emission tomography: mapping the second somatosensory area in

humans. Somatosens. Motor Res. 10, 297–308.

Burton, H., Sinclair, R.J., Hong, S.Y., Pruett, J.R.J., Whang, K.C., 1997.

Tactile–spatial and cross-modal attention effects in the second

somatosensory and 7b cortical areas of rhesus monkeys. Somatosens.

Motor Res. 14, 237–267.

Calvert, G.A., Campbell, R., Brammer, M.J., 2000. Evidence from

functional magnetic resonance imaging of crossmodal binding in the

human heteromodal cortex. Curr. Biol. 10, 649–657.

Collins, D.L., Neelin, P., Peters, T.M., Evans, A.C., 1994. Automatic 3D

intersubject registration of MR volumetric data in standardized

Talairach space. J. Comput. Assist. Tomogr. 18, 192–205.

Corbetta, M., Akbudak, E., Conturo, T.E., Snyder, A.Z., Ollinger, J.M.,

Drury, H.A., Linenweber, M.R., Petersen, S.E., Raichle, M.E., Van

Essen, D.C., Shulman, G.L., 1998. A common network of functional

areas for attention and eye movements. Neuron 21, 761–773.

Curtis, C.E., D’Esposito, M., 2003. Success and failure suppressing

reflexive behavior. J. Cogn. Neurosci. 15, 409–418.

Diederich, A., Colonius, H., Bockhorst, D., Tabeling, S., 2003. Visual–

tactile spatial interaction in saccade generation. Exp. Brain Res. 148,

328–337.

Driver, J., Spence, C., 1998. Attention and the crossmodal construction of

space. Trends Cogn. Sci. 2, 254–262.

Duhamel, J.R., Colby, C.L., Goldberg, M.E., 1998. Ventral intraparietal

area of the macaque: congruent visual and somatic response properties.

J. Neurophys. 79, 126–136.

Eimer, M., Driver, J., 2000. An event-related brain potential study of cross-

modal links in spatial attention between vision and touch. Psychophysi-

ology 37, 697–705.

Everling, S., Spantekow, A., Krappmann, P., Flohr, H., 1998. Event-related

potentials associated with correct and incorrect responses in a cued

antisaccade task. Exp. Brain Res. 118, 27–34.

Friston, K.J., Holmes, A.P., Price, C.J., Buchel, C., Worsley, K.J., 1999.

Multisubject fMRI studies and conjunction analyses. NeuroImage 10,

385–396.

Friston, K.J., Glaser, D.E., Henson, R.N., Kiebel, S., Phillips, C.,

Ashburner, J., 2002. Classical and Bayesian inference in neuroimaging:

applications. NeuroImage 16, 484–512.

Goodale, M.A., Milner, A.D., 1992. Separate visual pathways for

perception and action. Trends Neurosci. 15, 20–25.

Graziano, M.S., Gross, C.G., 1995. The representation of extrapersonal

space: a possible role for bimodal, visuo-tactile neurons. In: Gazzaniga,

M.S. (Ed.), The Cognitive Neurosciences. MIT Press, Cambridge, USA,

pp. 1021–1034.

Henson, R.N.A., Buechel, C., Josephs, O., Friston, K., 1999. The slice-

timing problem in event-related fMRI. NeuroImage 9, 125.

Holmes, A.P., Friston, K.J., 1998. Generalisability, random effects and

population inference. NeuroImage, S754.

Kennett, S., Eimer, M., Spence, C., Driver, J., 2001. Tactile–visual links in

exogenous spatial attention under different postures: convergent evi-

dence from psychophysics and ERPs. J. Cogn. Neurosci. 13, 462–478.

Macaluso, E., Driver, J., 2001. Spatial attention and crossmodal interactions

between vision and touch. Neuropsychology 39, 1304–1316.

Macaluso, E., Frith, C.D., Driver, J., 2001. Multisensory integration and

crossmodal attention effects in the human brain. Science [Technical

response] 292, 1791.

Macaluso, E., Frith, C., Driver, J., 2000a. Selective spatial attention in

vision and touch: unimodal and multimodal mechanisms revealed by

PET. J. Neurophysiol. 83, 3062–3075.

Macaluso, E., Frith, C.D., Driver, J., 2000b. Modulation of human visual

cortex by crossmodal spatial attention. Science 289, 1206–1208.

Macaluso, E., Frith, C.D., Driver, J., 2002a. Crossmodal spatial influences

of touch on extrastriate visual areas take current gaze direction into

account. Neuron 34, 647–658.

Macaluso, E., Frith, C.D., Driver, J., 2002b. Directing attention to locations

and to sensory modalities: multiple levels of selective processing

revealed with PET. Cereb. Cortex 12, 357–368.

Macaluso, E., Driver, J., Frith, C.D., 2003. Multimodal spatial representa-

tions in human parietal cortex engaged during both saccadic and manual

spatial orienting. Curr. Biol. 13, 990–999.


McDonald, J.J., Teder-Salejarvi, W.A., Hillyard, S.A., 2000. Involuntary

orienting to sound improves visual perception. Nature 407, 906–908.

McDonald, J.J., Teder-Salejarvi, W.A., Ward, L., 2001. Multisensory

integration and crossmodal attention effects in the human brain. Science

[Technical Comment] 292, 1791.

Meredith, M.A., Stein, B.E., 1986a. Spatial factors determine the activity

of multisensory neurons in cat superior colliculus. Brain Res. 365,

350–354.

Meredith, M.A., Stein, B.E., 1986b. Visual, auditory, and somatosensory

convergence on cells in superior colliculus results in multisensory

integration. J. Neurophysiol. 56, 640–662.

Meredith, M.A., Stein, B.E., 1996. Spatial determinants of multisensory

integration in cat superior colliculus neurons. J. Neurophysiol. 75,

1843–1857.

Meredith, M.A., Nemitz, J.W., Stein, B.E., 1987. Determinants of multi-

sensory integration in superior colliculus neurons. I. Temporal factors.

J. Neurosci. 7, 3215–3229.

Misaki, M., Matsumoto, E., Miyauchi, S., 2002. Dorsal visual cortex

activity elicited by posture change in a visuo-tactile matching task.

NeuroReport 13, 1797–1800.

Price, C.J., Friston, K.J., 1997. Cognitive conjunction: a new approach to

brain activation experiments. NeuroImage 5, 261–270.

Rorden, C., Greene, K., Sasine, G., Baylis, G., 2002. Enhanced tactile

performance at the destination of an upcoming saccade. Curr. Biol. 12,

1429–1434.

Ruben, J., Schwiemann, J., Deuchert, M., Meyer, R., Krause, T., Curio, G.,

Villringer, K., Kurth, R., Villringer, A., 2001. Somatotopic organisa-

tion of human secondary somatosensory cortex. Cereb. Cortex 11,

463–473.

Spence, C., Driver, J., 1997. Audiovisual links in exogenous covert spatial

orienting. Percept. Psychophys. 59, 1–22.

Spence, C., Driver, J., 2004. Crossmodal Space and Crossmodal Attention.

OUP, Oxford, UK.

Spence, C., Nicholls, M.E., Gillespie, N., Driver, J., 1998. Cross-modal

links in exogenous covert spatial orienting between touch, audition, and

vision. Percept. Psychophys. 60, 544–557.

Stein, B.E., Meredith, M.A., 1993. The Merging of the Senses. MIT Press,

Cambridge, USA.

Van Veen, V., Cohen, J.D., Botvinick, M.M., Stenger, V.A., Carter, C.S.,

2001. Anterior cingulate cortex, conflict monitoring, and levels of

processing. NeuroImage 14, 1302–1308.

Worsley, K.J., Marrett, S., Neelin, P., Vandal, A.C., Friston, K.J., Evans,

A.C., 1996. A unified statistical approach for determining significant

signals in images of cerebral activation. Hum. Brain Mapp. 4, 58–73.

Multisensory stimulation with or without saccades: fMRI evidence for crossmodal effects on...

Documents

Transcript of Multisensory stimulation with or without saccades: fMRI evidence for crossmodal effects on...