PLEASE SCROLL DOWN FOR ARTICLE

This article was downloaded by: [Bach, Patric]On: 8 May 2009Access details: Access Details: [subscription number 911016206]Publisher Psychology PressInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Social NeurosciencePublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t741771143

N400-like negativities in action perception reflect the activation of twocomponents of an action representationPatric Bach ab; Thomas C. Gunter a; Günther Knoblich ac; Wolfgang Prinz a; Angela D. Friederici a

a Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany b University of Wales,Bangor, UK c Rutgers University, Newark, New Jersey, USA

First Published on: 20 November 2008

To cite this Article Bach, Patric, Gunter, Thomas C., Knoblich, Günther, Prinz, Wolfgang and Friederici, Angela D.(2008)'N400-likenegativities in action perception reflect the activation of two components of an action representation',Social Neuroscience,4:3,212 —232To link to this Article: DOI: 10.1080/17470910802362546URL: http://dx.doi.org/10.1080/17470910802362546

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.

N400-like negativities in action perception reflect theactivation of two components of an action

representation

Patric Bach

Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany, and University ofWales, Bangor, UK

Thomas C. Gunter

Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany

Gunther Knoblich

Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany and RutgersUniversity, Newark, New Jersey, USA

Wolfgang Prinz and Angela D. Friederici

Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany

The understanding of actions of tool use depends on the motor act that is performed and on the functionof the objects involved in the action. We used event-related potentials (ERPs) to investigate theprocesses that derive both kinds of information in a task in which inserting actions had to be judged. Theactions were presented as two consecutive frames, one showing an effector/instrument and the othershowing a potential target object of the action. Two mismatches were possible. An orientation mismatchoccurred when the spatial object properties were not consistent with a motor act of insertion beingperformed (i.e., different orientations of insert and slot). A functional mismatch happened when theinstrument (e.g., screwdriver) would usually not be applied to the target object (e.g., keyhole). The orderin which instrument and target object were presented was also varied. The two kinds of mismatch gaverise to similar but not identical negativities in the latency range of the N400 followed by a positivemodulation. The results indicate that the motor act and the function of the objects are derived by two atleast partially different subprocesses and become integrated into a common representation of theobserved action.

Keywords: Event-related potentials; Action observation; Tool use; Mirror neurons; Language comprehension.

INTRODUCTION

Current accounts of action understanding rely onthe idea that humans map observed actions on the

representations they would use to perform theactions themselves. This process allows them tointernalize the actions of others so that theseactions can activate the same internal states that

# 2008 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business

Correspondence should be addressed to: Patric Bach, Centre for Cognitive Neuroscience, School of Psychology, University ofWales, Bangor, Gwynedd LL57 2DG, UK. E-mail: p.bach@bangor.ac.uk

We thank Ina Koch, Conny Schmid and Sandra Boehme for the data collection, and Steven P. Tipper and Stefanie Schuch forconstructive discussions concerning the research. The work was partly funded by a Wellcome Trust Programme Grant.

SOCIAL NEUROSCIENCE, 2009, 4 (3), 212!232

www.psypress.com/socialneuroscience DOI:10.1080/17470910802362546

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

would precede or follow them when actuallyperformed (cf. Bach & Tipper, 2007; Barsalou,Niedenthal, Barbey, & Ruppert, 2003; Wilson &Knoblich, 2005).

Behavioral evidence for such a link betweenaction and action perception comes from find-ings that humans have a tendency to imitate theactions of others, even when the actions are task-irrelevant (e.g., Bach, Peatfield, & Tipper, 2007;Brass, Bekkering, Wohlschlager, & Prinz, 2000;Hommel, Musseler, Aschersleben, & Prinz, 2001).On the neurophysiological level, neurons with so-called mirror properties have been discovered inthe premotor and parietal cortices of the macaquemonkey. Mirror neurons fire not only when themonkey performs a particular action, but alsowhen it observes the action being performed byanother creature (DiPellegrino, Fadiga, Fogassi,Gallese, & Rizzolatti, 1992; Gallese, Fadiga,Fogassi, & Rizzolatti, 1996; Rizzolatti, Fadiga,Gallese, & Fogassi, 1996; for similar data onhumans, see Grezes, Armony, Rowe, & Passing-ham, 2003; Manthey, Schubotz, & von Cramon,2003; Buccino et al., 2001; Constantini et al.,2005).

The mirror neurons seem to represent actionsin terms of viewpoint-independent features thatare common to observed as well as producedactions, such as sounds (Kohler et al., 2002), actiongoals (Bekkering, Wohlschlager, & Gattis, 2002;Johnson-Frey et al., 2003), or the spatial relation-ship that is required for the actions to succeed (cf.Oztop and Arbib, 2002; see also Gallese, 2003).For example, for grasping, the hand’s fingerconfiguration, its orientation and grip apertureshould match the corresponding features of thetarget object. These viewpoint-independent rela-tional action features can be used to guide actionproduction, but also allow an observed action tobe identified (for a computational demonstration,see Oztop & Arbib, 2002).

These common representations for perceptionand action can explain how observers differentiatebetween the different motor acts they observe.However, mirror neurons are blind to the identityand function of the objects involved in the action(Jacob & Jeannerod, 2005; Nelissen, Luppino,Vanduffel, Rizzolatti, & Orban, 2005), and fireeven when actions on ‘‘meaningless’’ 3D-solidsare observed. They therefore cannot account forhow object use shapes action understanding. Thislimitation is crucial, because the outcomes ofmany actions depend heavily on the objectsused, even when the motor acts involved are the

same. Compare, for instance, inserting a creditcard into a cash machine with inserting a ticketinto a ticket canceller.

Evidence that humans have a dedicated systemthat links objects to possible functions comesmostly from lesion studies. Patients with damageto the temporal lobe and prefrontal cortex mayperform the appropriate motor acts of everydaytasks, but use inappropriate instruments (e.g.,brushing teeth with a comb; Ochipa, Rothi, &Heilman, 1989). They also exhibit impairmentsin selecting a typical target object for a given tool,or selecting tools with a similar function (Hodges,Spatt, & Patterson, 1999). The reverse patternhas also been observed. There are patients thatcan describe the function of a tool but cannotpantomime its usage (Buxbaum & Saffran, 1998).As of yet, imaging studies have failed to reveala similar dissociation (e.g., Kellenbach, Brett, &Patterson, 2003; Boronat et al., 2005).

Recent studies have confirmed the use offunctional information during action observation.When judging the appropriateness of motor acts,observers cannot ignore the function of theobjects involved, even when the identity of theobjects has no bearing on how the motor act isperformed (Bach, Knoblich, Gunter, Friederici, &Prinz, 2005). Similarly, patients suffering fromvisual extinction can see an action instead of twoobjects in isolation, only if the objects are atthe same time spatially and functionally appro-priate for an action to be carried out with them(Riddoch, Humphreys, Edwards, Baker, &Wilson, 2003; for a similar result in healthyparticipants, see Green & Hummel, 2006).

Finally, the use of functional information inaction perception has been demonstrated bystudies using event-related potentials (ERPs).N400-like negativities are elicited when an object(or a gesture) with an inappropriate function ischosen for an everyday task (e.g., for shaving, arolling pin was used instead of a razor, Sitnikova,Kuperberg, & Holcomb, 2003; see also Gunter &Bach, 2004). The N400-component is widelyaccepted as a measure of the difficulty of integrat-ing a new stimulus into the surrounding semanticcontext (for a review, see Kutas & Federmeier,2000), which suggests that knowledge about objectfunction also provides a context during actionobservation.

The present study investigated the ERP-corre-lates of the processes that integrate the motor actand object function into a coherent representa-tion of an observed action. We presented the

ACTION COMPREHENSION 213

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

participants with static images frames of insertingactions and asked them whether the observedaction was appropriate or not. The stimuli showedactions in mid-flight before they were fullyexecuted. Such stimuli should be particularlyeffective in activating the mirror neurons becausethese neurons stop firing when an action’s goal isreached (Rizzolatti, Fogassi, & Gallese, 2001).Static frames were used because, other than formovies, there is a clear onset for the stimuli. Yetthey are processed just like full action displays(Sturmer, Aschersleben, & Prinz, 2000; Kourtzi &Kanwisher, 2000) and activate the mirror system(Johnson-Frey et al., 2003). A rating task in aprior study showed that our stimuli indeedstrongly evoke the goal of the presented actions(Bach et al., 2005).

For each of the actions, the appropriateness ofthe motor act and the appropriateness of theobject function were varied independently (seeFigure 1 for examples). The appropriatenessof the motor act was varied by manipulating thespatial relationship between the insert and theslot of the target object. For a motor act ofinsertion to succeed, both the insert and the slotmust have identical orientations. With orthogonal

orientations, the motor act is not successful(orientation mismatch). The appropriateness ofobject function was varied by manipulating thefunctional relationship between instruments andtarget objects. Some combinations of inserts andtarget objects were functionally related becausethe objects are usually used together to realize aprototypical action goal (e.g., key, keyhole), whileothers were not functionally related (functionalmismatch; e.g., key, screw). Therefore, all in all,the actions could either be fully matching or theycould contain one of three mismatches: functionalmismatch, orientation mismatch, or double mis-match (orientation and functional mismatch). Theparticipants pressed one key if the actions werefully matching and another key if the actionscontained any mismatch.

This experimental design allowed us to dis-sociate the processing of the function of theobjects from the processing of the motor act.Behavioral experiments using the same task andthe same stimuli (Bach et al., 2005; Bach,Knoblich, Gunter, Friederici & Prinz, 2008)have shown that the difficulties of detectingorientation and functional mismatches are highlycomparable (as measured by response times and

Figure 1. Examples for the stimuli used in the experiments: The two panels on the left (a, c) show actions in which the motor actwas performed correctly, whereas the two panels on the right (b, d) contain orientation mismatches. The upper two panels (a, b)show functionally matching actions whereas the two lower panels (c, d) show functional mismatches.

214 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

error rates). Therefore, any differences betweenthe ERPs elicited by the two mismatches arelikely due to differences in the underlying pro-cesses, rather than to the general processing diffi-culty of the stimuli or the task.

On the basis of the prior research on actionobservation, we predicted that, relative to fullmatches, functional mismatches would evoke anN400-like negativity, and possibly a later positivemodulation (e.g., Sitnikova et al., 2003; Gunter &Bach, 2004). To our knowledge, no prior studieshave investigated orientation mismatches in ac-tion perception tasks. The main aim of the presentstudy was to characterize the ERPs evoked bythese mismatches, and to establish whether theyreflect processes that are independent of thosedetecting functional mismatches.

Some predictions can be derived from theidea that the detection of orientation mismatchesrelies on the same parietal and motor structuresthat would also be recruited for the actual (orimagined) rotation of the observed objects. Tasksthat require orientation judgments of externalobjects usually rely strongly on right hemisphereregions (e.g., Inoue, Yoshino, Suzuki, Ogasa-wara, & Nomura, 1998; Keehner, Guerin, Turk,Miller, & Hegarty, in press; Pegna et al., 1997;Turnbull, Beschin, & Della Sala, 1997). When,however, the same tasks are performed onobserved body parts or tools, parietal and motorregions of the left hemisphere are involved to agreater extent (Amorim, Isableu, & Jarraya,2006; Gerardine et al., 2000; Goldenberg, 2001;Petit, Pegna, Harris, & Michel, 2006; Overney,Michel, Harris, & Pegna, 2005). This doubledissociation has been confirmed in lesion andimaging studies (Kosslyn, DiGirolamo, Thomp-son, & Alpert, 1998; Tomasino, Toraldo, &Rumiati, 2003).

These findings suggest that the left and rightlateralized brain areas should also be involvedwhen, in the present experiments, the motor acthas to be derived from the orientations of theeffectors/instruments and the target objects. Totest this hypothesis, effector/instrument and targetobject were presented sequentially and the orderof presentation was varied across experiments(Experiment 1: instrument before target object;Experiment 2: target object before instrument).As ERPs were time-locked to the presentation ofthe second stimulus, they reflected the processingof target objects in the first experiment, and theprocessing of effectors/instruments in the secondexperiment. If deriving the motor act relies on

the same neuronal structures that are also usedfor (mental) object rotation, the analysis of theorientation of target objects should evoke a rightlateralized response, whereas the spatial analysisof effectors/instruments should evoke a strongerleft hemisphere response.

EXPERIMENT 1: EFFECTORS/INSTRUMENTS PRECEDE TARGET

OBJECTS

In the first experiment, the effector/instrumentwas presented before the target object. ERPswere time-locked to the presentation of thesecond stimulus. Therefore, they reflect the pro-cessing of the target object relative to the instru-ment/effector presented in the first frame. Weexpected that functional mismatches would elicitan N400-like negativity relative to full matches.For orientation mismatches relative to fullmatches, a right lateralized effect was expected,because the ERPs measured the processing ofexternal objects. If the motor act and the functionof objects are processed separately, the ERPselicited by functional and orientation mismatchesshould differ in their scalp distributions, timecourse, or polarity.

Method

Participants

Thirty-two students (16 female) ranging in agefrom 22 to 39 years participated in the studyand received approximately t25 for their partici-pation. One participant was excluded due toexcessive drifts and oculomotor artifacts. Allparticipants were right-handed and had normalor corrected-to-normal vision. The key assign-ment was counterbalanced across participants.

Material

Thirty-two photographs made up the stimulusset. They subtended visual 38 vertically and 48horizontally, given the viewing distance of 100 cm.Sixteen pictures showed a hand holding one ofthe eight different objects to be inserted (screw-driver, coin, key, trolley-pin, safety belt, knife,ticket, credit card) in two different orientationseach (vertically or horizontally). The other half ofthe pictures showed the corresponding eighttarget objects with the respective openings (screw,

ACTION COMPREHENSION 215

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

slot for the coin, lock, trolley, credit card reader,ticket canceller, knife holder, slot for safety belt),again in both orientations.

Out of these photographs, two-frame se-quences were assembled. In total there were 64different two-frame sequences. For each stimulusdepicting a target object, there were four combi-nations with different objects to be inserted (seeFigure 1 for examples). One combination createda match on both dimensions, so that the insert andthe target opening matched functionally and withregard to their orientation. One combinationcreated an orientation mismatch only, while thefunctional match was preserved. One combina-tion created a functional mismatch only, while thematch of the orientations remained intact. Thelast combination created a mismatch on bothdimensions (double mismatch). The resultingstimulus matrix was completely balanced (seeTable 1 for an example). Any target object orinsert that created a mismatch in combinationwith a given stimulus was completely appropriatewhen combined with another one. Therefore, inboth the first and the second frame, exactly thesame stimuli were presented in each condition.Visual between-stimulus differences cannot affectthe results.

Procedure and design

Participants were seated in a dimly lit roomfacing a color monitor at a distance of 100 cm.They received a short computer-driven instruc-tion and some examples were given. After a shorttraining phase of 24 trials the experiment properstarted. It lasted for about 20 min and con-sisted of two blocks of 96 trials each. The blockswere separated by a short pause. The order ofsequences in each block was randomized. Correct

sequences were presented three times as often aseach of the mismatches, to avoid response bias.Therefore, there were 96 correct sequences and96 sequences containing a mismatch. Of these 96mismatching sequences, 32 contained a functionalmismatch, 32 contained an orientation mismatch,and 32 contained both mismatches.

The course of each trial was as follows: Thefirst frame was displayed for 1000 ms, showingone of the eight instruments in one of the twoorientations. After a blank of 300 ms, the secondframe was presented for another 1000 ms, show-ing one of the eight target objects, again in one ofthe two orientations. Participants were instructednot to react as long as this frame was presented.Afterwards the question ‘‘Correct? Incorrect?’’was displayed and participants gave their judg-ment of the sequence at leisure by pressing a key.If their judgment was correct, the next trialstarted after an intertrial interval of 200 ms. Ifthey made an error or did not react within thegiven time interval of 1500 ms, an error-messagewas displayed.

Recordings and data analyses

The EEG was recorded using 56 Ag!AgClelectrodes embedded in an elastic cap (electro-cap) and was referred to the left mastoid. Offline,the ERPs were re-referenced to the mean of theleft and right mastoids. Bipolar horizontal andvertical EOG was recorded. Electrode resistancewas kept under 5 kV. The signals were recordedcontinuously with a band pass between DC and70 Hz and digitized at 250 Hz. Average ERPs,starting 200 ms before and lasting 1000 ms afterthe start of each trial, were computed for eachelectrode position for each of the four conditions.Trials were only included in the average when the

TABLE 1Example stimulus matrix for the different combinations of a ticket, a ticket canceller, a credit card, and a credit card reader.

Target object Screwdriver Screwdriver Key KeyInsert Orientation Horizontal Vertical Horizontal Vertical

Screwdriver Horizontal Full match Orientation-onlymismatch

Functional mismatch Double mismatch

Screwdriver Vertical Orientation-onlymismatch

Full match Double mismatch Functional mismatch

Key Horizontal Functional mismatch Double mismatch Full match Orientation-onlymismatch

Key Vertical Double mismatch Functional mismatch Orientation-onlymismatch

Full match

The lines show the different instruments and the columns show the target objects with which they are combined. Each stimulusoccurs once in all four conditions.

216 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

action was correctly judged. Approximately 8%of the trials were excluded from the averages dueto ocular artifacts (EOG rejection940 mV). Theefficacy of the eye activity correction was checkedby visual inspection, and trials still containingartifacts were rejected. Averages were aligned toa 200 ms pre-stimulus baseline.

To compare the effects elicited by the differ-ent conditions, latency windows were assigned tothe evoked potentials on the basis of visualinspection of the grand averages, according tothe following procedure. First, for each of themismatching conditions latency windows wereassigned separately. The start of a latency win-dow was assigned when one of the electrodesFC5, FC6, CP5 and CP6 (nomenclature followsAmerican Electroencephalographic Society,1991) showed a difference of more than 1 mVbetween the fully matching and the respectivemismatching condition (and continued to in-crease). The end of a latency window wasassigned if at the four electrodes the differencewas lower than 1 mV and continued to decrease.Statistical analyses were then carried out on thelatency window with the maximum overlapbetween the three latency windows of the mis-matching conditions. If there was no overlap, orthe overlap did not include the interval in whichthe ERP effects of all three mismatching condi-tions reached their maximum, separate butidentical statistical analyses were performed oneach of the latency windows assigned to thethree mismatching conditions.

The main analysis included the within-subjectfactors Function (functional mismatch present/notpresent) and Orientation (orientation mismatchpresent/not present). For the assessment of effectsof scalp distribution, we differentiated between sixanterior and six posterior regions of interest, eachreflecting the collapsed data of two electrodes(ROIs, see Figure 2).1 Three additional factorswere entered into the analysis to account fordifferences between these ROIs. The first two ofthese factorswereAntPos (anterior electrode sites/posterior electrode sites, indicatedby the first three

letters of the ROIs) and Hemisphere (left/righthemisphere, indicated by the fourth letter of theROIs). The third factor was Eccentricity (outer,middle, inner electrodes, indicated by the final digitof the ROIs) to assess differences between broadand more central scalp distributions.

For significant effects of scalp distribution, wereport the results of tests run on the data normal-ized according to McCarthy and Wood (1985).The Geisser-Greenhouse correction (Geisser &Greenhouse, 1959; Vasey & Thayer, 1987) wasapplied when evaluating effects with more thanone degree of freedom. Unless otherwise stated,the significances reported for post-hoc analysesremained significant when corrected for multiplecomparisons according to the Bonferroni-Holmprocedure (Holm, 1979).

Results

Participants made almost no (0.2%) errors jud-ging the fully matching condition. Orientationmismatches were incorrectly judged in 3% of thetrials, functional mismatches were incorrectlyjudged in 0.6% of the trials and double mis-matches were incorrectly judged in 2% of thetrials. The differences between the conditionswere not significant.

All three mismatches gave rise to a negativityfollowed by a positive modulation (see Figure 3for the evoked potentials). To investigate theERPs statistically, three latency windows wereassigned to the evoked potentials on the basis ofthe visual inspection procedure described above.2

This resulted in one latency window for thenegative inflection (360!480 ms), capturing thenegative effect evoked by orientation mismatchesand also the interval in which the negativitiesfollowing functional and double mismatchesreached their maximum. Two latency windows

1 The ROIs were similar to our previous study on N400-effects in action/gesture perception (Gunter & Bach, 2004).Because our hypothesis primarily concerned lateralitydifferences, there was no ROI for midline electrodes, butfour ROIs capturing the most lateral electrodes were added(F7/F8, FT7/FT8, TP7/TP8, P7/P8). Note that the 10 chosenROIs also include the electrode sites at which there weresignificant differences in the N400 time window in the otherprior ERP-study on action observation (Sitnikova et al., 2003).

2 To further validate the selection of latency windows, wesplit the analysis window into 40 ms intervals. For each of theseintervals and for each of the ROIs, we investigated whetherfull matches and either of the single mismatches differedsignificantly (after correcting for multiple comparisons withrespect to number of ROIs and latency windows). This showedthat significant deviations of either single mismatchingcondition from the full matches indeed started to appear inthe interval of 360 to 400 ms and remained significantthroughout the selected latency window. One exception wasthe interval between 480 and 520 ms, which corresponded tothe end of our first latency window (capturing the negativeinflections) and the start of latency window 2 (start of thepositive modulation).

ACTION COMPREHENSION 217

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

were chosen to describe the positive modulations,one for the positivity evoked by orientationmismatches (500!700 ms), and one for the func-tional mismatches, which occurred around 100 mslater (600!800 ms). Identical repeated measure-ments ANOVAs were run for each of theseintervals, as described above. Table 2 shows theresults of this analysis. See Figure 4 for the scalpdistributions of the differences between the fullymatching and each mismatching condition in thethree latency windows.

Latency window 1 (360!480 ms)

The ANOVA revealed main effects of Orien-tation and Function, confirming that both types of

mismatch elicit negative inflections. As indicatedby the interactions of Orientation and Hemi-sphere and of Orientation and Eccentricity, thenegativity elicited by orientation mismatches wasright lateralized and was more pronounced overcentral electrodes.

The negativity elicited by functional mis-matches had different scalp distributions onanterior and posterior ROIs (interactions ofFunction, Hemisphere and AntPos, and of Func-tion, Eccentricity and AntPos). On posteriorROIs, a right lateralization, F(1, 30)"5.8,MSE"0.4, pB.05, with a central maximumwas evident, F(2, 60)"7.6, MSE"0.1, pB.005,o".672, but neither effect was present on ante-rior ROIs (both, FB1).

Figure 2. The six anterior and posterior ROIs used for the analyses in each experiment.

218 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

It was important to investigate whether thescalp distribution of the negativity differed be-tween orientation and functional mismatches.

Planned, pairwise comparisons with the fact-ors Condition (orientation mismatches, func-tional mismatches), AntPos, Hemisphere and

TABLE 2The results of the main analysis of Experiment 1 for the three latency windows (360!480, 500!700, 600!800).

360!480 500!700 600!800

Source Df F MSE o F MSE o F MSE o

Orient 1, 30 13.0*** 66.4 9.0** 73.0 4.9 71.3Orient*Hemi 1, 30 6.8* 0.7 0.6 1.7 0.1 3.1Orient*Hemi*AntPos 1, 30 1.0 0.2 0.1 0.7 0.1 1.8Orient*Eccen 2, 60 4.2* 0.3 .60 3.6* 0.6 .62 0.0 0.8 .66Orient*Eccen*AntPos 2, 60 0.2 0.0 .83 1.7 0.7 .91 0.4 0.3 .98Funct 1, 30 11.8*** 80.9 0.2 80.4 10.7** 96.1Funct*Hemi 1, 30 1.5 0.6 3.1* 1.3 7.9** 2.8Funct*Hemi*AntPos 1, 30 8.4** 0.2 3.0* 0.5 1.3 1.2Funct*Eccen 2, 60 3.2* 0.4 .68 2.3 0.3 .75 10.6*** 0.8 .76Funct*Eccen*AntPos 2, 60 3.8* 0.1 .80 2.2 0.2 .80 2.8* 0.4 .82Orient*Funct 1, 30 28.4*** 39.2 0.1 67.0 1.1 67.6Orient*Funct*Hemi 1, 30 0.3 1.0 0.0 1.1 0.5 2.5Ori*Fun*Hemi*AntPos 1, 30 0.1 0.2 0.9 0.5 0.1 1.4Ori*Fun*Eccen 2, 60 4.5* 0.2 .70 0.3 0.4 .74 0.7 0.9 .76Ori*Fun*Eccen*AntPos 2, 60 0.1 0.1 .76 2.0 0.1 .79 1.4 0.3 .78

Only those effects are listed that reached significances in at least one of the latency windows in one of the experiments andincluded either the factor Orientation or Function or both (*pB.1, **pB.01, ***pB.001). For effects concerning scalp distributions,the results are based on the correction of McCarthy & Wood (1985).

Figure 3. Evoked potentials for the three mismatching conditions plotted against the fully matching condition (Experiment 1).

ACTION COMPREHENSION 219

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

Eccentricity did not reveal differences betweenorientation and functional mismatches, on eitherthe anterior or posterior ROIs.

Finally, there was an interaction of Orientationand Function. This means that the negativityelicited by double mismatches was not an additivefunction of the two single mismatches. In parti-cular, although both orientation and functionalmismatches were more negative than the correctcondition (orientation: F(2, 30)"32.0, MSE"57.4, pB.001; function, F(2, 30)"29.2, MSE"73.9, pB.001), double mismatches did not elicita larger negativity than either of these singlemismatches (both, FB1).

A similar interaction was also present in thescalp distributions (interaction of Orientation,Function and Eccentricity). Again, even thoughboth single mismatches showed a more centralscalp distribution than full matches (orientation:F(2, 60)"9.0, MSE"0.2, pB.005; function, F(2,60)"6.8,MSE"0.2,pB.005), the centrality of the

scalp distribution of double mismatches was notmore pronounced than the scalp distribution ofeither of the two single mismatches (both, FB1).

Latency window 2 (500!700 ms)

The main effect of Orientation confirmed thatorientation mismatches evoked an overall positivemodulation in this interval. Consistent with visualinspection, the positivity was centrally distributed(interaction of Orientation with Eccentricity).Neither effects of lateralization nor different scalpdistributions on anterior or posterior ROIs werestatistically detectable.

There was no similar overall main effect offunction. Although also for functional mis-matches a positivity started to emerge in thisinterval, a remainder of the preceding negativitywas still present in the right hemisphere, asindicated by the interaction of Function andHemisphere. As was the case for the negativity

Figure 4. Scalp distributions of the voltage differences between the fully matching condition and the three types of mismatchingconditions (from left to right: functional only mismatches, orientation only mismatches, double mismatches) in the three latencywindows (from top to bottom: 360!480 ms, 500!700 ms, 600!800 ms) in Experiment 1.

220 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

in the first latency window, the amount oflateralization differed on anterior and posteriorROIs (interactions of Function, Hemisphere andAntPos). On posterior ROIs, the positivity wasmore negative on the right than on the left,F(1, 30)"12.0, MSE"0.7, pB.005. On anteriorROIs no effect of lateralization was detectable,F(1, 30)"1.19.

As before, we investigated whether theselateralization effects at posterior ROIs differedbetween conditions. Functional mismatches,F(1, 30)"5.2, MSE"1.0, pB.05, and doublemismatches, F(1, 30)"8.9, MSE"0.7, pB.01,showed this lateralization when compared tofull matches, but orientation mismatches didnot (FB1). Double mismatches showed a stron-ger lateralization than orientation mismatches,F(1, 30)"8.1, MSE"0.4, pB.01, but for func-tional mismatches this difference to orientationmismatches was not significant, F(1, 30)"2.2,MSE"1.1, pB.15.

Latency window 3 (600!800 ms)

This latency window captures the positivemodulation elicited by functional mismatches,but the positivity elicited by orientationmismatches had already started to disappear.Consistently, no effect of orientation mismatcheswas detectable in this interval: There was no maineffect of Orientation, and Orientation did notinteract with Function or with any factor describ-ing the scalp distribution.

There was, of course, a main effect of Function,confirming the positivity elicited by functionalmismatches. It had a different scalp distributionon anterior and posterior ROIs (interactionsof Function and Eccentricity and of Function,Eccentricity and AntPos). Post-hoc analysesshowed that the effect had a central scalpdistribution on anterior ROIs, F(2, 60)"11.2,MSE"0.7, pB.001, o".74, but less so on pos-terior electrode sites, F(2, 60)"3.0, MSE"0.5,pB.08, o".69; not significant when corrected formultiple comparisons.

As before, the functional positivity was morenegative on the right than on the left (interactionof Function and Hemisphere), indicating that aremainder of the preceding negativity was stillpresent. This right lateralization was presentwhen functional mismatches were compared tofull matches, F(1, 30)"7.8, MSE"3.5, pB.01,and to orientation mismatches, F(1, 30)"4.3,MSE"3.2, pB.05, but not for the comparison

of orientation and full matches, F(1, 30)"1.1.It was also marginally significant for the com-parison of double mismatches and full matches,F(1, 30)"3.4, MSE"2.7, pB.08. Thus, even inthis latest latency window a remainder of thepreceding negativity was present for functionaland double mismatches, but not orientation mis-matches.

Discussion

Both functional and orientation mismatches gaverise to a negativity followed by a positive mod-ulation. The negativity elicited by functionalmismatches was similar to the N400 componentusually observed for semantic mismatches (VanPetten, 1995; Kutas & Federmeier, 2000), withregard to both latency and scalp distribution ofpictorial stimuli (West & Holcomb, 2002). Inparticular, a slight right lateralized distributionhas often been observed before (Kutas & Hill-yard, 1984). An N400-like negativity for func-tional mismatches was predicted from priorresearch. It indicates that the functional mis-matches in the action snapshots of the presentstudy evoked very similar brain responses asmismatches in actions that were fully executedand presented as movies (Sitnikova et al., 2003).

Interestingly, a very similar negativity!positiv-ity pattern was also observed for orientation (anddouble) mismatches. This similarity might indi-cate a similarity of the underlying processes, orthat both kinds of mismatches are detected on thebasis of the same action representation. Consis-tent with this view, the negativities in the twoconditions were not independent of one another.In contrast to what would have been expected ifboth negativities reflected independent processes,the negativity elicited by double mismatcheswas not larger than the negativities elicitedby the single mismatches. Although it is prema-ture to provide a specific interpretation for thisinteraction, it indicates that the processes detect-ing both kinds of mismatch operate on over-lapping representations.

Other aspects of the findings indicated that theprocesses were at least partially different. First,the observed right lateralization of orientationmismatches relative to full matches was predictedfrom previous findings, demonstrating that spatialjudgments of objects are predominantly carriedout in the right hemisphere (e.g., Inoue et al.,1998, Pegna et al., 1997, Turnbull et al., 1997). For

ACTION COMPREHENSION 221

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

functional mismatches, the right lateralizationwas less pronounced and restricted to posteriorelectrode sites. However, these differences mustbe interpreted with caution because they were notsignificant when the negativities elicited by func-tional and orientation mismatches were directlycompared.

Second, the two mismatch effects also followeddifferent time courses. The negativity followingfunctional mismatches lasted longer than thenegativity elicited by orientation mismatches. Inthe second latency window (500!700 ms), thenegativity elicited by orientation mismatcheshad already disappeared and was replaced by apositive modulation. For functional mismatches,a remainder of the negativity was still present inthe right hemisphere, where the negative mod-ulation was also the strongest in the first latencywindow (360!480 ms). Even in the last latencywindow (600!800 ms), the positive modulationevoked by functional mismatches was still morenegative in the right than in the left hemisphere.Because the effects of orientation and functionalmismatches were of equal peak amplitude in thefirst latency window, and because the detectiontimes and difficulties were comparable (e.g., Bachet al., 2005), the difference in time course suggeststhat the underlying processes are at least partiallydifferent.

Interestingly, this right lateralization could alsobe confirmed for double mismatches, indicatingthat a remainder of the right lateralized negativityelicited by functional mismatches was also pre-sent for double mismatches.

EXPERIMENT 2: TARGET OBJECTSPRECEDE EFFECTORS/INSTRUMENTS

In this experiment, the target object was pre-sented before the effector/instrument. ERPswere time-locked to the presentation of thesecond stimulus. Therefore, they now reflectorientation and functional processing of theeffector/instrument relative to the target objectpresented in the first frame. For functional mis-matches, we expect an effect similar to Experi-ment 1. However, now, a left lateralization fororientation mismatches was predicted for orien-tation mismatches. The reason is that studies onthe spatial manipulation of body parts haverevealed a stronger involvement of the left

hemisphere than the spatial manipulation ofexternal objects.

Method

Participants

Thirty-two German students (16 female) ran-ging in age from 20 to 30 years participated in thestudy and received approximately t25 for theirparticipation. None of them had participatedin Experiment 1. One participant was excludedfor excessive drifts. All participants were right-handed and had normal or corrected-to-normalvision. The key assignment was counterbalancedacross participants.

Material

The material was identical to that of the pre-vious experiment.

Procedure and design

The experimental setup was identical to that ofthe previous experiment. The course of each trialwas also identical to the previous experiment,with the exception that the target objects werenow shown in the first frame and the effectors/instruments in the second.

Recordings and data analyses

The recordings were carried out as in theprevious experiment. Latency windows were as-signed according to the procedure described inExperiment 1.

Results

Participants committed 3% errors judging thematching condition. Orientation mismatches wereincorrectly judged in 5% of the trials, functionalmismatches were incorrectly judged in 4% ofthe trials and double mismatches were incor-rectly judged in 3% of the trials. There were nosignificant differences between the conditions.

As can be seen from Figure 5, all threemismatches again gave rise to a negativityfollowed by a positive modulation. To investigatethe ERPs statistically, three latency windows

222 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

were assigned to the evoked potentials accordingto the procedure described above.3 There wasagain one latency window that captured thenegativities elicited by orientation, functionaland double mismatches (320!440 ms). In thepresent experiment it was not strictly necessaryto have two latency windows to describe thepositive effects, because there was considerableoverlap of the latency ranges in which the effectsin the mismatching conditions reached theirmaximum. Still, two separate latency windowswith identical relative timings and durations asin Experiment 1 were used in order to obtaincomparable results between the experiments(450!650 ms, 550!750 ms). Table 3 shows theresults of these ANOVAs carried out for eachof the latency windows. Figure 6 shows the scalpdistributions of the differences between the fullymatching and each mismatching condition in thethree latency windows.

Latency window 1 (320!440 ms)

Functional mismatches again gave rise to anegative inflection, as indicated by the maineffect of Function. Its scalp distribution differedbetween anterior and posterior ROIs (interactionof Function, Eccentricity and AntPos), beingcentrally distributed on posterior ROIs, F(2,60)"4.3, MSE"0.2, pB.06, o".585, but not onanterior ROIs (FB1).

There was no main effect of Orientation, butan interaction of Function and Orientation. Tofurther investigate this interaction, we comparedfull matches with each of the mismatchingconditions. All mismatches gave rise to a nega-tive inflection (all, pB.001) compared to fullmatches. It was larger for functional mismatchesthan for orientation mismatches, F(1, 30)"8.5,MSE"72.8, pB.05, and double mismatches, F(1,30)"9.8, MSE"67.3, pB.005. Orientation anddouble mismatches did not differ statistically.

The interaction of Orientation, Function andEccentricity indicated that the conditions dif-fered in how pronounced the central scalpdistribution was. To investigate these differences,separate pairwise comparisons between the threemismatching conditions and the fully matchingcondition were run. The factors were Condition

Figure 5. Evoked potentials for the three mismatching conditions plotted against the fully matching condition (Experiment 2).

3 The selection of the latency windows was again validatedwith the procedure described in footnote 1. After adjusting formultiple comparisons, significant differences started to appearafter 320 ms and remained present in all 40 ms intervals withthe exception of the interval between 440 and 480 ms, whichagain corresponded to the end of our first latency window andthe start of latency window 2.

ACTION COMPREHENSION 223

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

(full matches, orientation mismatches, functionalmismatches, or double mismatches) and Eccen-tricity. Compared to full matches, both orienta-tion mismatches, F(2, 60)"5.6, MSE"0.3, pB.05, o".60, and functional mismatches, F(2,60)"19.3, MSE"0.3, pB.001, o".60, had amore pronounced central scalp distribution, butdouble mismatches did not (FB1). Importantly,the central scalp distribution was more pro-nounced for functional than for orientationmismatches, F(2, 60)"6.5, MSE"0.2, pB.01,o".64.

Analogous pairwise comparisons were alsorun to investigate the interaction of Function,Orientation and Hemisphere that was significantafter the normalization procedure of McCarthyand Wood (1985) was applied, F(1, 30)"5.2,MSE"0.6, pB.05. Orientation mismatches andfull matches differed in their lateralization, F(1,30)"9.1, MSE"0.5, pB.005. This was consis-tent with visual inspection, according to whichorientation mismatches gave rise to a left later-alized negativity. No hemispheric asymmetrieswere apparent for the comparison of functionalor double mismatches with the fully matchingcondition (both, FB1). The direct comparison oforientation mismatches with functional mis-matches revealed hemispheric asymmetries forthe uncorrected tests, F(1, 30)"4.8, MSE"6.9,

pB.05, but not for the correction of McCarthyand Wood, F(1, 30)"1.99.

Latency window 2 (450!650 ms)

As in Experiment 1, there was a main effect forOrientation confirming the positive modulationelicited by orientation mismatches. As before, itwas centrally distributed (interaction of Orienta-tion and Eccentricity), and showed neither differ-ences between anterior and posterior ROIs nor alateralization.

Although there was no main effect of Func-tion, there were interactions of Function andEccentricity, and of Function, AntPos and Eccen-tricity, indicating different scalp distributions overanterior and posterior ROIs. The conditionscontaining a functional mismatch elicited a posi-tive modulation with a central scalp distributionon anterior ROIs, F(1, 30)"10.59, MSE"0.3,pB.001, o".71, but a flat distribution on poster-ior ROIs (FB1). This was consistent with visualinspection according to which the positive mod-ulation after functional mismatches emerged firstover anterior central electrodes.

Finally, there was an interaction of Function,Orientation and Eccentricity. Although all mis-matching conditions had a more pronouncedcentral distribution than full matches (all,pB.01), they did not differ from each other.

TABLE 3The results of the main analysis of Experiment 2 for the three latency windows (320!440, 450!650, 550!750).

320!440 450!650 550!750

Source Df F MSE o F MSE o F MSE o

Orient 1, 30 1.5 95.9 18.8*** 80.1 10.1* 73.4Orient*Hemi 1, 30 2.7 0.4 0.7 0.8 0.0 2.0Orient*Hemi*AntPos 1, 30 2.5 0.2 0.2 0.4 0.0 1.2Orient*Eccen 2, 60 2.2 0.2 .61 12.9*** 0.5 .61 10.1*** 0.6 .64Orient*Eccen*AntPos 2, 60 0.0 0.1 .80 0.4 0.1 .76 0.0 0.2 .78Funct 1, 30 22.7*** 97.9 0.5 111.9 4.1* 111.3Funct*Hemi 1, 30 0.1 0.6 2.7 0.8 5.3* 1.6Funct*Hemi*AntPos 1, 30 .01 0.2 3.1* 0.4 5.2* 0.7Funct*Eccen 2, 60 0.4 0.5 .56 3.6* 0.5 .59 4.8* 0.6 .64Funct*Eccen*AntPos 2, 60 6.3** 0.1 .93 13.2*** 0.1 .85 9.4*** 0.2 .88Orient*Funct 1, 30 45.1*** 51.6 1.1 75.4 5.8* 77.6Orient*Funct*Hemi 1, 30 2.1 0.9 0.7 1.0 0.0 1.7Ori*Fun*Hemi*AntPos 1, 30 0.0 0.1 0.5 0.3 0.0 0.6Ori*Fun*Eccen 2, 60 24.8*** 0.3 .67 2.8* 0.4 .66 3.5* 0.8 .65Ori*Fun*Eccen*AntPos 2, 60 1.4 0.1 .85 0.1 0.1 .92 0.1 0.2 .93

Only those effects are listed that reached significance in at least one of the latency windows in one of the experiments andincluded either the factor Orientation or Function or both (*pB.1, **pB.01, ***pB.001). For effects concerning scalp distributions,the results are based on the correction of McCarthy & Wood (1985).

224 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

Latency window 3 (550!750 ms)

As in the previous latency window, there wasa main effect of orientation mismatches. Thepositive modulation elicited by orientation mis-matches was again centrally distributed (interac-tion of Orientation and Eccentricity) withoutevidence of lateralization.

A main effect of Function was also obtained.This replicates the finding from Experiment 1 thatthe positivity elicited by functional mismatcheswas fully present only in the latest latency window.As always, its scalp distribution differed betweenanterior and posterior ROIs (interactions of Func-tion and Hemisphere, of Function, AntPos andHemisphere, of Function and Eccentricity, and ofFunction, AntPos and Eccentricity). On anteriorROIs, the positivity was centrally distributed, F(2,60)"12.1, MSE"0.4, pB.001, o".77, withoutevidence of lateralization (FB1). On posteriorROIs, the positivity had a flat distribution (FB1)

that was more negative on the right than on theleft, F(1, 30)"18.4, MSE"0.8, pB.001.

As before, we investigated whether the later-alization over posterior ROIs was detectable forboth conditions containing function mismatches.It was present when the fully matching conditionwas compared to double mismatches, F(1, 30)"10.6, MSE"1.1, pB.005, and to functional mis-matches, F(1, 30)"5.9, MSE"1.1, pB.05, butnot when it was compared to orientation mis-matches (FB1). In addition, the lateralization oforientation mismatches differed significantlyfrom the lateralization of functional mismatches,F(1, 30)"4.8, MSE"0.8, pB.05, and doublemismatches, F(1, 30)"11.4, MSE"0.7, pB.005.

Finally, there were interactions of Orientationand Function and of Function, Orientation andEccentricity. Even though all mismatching condi-tions showed an overall difference when comparedto full matches (all, pB.01), their amplitudesdid not differ from each other. Similarly, even

Figure 6. Scalp distributions of the voltage differences between the fully matching condition and the three types of mismatchingconditions (from left to right: functional only mismatches, orientation only mismatches, double mismatches) in the three latencywindows (from top to bottom: 320!440 ms, 450!650 ms, 550!650 ms) in Experiment 2.

ACTION COMPREHENSION 225

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

though all mismatching conditions had a morepronounced central scalp distribution than fullmatches (all, pB.01), there were no differences inEccentricity between the mismatching conditions.

Discussion

Experiment 2 replicated the main findings ofExperiment 1. Orientation and functional mis-matches elicited a negative inflection followed bya positive modulation. Again, the two processeswere not independent of one another. Thenegativity elicited by double mismatches wasnot an additive effect of the negativities elicitedby the two single mismatches.

Importantly, in the present experiment, thenegativities elicited by orientation and functionalmismatches differed in their scalp distributions.The negativity evoked by functional mismatcheshad a more pronounced central scalp distributionthan the negativity elicited by orientation mis-matches. In addition, the negativity elicited byorientation mismatches showed a lateralization tothe left, which was not obtained for functionalmismatches. This lateralization was predictedfrom the finding that orientation processing ofinstruments/effectors would give rise to a lefthemispheric response. It also confirms that theparticipants differentiated between two objectsinvolved in the actions. Instead of solving the taskby treating both stimuli as external objects, theyseemed to be aware that one was an externaltarget object and the other an instrument held bya hand.

Although less pronounced, the differences intime course were also similar to Experiment 1.The negativity elicited by functional mismatcheswas longer lasting than the negativity elicited byorientation mismatches and seemed to overlapwith the following positive modulation. As in theprevious experiment, the positivity elicited byfunctional mismatches appeared around 100 mslater than the positivity elicited by orientationmismatches. Moreover, at least for the lastlatency window, the positivity elicited by func-tional mismatches was lateralized to the left overposterior ROIs, which would be expected if ! asin the previous experiment ! a remainder of thefunctional negativity was still present in the righthemisphere. This lateralization could again be

demonstrated for functional and double but notorientation mismatches, consistent with the viewthat the processes evoked by functional proces-sing also take place in double mismatches.

COMPARISON OF THE TWOEXPERIMENTS

A comparison of the results of Experiments 1 and2 suggests that stimulus type (effector/instrumentor target object) affects orientation processing,but not the processing of object function.Whereas the negativities elicited by functionalmismatches were similar in the two experiments,a right lateralization was observed for the proces-sing of the orientation of the target objects, and aleft lateralization for the spatial processing ofeffectors/instruments. To confirm this differentialeffect of stimulus type statistically, the data fromthe two experiments was entered into oneANOVA with the between-subjects factor Type(ERPs reflect the processing of effector/instru-ment, or the target object) and the within-subjectfactors Orientation, Function, and Hemisphere.This was done for each of the three latencywindows separately. For the sake of brevity, onlythe effects qualified by stimulus Type are dis-cussed.

Results

Latency window 1 (Experiment 1, 360!480 ms;Experiment 2, 320!440 ms)

The ANOVA revealed a significant interactionof Orientation, Hemisphere and Type, F(1, 60)"9.65, MSE"0.5, pB.005, but no analogous inter-action of Function, Hemisphere and Type, F(1,60)B1. Therefore, stimulus type affected thelateralization of orientation mismatches but notof functional mismatches. The direct comparisonof orientation and functional mismatches furtherconfirmed this notion. The marginally significantinteraction of Hemisphere, Condition and Type,F(1, 60)"3.50, MSE"0.4, pB.07, indicated thatthe effect of stimulus type was stronger fororientation than for functional mismatches.

We investigated whether this pattern could beshown for each of the mismatching conditionswhen compared to the fully matching conditions.

226 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

Indeed, there was an interaction of Condition,Hemisphere and Type when orientation mis-matches were compared to the fully matchingcondition, F(1, 60)"10.6, MSE"0.6, pB.005,but not when functional mismatches were com-pared to the fully matching condition, F(1, 60)B1. In addition, the interaction of Hemisphere,Condition and Type was also present for doublemismatches, F(1, 60)"5.5, MSE"0.7, pB.05.This is consistent with the view that doublemismatches show contributions of the spatialand the functional process.

Latency windows 2 and 3 (Experiment 1, 500!700 ms and 600!800 ms; Experiment 2, 450!650 ms and 550!750 ms)

All effects mirror those of the analyses ofExperiment 1 and 2. No effect was qualified byType.

Summary

The comparison of the two experiments revealedtwo important findings. First, stimulus type onlyaffected spatial processing, not functional proces-sing. The negativity following functional mis-matches was not affected by the manipulation ofstimulus type. The spatial processing of targetobjects in Experiment 1 and effectors/instrumentsin Experiment 2 was associated with left and rightlateralized negativities, respectively. Such a hemi-spheric asymmetry was predicted if the detectionof orientation but not functional mismatches wasbased on the same processes that also occur whenhumans (mentally) rotate either body parts orexternal objects.

Second, Stimulus type only affected the nega-tive modulations, not the positive modulations.This can be interpreted as the negative effectsbeing specific to the type of mismatch (spatial,functional, double) and stimulus type (effectorsvs. target objects), whereas the positive effectsoccurred for all stimulus and mismatch types.

GENERAL DISCUSSION

The understanding of actions of tool use dependsnot only on the motor act that is performed, butalso on the objects that are used. We investigatedthe electrophysiological correlates of the pro-

cesses that integrate both aspects into a repre-sentation of the observed action.

ERPs were recorded while the participantswatched inserting actions that could mismatch intwo ways. Functional mismatches occurred whenthe function of the objects was not appropriate toachieve the given action outcome (e.g., credit cardbeing applied to a ticket canceller). Orientationmismatches occurred when the orientations ofinsert and slot of the target object were notconsistent with an inserting action being per-formed (different orientations of insert and slot).Because the two mismatches were independent ofeach other they could also occur simultaneously(double mismatch).

Both types of mismatch gave rise to anegative inflection in the latency range of theN400-component followed by a positive modula-tion. For functional mismatches, such a patternwas predicted on the basis of prior research onaction and gesture perception (Gunter & Bach,2004; Sitnikova et al., 2003). The new findingwas that a very similar N400-like negativity anda later occurring positivity were also evoked byorientation mismatches. This similarity in theERPs was surprising because the detection oforientation and functional mismatches could bebased on very different kinds of information.Whereas the detection of functional mismatchesrequired deriving ‘‘semantic’’ memory-based ob-ject representations, the detection of orientationmismatches depended on the objects’ spatialproperties. N400-effects have, however, beenobserved before when very different kinds ofsemantic information ! even across sensorymodalities ! had to be integrated into a commonrepresentation (for a review, see Kutas &Federmeier, 2000; Kounios, 2002). As such, thepresent data suggest that the negativities wereevoked when either an orientation or a func-tional mismatch precluded the integration of thenew object into an action representation thatspecified which objects had to be used and whichmotor act had to be performed for the action tobe successful (cf. Bach et al., 2005).

The interaction between the negativities con-firmed that action knowledge provided a commoncontext into which the spatial and functionalobject properties were integrated. The negativ-ities elicited by double mismatches were not anadditive function of the negativities elicited byspatial and functional mismatches. They wereeither of the same amplitude (Experiment 1) oronly as large as the smaller of the two negativities

ACTION COMPREHENSION 227

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

elicited by these mismatches (Experiment 2). Thispattern suggested that whichever mismatch wasdetected more easily disrupted the action repre-sentation that had been established, so that anintegration of the second dimension was notundertaken. A similar finding was provided by abehavioral study using the same stimuli (Bachet al., 2005). There, the presence of an orientationmismatch interfered with the judgment of func-tionally matching actions, but not with actionsthat already contained a functional mismatch.Thus, consistent with the present findings, thepresence of a mismatch was irrelevant if norepresentation of the observed action could beestablished due to another mismatch.

Of course, the notion that a common actionrepresentation is established from the motor actand the function of the objects does not mean thatthe two processes deriving both kinds of informa-tion are identical. Indeed, there were reliabledifferences between the ERPs elicited by the twokinds of mismatches. Because the detection timesand difficulties of the two mismatches are highlycomparable (e.g., Bach et al., 2005) and partici-pants saw exactly the same instruments and targetobjects in each condition, these differences in-dicate that the underlying processes or brainstructures are at least partially different.

First of all, there were differences in timecourse and scalp distribution. Consistent withprior research on N400-like negativities evokedby pictorial stimuli (Sitnikova et al., 2003;West & Holcomb, 2002), the negativities elicitedby functional mismatches had a central scalpdistribution and were long lasting. In contrast,the negativity elicited by orientation mismatcheswas shorter lasting and also had a flatter scalpdistribution, at least when the ERPs reflected theprocessing of effectors and instruments (Experi-ment 2). The differences in scalp distribution andtime course were particularly apparent in thelater latency windows in both experiments, wherea remainder of the preceding negativity was stillpresent in the right hemisphere for function (anddouble) mismatches, but not for orientation mis-matches.

The negativities also seemed to be differen-tially affected by the manipulation of stimulustype. Orientation mismatches gave rise to leftand right lateralized negativities for the proces-sing of instruments/effectors (Experiment 2) andtarget objects (Experiment 1), respectively. Incontrast, the negativities elicited by functionalmismatches had an equally pronounced right-

ward lateralization in both experiments. Such anasymmetry was predicted if the detection oforientation mismatches ! but not functionalmismatches ! relied on the parietal and motorstructures that are also involved in orientationjudgments of objects and body parts (e.g.,Kosslyn et al., 1998; Tomasino et al., 2003).The present findings are therefore consistentwith the idea that the recognition of the motoract relies on structures in the parietal!premotorpathway, most notably the mirror neurons, andthat the integration of object function into anaction is supported by a separate pathway,possibly involving temporal and prefrontalstructures (Hodges, Bozeat, Lambon Ralph,Patterson & Spatt, 2000). The parallel involve-ment of both of these systems in actionperception has demonstrated before in a mag-netoencephalography study on the perception ofsymbolic gestures (Nakamura et al., 2004).

Taken together, the negativities elicited byboth mismatches did not only reflect a generalprocess that established an action representationform the motor act and from the function of theobjects. They also reflected distinct subprocessesthat derive both kinds of information from anobserved action. Such a hybrid view of the N400is consistent with prior research that describes therole of the N400 as deriving meaning by bridgingmodality specific sensorimotor information andmore general higher-level representations (Kutas& Federmeier, 2000; Holcomb & West; 2002;Kounios, 2002).

Further studies need to clarify the functionalrole of the positive modulation that followed thenegativities elicited by both types of mismatch inour study and previous studies on action percep-tion (Gunter, Knoblich, Bach, Prinz, & Friederici,2002; Sitnikova et al., 2003). As noted by Sitni-kova and colleagues (2003), this late positivity isreminiscent of the P600 effect obtained in lan-guage studies. The P600 is elicited by an anom-alous sentence structure or by syntacticmismatches (Friederici, 1995; Hagoort & Brown,2000; Osterhout, Holcomb, & Swinney, 1994;Osterhout & Mobley, 1995). Therefore, Sitnikovaand colleagues (2003) argued that their functio-nal mismatches might have contained structuralmismatches as well, which were responsiblefor the P600-like modulation. In contrast tothis hypothesis, the present findings suggest thatthe late positive effect can be elicited by anymismatch in the perceived action, irrespective ofwhether it is functional or structural.

228 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

This does not rule out that the late positivity isa P600. Recent research has started to challengepurely structural/syntactic interpretations of theP600. P600 effects have been reported for seman-tic anomalies in sentences when these anomalieswere realized as a mismatch between a verb andthe semantic/thematic features of its noun argu-ments (e.g., van Herten, Kolk, & Chwilla, 2005).In such cases the P600 might also follow N400effects (e.g., Friederici & Frisch, 2000; Frisch &Schlesewsky, 2001), as was the case in the presentstudy. It has therefore been proposed that theP600 reflects a more general checking mechan-ism. Accordingly, the P600 is elicited when thereis a mismatch between the world knowledge ofthe listener and an unexpected sentence meaning(van Herten et al., 2005). Such a more generalinterpretation of P600 effects would also accom-modate the present findings. According to thisview, the late positive modulations in our studywere elicited because orientation and functionalmismatches required a ‘‘reading’’ of the perceivedaction that did not correspond to the worldknowledge of the observers.

It is intriguing that mismatches in actions cangive rise to similar ERPs as mismatches insentences. This observation resonates well withthe suggestion that the capacity for language hasdeveloped out of the capacity to produce andunderstand actions (Arbib, 2005; Rizzolatti &Arbib, 1998), or that the same types of concep-tual knowledge are involved in both domains(Lindemann, Stenneken, van Schie, & Bekkering,2006). Of particular interest might be that thenegativity!positivity pattern in the present ex-periments matches the N400!P600 patternelicited by verb!argument structure violationsin language. It has been speculated that thelanguage system could have developed naturallyfrom action representations that were alreadyorganized in a prototypical verb!argument gram-mar, where the verb reflects the actions, and thenouns reflect the objects involved in it (Arbib,2005; Rizzolatti & Arbib, 1998). Of course,although our data are certainly suggestive ofsuch a view, future studies are needed thatinvestigate more directly whether the similarityin the ERPs elicited mirrors a similarity in theunderlying brain structures or processes.

Since the present study is one of the firstinvestigating action observation with ERPs, openquestions remain. First, because our instructionrequired participants to parse the scenes asactions, our study cannot say anything about the

automaticity of the processes that establish theaction representations. Two recent studies con-firmed, however, that the spatial and functionalprocesses that were manipulated explicitly in thepresent study do occur automatically without aninstruction. Riddoch and colleagues (2003)showed that patients suffering from visual extinc-tion were ‘‘seeing an action’’ instead of twoobjects in isolation if the objects were at thesame time functionally and spatially appropriatefor an action to be carried out with them (e.g., penpresented above paper). Green and Hummel(2006) replicated this effect in healthy partici-pants and found again that action-like processingdepended on the spatial and functional appropri-ateness of the presented objects.

A second question results from the observationthat objects by themselves can elicit actiontendencies in the observer, even when no actionis presented (‘‘affordances’’, Gibson, 1979; Tucker& Ellis, 1998, 2001). This might be particularlytrue for functionally related objects. Patientsexhibiting utilization behaviour unintentionallyperform even complex actions of tool use whenthe objects involved in the action are placed infront of them (e.g., Lhermitte, 1983). Because weused static images that showed the actions in mid-flight before they were fully executed, it couldtherefore be the case that our ERP-effects reflectsuch complex action affordances being evoked,rather than the observation of actions of others.

One way to rule out this possibility would be torun the experiment again and remove the bodyparts from the stimuli. However, humans tend toinfer actions even when no body parts arepresented, as for instance demonstrated by thestudy of Riddoch and colleagues (2003), in whichthe patients were ‘‘seeing an action’’ in theabsence of body parts. As such, even if such anexperiment would yield the same results as thepresent study, we could not be sure whether theERPs reflect object affordances or inferred ac-tions. One could, however, also make the reverseprediction: If our effects reflect action observa-tion and not just affordances evoked by theobjects, then identical ERP effects should beevoked if the actions were shown as movies andfully executed. This indeed seems to be the case.The study of Sitnikova and colleauges (2003)presented movies in which even the functionallymismatching actions were fully executed (e.g., forshaving a rolling pin was used instead of a razor).Nevertheless, these mismatches evoked the sameN400!P600 pattern, albeit with slightly different

ACTION COMPREHENSION 229

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

scalp distributions, as the functional mismatchesin the action snapshots in the present study. Weare therefore confident that our results reflectaction perception processes rather than mereaction affordances.

CONCLUSIONS

The present study demonstrates that actioncomprehension needs to be conceptualized withregards to how both the motor act and thefunction of the objects shape the understandingof an action. Although at least partially differentsubprocesses seem to be involved in derivingthese two aspects, they were integrated into onecoherent representation of the observed action.N400-like negativities therefore seem to be ageneral marker of the difficulty of establishinga meaningful representation of the perceivedaction on the basis of motor and functionalinformation. The present results may also serveas a starting point for studies that investigatemore directly whether action and language un-derstanding relies on overlapping brain structuresor processes, as suggested by the negativityposi-tivity pattern evoked by mismatches in the twodomains.

Manuscript received 24 July 2006Manuscript accepted 18 July 2008

First published online 20 November 2008

REFERENCES

American Electroencephalographic Society. (1991).Guidelines for standard electrode position nomen-clature. Journal of Clinical Neurophysiology, 8,200!202.

Amorim, M.-A., Isableu, B., & Jarraya, M. (2006).Embodied spatial transformations: ‘‘Body analogy’’for the mental rotation of objects. Journal ofExperimental Psychology: General, 135(3), 327!347.

Arbib, M. A. (2005) From monkey-like action recogni-tion to human language: An evolutionary frame-work for neurolinguistics. Behavioral and BrainSciences, 28, 105!167.

Bach, P., Knoblich, G., Gunter, T. C., Friederici, A. D.,& Prinz, W. (2005). Action comprehension: Derivingspatial and functional relations. Journal of Experi-mental Psychology: Human Perception & Perfor-mance, 31(3), 465!479.

Bach, P., Knoblich, G., Gunter, T. C., Friederici, A. D.,& Prinz, W. (2008). Understanding instrumentalactions: Parallel integration of object and motorinformation. Manuscript submitted for publication.

Bach, P., Peatfield, N. A., & Tipper, S. P. (2007).Focusing on body sites: The role of spatial attentionin action perception. Experimental Brain Research,178, 509!517.

Bach, P., & Tipper, S. P. (2007). Implicit action encodinginfluences personal-trait judgments. Cognition, 102,151!178.

Barrett, S. E., & Rugg, M. D. (1990). Event-relatedpotentials and the semantic matching of pictures.Brain and Cognition, 14, 201!212.

Barsalou, L. W., Niedenthal, P. M., Barbey, A., &Ruppert, J. (2003). Social embodiment. In B. Ross(Ed.), The psychology of learning and motivation(Vol. 43, pp. 43!92). San Diego: Academic Press.

Bekkering, H., Wohlschlager, A., & Gattis, M. (2000).Imitation of gestures in children is goal-directed.Quarterly Journal of Experimental Psychology,53A, 153!164.

Boronat, C., Buxbaum, L. J., Coslett, H. B., Tang, K.,Saffran, E. M., Kimberg, D. Y., et al. (2005).Distinctions between function and manipulationknowledge of objects: Evidence from functionalmagnetic resonance imaging. Cognitive BrainResearch, 23, 361!373.

Brass, M., Bekkering, H., Wohlschlager, A., & Prinz, W.(2000). Compatibility between observed and exe-cuted finger movements: Comparing symbolic, spa-tial and imitative cues. Brain and Cognition, 44,124!143.

Bremmer, F., Duhamel, J.-R., & Ben Hamed, S. (1996).Non-retinocentric coding of visual space in themacaque ventral intraparietal area (VIP). Societyfor Neuroscience Abstracts, 22, 666!668.

Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L.,Fogassi, L., Gallese, V., et al. (2001). Actionobservation activates premotor and parietal areasin a somatotopic manner: an fMRI study. EuropeanJournal of Neuroscience, 13, 400!404.

Buxbaum, L. J., & Saffran, E. M. (1998). Knowing‘‘how’’ vs ‘‘what for’’: A new dissociation. Brain andLanguage, 65, 73!86.

Chao, L. L., & Martin, A. (2000). Representation ofmanipulable man-made objects in the dorsal stream.NeuroImage, 12, 478!484.

Costantini, M., Galati, G., Ferretti, A., Caulo, M.,Tartaro, A., Romani, G. L., & Aglioti, S. M.(2005). Neural systems underlying observation ofhumanly impossible movements: An fMRI study.Cerebral Cortex, 25, 1761!1767.

DiPellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., &Rizzolatti, G. (1992). Understanding motor events:A neurophysiological study. Experimental BrainResearch, 91, 176!189.

Friederici, A. D. (1995). The time course of syntacticactivation during language processing: A modelbased on neuropsychological and neurophysiologi-cal data. Brain and Language, 50, 259!281.

Friederici, A. D., & Frisch, S. (2000). Verb argumentstructure processing: The role of verb-specific andargument-specific information. Journal of Memoryand Language, 43, 476!507.

Frisch, S., & Schlesewsky, M. (2001). The N400 reflectsproblems of thematic hierarchizing. NeuroReport,12, 3391!3394.

230 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

Gallese, V. (2003). A neuroscientific grasp of concepts:From control to representation. Philosphical Trans-actions of the Royal Society of London, Series B, 358,1231!1240.

Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G.(1996). Action recognition in the premotor cortex.Brain, 119, 593!609.

Gallese, V., Fogassi, L., Fadiga, L., & Rizzolatti, G.(2002). Action representation and the inferiorparietal lobule. In W. Prinz & B. Hommel (Eds.),Attention & performance XIX. Common mechan-isms in perception and action (pp. 247!266). Oxford:Oxford University Press.

Geisser, S., & Greenhouse, S. W. (1959). On methods inthe analysis of profile data. Psychometrika, 24,95!112.

Gerardine, E., Sirigu, A., Leherecy, S., Poline, J.,Gaymard, B., Marsault, C., et al. (2000). Partiallyoverlapping networks for real and imagined handmovements. Cerebral Cortex, 10, 1093!1104.

Gibson, J. J. (1979). The ecological approach to visualperception. Boston: Houghton Mifflin.

Goldenberg, G. (2001). Imitation and matching of handand finger postures. NeuroImage, 14, 132!136.

Green, C. B., & Hummel, J. E. (2006). Familiarinteracting object pairs are perceptually grouped.Journal of Experimental Psychology: Human Per-ception and Performance, 32(5), 1107!1119.

Grezes, J., Armony, L., Rowe, J., & Passingham, R. E.(2003). Activations related to ‘‘mirror’’ and ‘‘cano-nical’’ neurones in the human brain: An fMRI study.NeuroImage, 18, 928!937.

Gunter, T. C., & Bach, P. (2004). Communicatinghands: ERPs elicited by meaningful symbolic handpostures. Neuroscience Letters, 372, 52!56.

Gunter, T. C., Knoblich, G., Bach, P., Prinz, W., &Friederici, A. D. (2002). Meaning and structure inaction comprehension: Electrophysiological evi-dence. Journal of Cognitive Neuroscience (Suppl.80). [9th annual meeting of Cognitive NeuroscienceSociety, San Francisco, April 2002.]

Hagoort, P., & Brown, C.M. (2000). ERPs effects oflistening to speech compared to reading: The P600/SPS to syntactic violations in spoken sentences andrapid serial visual presentation. Neuropsychologia,38, 1513!1549.

Hodges, J. R., Bozeat, S., Lambon Ralph, M. A.,Patterson, K., & Spatt, J. (2000). The role ofconceptual knowledge in object use: Evidencefrom semantic dementia. Brain, 123, 1913!1925.

Hodges, J., Spatt, J., & Patterson, K. (1999). ‘‘What’’and ‘‘how’’: Evidence for the dissociation of objectknowledge and mechanical problem-solving skills inthe human brain. Proceedings of the NationalAcademy of Science, 96, 9444!9448.

Holcomb, P. J. (1993). Semantic priming and stimulusdegradation: Implications for the role of the N400 inlanguage processing. Psychophysiology, 30, 47!61.

Holcomb, P. J., Kounios, J., Anderson, J. E., & West, W.(1999). Dual coding, context availability, and con-creteness effects in sentence comprehension: Anelectrophysiological investigation. Journal of Ex-perimental Psychology: Learning, Memory, andCognition, 2, 804!823.

Holcomb, P. J., & McPherson, W. B. (1994). Event-related brain potentials reflect semantic priming inan object decision task. Brain and Cognition, 24,259!276.

Holm, S. (1979). A simple sequentially rejective multi-ple test procedure. Scandinavian Journal of Statis-tics, 6, 65!70.

Hommel, B., Musseler, J., Aschersleben, G., & Prinz,W. (2001). The Theory of Event Coding (TEC): Aframework for perception and action planning.Behavioral and Brain Sciences, 24, 849!878.

Inoue, M., Yoshino, A., Suzuki, A., Ogasawara, T., &Nomura, S. (1998). Topographic study of humanevent-related potentials using a task requiringmental rotation. Neuroscience Letters, 253, 107!110.

Jacob, P., & Jeannerod, M. (2005). The motor theory ofsocial cognition: A critique. Trends in CognitiveSciences, 9(1), 21!25.

Jeannerod, M., Arbib, M. A., Rizzolatti, G., & Sakata,H. (1995). Grasping objects: The cortical mechan-isms of visuomotor transformation. Trends in Neu-roscience, 18, 314!320.

Johnson-Frey, S. H., Maloof, F. R, Newman-Norlund,R., Farrer, C., Inati, S., & Grafton, S. T. (2003).Actions or hand!object interactions? Human infer-ior frontal cortex and action observation. Neuron,39, 1053!1058.

Keehner, M. M., Guerin, S. A., Turk, D. J., Miller, M.,& Hegarty, M. (in press). Modulation of neuralactivity by angle of rotation during imagined spatialtransformations. NeuroImage.

Kellenbach, M. L., Brett, M., & Patterson, K. (2003).Actions speak louder than functions: the importanceof manipulability and action in tool representation.Journal of Cognitive Neuroscience, 15(1), 30!46.

Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L.,Gallese, V., & Rizzolatti, G. (2002). Hearing sounds,understanding actions: Action representation inmirror neurons. Science, 297, 846!848.

Kosslyn, S. M., DiGirolamo, G. J., Thompson, W. L., &Alpert, N. M. (1998). Mental rotation of objectsversus hands: Neural mechanisms revealed by posi-tron emission tomography. Psychophysiology, 35(2),151!161.

Kounios, J. (2002). A neural mechanism for nonverbaldiscourse comprehension. Trends in CognitiveSciences, 6, 272!275.

Kourtzi, Z., & Kanwisher, N. (2000). Activation inhuman MT/MST by static images with impliedmotion. Journal of Cognitive Neuroscience, 12(1),48!55.

Kutas, M., & Federmeier, K. D. (2000). Electrophysiol-ogy reveals semantic memory use in languagecomprehension. Trends in Cognitive Sciences,4(12), 463!470.

Kutas, M., & Hillyard, S. A. (1984). Brain potentialsduring reading reflect word expectancy and seman-tic association. Nature, 307, 161!163.

Lhermitte, F. (1983). Utilization behavior and itsrelation to lesions of the frontal lobes. Brain, 106,237!255.

Lindemann, O., Stenneken, P., van Schie, H. T., &Bekkering, H. (2006). Semantic activation in action

ACTION COMPREHENSION 231

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009

planning. Journal of Experimental Psychology:Human Perception and Performance, 32(3), 633!643.

Manthey, S., Schubotz, R. I., & von Cramon, D. Y.(2003). Premotor cortex in observing erroneousaction: An fMRI study. Cognitive Brain Research,15, 296!307.

McCarthy, G., & Wood, C. C. (1985). Scalp distribu-tions of event-related potentials: An ambiguityassociated with analysis of variance models. Electro-encephalography and Clinical Neurophysiology, 69,218!233.

Murata, A., Gallese, V., Luppino, G., Kaseda, M., &Sakata, S. (2000). Selectivity for the shape, size, andorientation of objects for grasping in neurons ofmonkey parietal area AIP. Journal of Neurophysiol-ogy, 83, 2580!2601.

Nakamura, A., Maess, B., Knosche, T. R., Gunter, T. C.,Bach, P., & Friederici, A. D. (2004). Cooperation ofdifferent neuronal systems during hand sign recog-nition. NeuroImage, 23, 25!34.

Nelissen, K., Luppino, G., Vanduffel, W., Rizzolatti, G.,& Orban, G. A. (2005). Observing others: Multipleaction representation in the frontal lobe. Science,310, 332!336.

Ochipa, C., Rothi, L. J. G., & Heilman, K. M. (1989).Ideational apraxia: A deficit in tool selection anduse. Annuals of Neurology, 25, 190!193.

Osterhout, L., Holcomb, P. J., & Swinney, D. A. (1994).Brain potentials elicited by garden-path sentences:Evidence of the application of verb informationduring parsing. Journal of Experimental Psychology:Learning, Memory, and Cognition, 20, 786!803.

Osterhout, L., & Mobley, L. A. (1995). Event-relatedbrain potentials elicited by failure to agree. Journalof Memory and Language, 34, 739!773.

Oztop, E., & Arbib., M. A. (2002). Schema design andimplementation of the grasp-related mirror neuronsystem. Biological Cybernetics, 87, 116!140.

Overney, L. S., Michel, C. M., Harris, I. M., & Pegna, A.J (2005). Cerebral processes in mental transforma-tions of body parts: Recognition prior to rotation.Cognitive Brain Research, 25(3), 722!734.

Pegna, A. J., Khateb, A., Spinelli, L., Seeck, M., Landis,T., & Michel, C. M. (1997). Unraveling the cerebraldynamics of mental imagery. Human Brain Map-ping, 5, 410!421.

Petit, L. S., Pegna, A. J., Harris, I. M., & Michel, C. M.(2006). Automatic motor cortex activation for nat-ural as compared to awkward grips of a manipulableobject. Experimental Brain Research, 168(1!2),120!130.

Riddoch, J. M., Humphreys, G. W., Edwards, S., Baker,T., & Wilson, K. (2003). Seeing the action: Neuro-physiological evidence for action-based effects onobject selection. Nature Neuroscience, 6(1), 82!89.

Rizzolatti, G., & Arbib, M. A. (1998). Language withinour grasp. Trends in Neuroscience, 21, 188!194.

Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L.(1996). Premotor cortex and the recognition of

motor actions. Cognitive Brain Research, 3(2),131!141.

Rizzolatti, G., Fogassi, L., & Gallese, V. (2001).Neurophysiological mechanisms underlying the un-derstanding and imitation of action. Nature ReviewsNeuroscience, 2(9), 661!670.

Sakata, H., & Taira, M. (1994). Parietal control of handaction. Current Opinion in Neurobiology, 4, 847!856.

Shmuelof, L., & Zohary, E. (2005). Dissociation be-tween ventral and dorsal fMRI activation duringobject and action recognition. Neuron, 47, 457!470.

Sitnikova, T., Kuperberg, G., & Holcomb, P. J. (2003).Semantic integration in videos of real-world events:An electrophysiological investigation. Psychophy-siology, 40(1), 160!164.

Sturmer, B., Aschersleben, G., & Prinz, W. (2000).Correspondence effects with manual gestures andpostures: A study of imitation. Journal of Experi-mental Psychology: Human Perception and Perfor-mance, 26(6), 1746!1759.

Tai, Y. F., Scherfler, C., Brooks, D. J., Sawamato, N., &Castiello, U. (2004). The human premotor cortex is‘‘mirror’’ only for biological movements. CurrentBiology, 14, 120!177.

Tomasino, B., Toraldo, A., & Rumiati, R. I. (2003).Dissociation between the mental rotation of visualimages and motor images in unilateral brain-da-maged patients. Brain and Cognition, 51(3),368!371.

Tucker, M., & Ellis, R. (1998). On the relationsbetween seen objects and components of potentialactions. Journal of Experimental Psychology:Human Perception and Performance, 24, 830!846.

Tucker, M., & Ellis, R. (2001). The potentiation ofgrasp types during visual object recognition. VisualCognition, 8, 769!800.

Turnbull, O. H., Beschin, N., & Della Sala, S. (1997).Agnosia for object orientation. Implications fortheories of object recognition. Neuropsychologia,24, 153!163.

van Herten, M., Kolk, H. H. J., & Chwilla, D. J. (2005).An ERP study of P600 effects elicited by semanticanomalies. Cognitive Brain Research, 22, 241!255.

Van Petten, C. (1995). Words and sentences: Event-related brain potential measures. Psychophysiology,32, 511!525.

Vasey, M. W., & Thayer, J. F. (1987). The continuingproblem of false positives in repeated measuresANOVA in psychophysiology: A multivariate solu-tion. Psychophysiology, 24, 479!486.

West, W. C., & Holcomb, P. J. (2002). Event-relatedpotentials during discourse-level semantic integra-tion of complex pictures. Cognitive Brain Research,13(3), 363!375.

Wilson, M., & Knoblich, G. (2005). The case for motorinvolvement in perceiving conspecifics. Psychologi-cal Bulletin, 131, 460!473.

232 BACH ET AL.

Downloaded By: [Bach, Patric] At: 14:46 8 May 2009