Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and...

Brain and Cognition 44, 124–143 (2000)

doi:10.1006/brcg.2000.1225, available online at http://www.idealibrary.com on

Compatibility between Observed and ExecutedFinger Movements: Comparing Symbolic,

Spatial, and Imitative Cues

Marcel Brass,* Harold Bekkering,† Andreas Wohlschlager,† andWolfgang Prinz†

*Department of Neurology, Max Planck Institute of Cognitive Neuroscience, Leipzig,Germany; and †Department of Cognition and Action, Max Planck Institute of

Psychological Research, Munich, Germany

Published online August 15, 2000

Intuitively, one can assume that imitating a movement is an easier task than re-sponding to a symbolic stimulus like a verbal instruction. Support for this suggestioncan be found in neuropsychological research as well as in research on stimulus–response compatibility. However controlled experimental evidence for this assump-tion is still lacking. We used a stimulus–response compatibility paradigm to testthe assumption. In a series of experiments, it was tested whether observed fingermovements have a stronger influence on finger movement execution than a symbolicor spatial cue. In the first experiment, we compared symbolic cues with observedfinger movements using an interference paradigm. Observing finger movementsstrongly influenced movement execution, irrespective of whether the finger move-ment was the relevant or the irrelevant stimulus dimension. In the second experi-ment, effects of observed finger movements and spatial finger cues were compared.The observed finger movement dominated the spatial finger cue. A reduction in thesimilarity of observed and executed action in the third experiment led to a decreaseof the influence of observed finger movement, which demonstrates the crucial roleof the imitative relation of observed and executed action for the described effects.The results are discussed in relation to recent models of stimulus–response compati-bility. Neurocognitive support for the strong relationship between movement obser-vation and movement execution is reported. 2000 Academic Press

I. INTRODUCTION

There are many possible ways to instruct an action. For example, if youwanted to teach a child how to type the letter ‘‘t’’ on a computer keyboard,you could provide a verbal command such as ‘‘press the sixth button of the

Address correspondence and reprint requests to Marcel Brass, Max Planck Institute of Cog-nitive Neuroscience, Dept. Neurology, Stephanstr. 1A, D-4103 Leipzig, Germany. E-mail:[email protected]. Fax: (49) 341 9940-221.

1240278-2626/00 $35.00Copyright 2000 by Academic PressAll rights of reproduction in any form reserved.

IDEOMOTOR COMPATIBILITY 125

second row with the index finger of your left hand.’’ You could also some-how mark the button and the finger to be used. A third possibility would beto demonstrate the typing; that is, you could show the required hand positionand the finger pressing on the ‘‘t.’’ Intuitively, one would assume that thethird instruction would be most easily interpreted by the child, as it couldsee what it is being asked to do.

Both, neuropsychological as well as cognitive research support the as-sumption that there are different mechanisms mediating action imitation andmediating responses to symbolic instruction. However controlled experimen-tal evidence for such a distinction in normal subjects is still missing. In thisstudy the assumption of a privileged path from movement observation tomovement execution was tested using a stimulus–response compatibilityparadigm.

Imitation in Neuropsychological Research

From neuropsychological studies it is well known that apraxic patientsare selectively disturbed for different command modalities (e.g., Schnider,Hanlon, Alexander, & Benson, 1997). On the one hand patients with ideomo-tor apraxia typically make the most errors on pantomime to verbal command,whereas imitation performance is less impaired or remains at least equallyimpaired (De Renzi, Motti, & Nichelli, 1980; Heilman & Rothi, 1985; Alex-ander et al., 1992). On the other hand, few cases with so-called conductionapraxia have been described where the patient had problems imitating a ges-ture while being less impaired on verbal command (Ochipa, Rothi, & Heil-man, 1994; Mehler, 1987). These distinct error patterns led some authorsto suggest that two different routes mediate imitative actions and verballyinstructed actions (Rothi, Ochipa, & Heilman, 1991; Roy & Hall, 1992).While in imitation the model provides some information that can be used forspecifying the response,executing thesameactiononverbalcommandrequiresthe internal generation of this information (Roy & Hall, 1992). The specialrole of imitative action becomes also obvious in patients showing compulsiveimitative behavior (Lhermitte, Pillon, & Serdaru, 1986). These patients cannotresist from imitating observed movements presented in front of them. Theseresults support the view that movement observation leads to an activation ofthe corresponding response, which is typically inhibited in normal subjects.But what are the cognitive mechanisms underlying imitative action?

A Functional Mechanism Underlying Action Imitation

Different cognitive theories have emphasized the tight relationship be-tween perception and action (e.g., Gibson, 1986; Prinz, 1990). A very promi-nent approach emphasizing this point is the ecological theory (Gibson, 1986).Central to this approach is the concept of affordance, which assumes that theperception of the external world always implies information about potential

126 BRASS ET AL.

related actions. Recently, Tucker and Ellis (1998) used a stimulus–responsecompatibility design to demonstrate that seen objects automatically potenti-ate components of the actions they afford. They referred to a modified ver-sion of the affordance concept introduced by Jeannerod (1994) to accountfor their results. In addition a study by Craighero et al. (1996, 1998) demon-strated that presentation of an object primes a response related to that object.While the affordance concept seems to be a reasonable account for responseactivation by visual presented objects, it is difficult to apply this concept toimitation of actions that are not object related. For example, observing some-one pointing in a specific direction typically does not afford a similar handgesture. Rather it provokes looking in the pointed direction.

An alternative view of imitative phenomena is provided by the theoryof ideomotor action (Greenwald, 1970a, 1970b). Greenwald argued that theanticipatory representation of an action’s sensory feedback (a response im-age) is used for action control. A key press, for example, is controlled bya representation of its sensory consequence (the visual impression of themovement or the tactile sensation of touching the key). When viewed fromthis perspective, the mechanism underlying action imitation can be formu-lated as follows: If a perceptual event is similar to the response image thatis used to control a response, then the perceived event should activate theresponse image and, hence, influence the initiation of the response. This de-scription of imitative action is rather consistent with the account presentedby Roy and Hall (1992), drawn from research on apraxia.

By introducing the concept of ideomotor compatibility, which Greenwalddefined as: ‘‘[The] dimension denoting the extent to which a stimulus corre-sponds to sensory feedback from its required response’’ (Greenwald, 1970b,p. 52), Greenwald extended this idea to compatibility phenomena as well. Theideomotor theory predicts that the ease with which a stimulus can be trans-formed into an action depends on the similarity between the observed eventand the executed action. In situations involving a verbal or a symbolic instruc-tion, this similarity is minimal, whereas in situations involving an observedaction that matches the required action, this similarity is maximal. To testthis assumption we have used a stimulus–response compatibility paradigm.

Investigating Ideomotor Compatibility with a Stimulus–ResponseCompatibility Paradigm

In cognitive psychology experimental studies on functional aspects of imi-tation are rare. It is the field of stimulus–response compatibility (SRC) whichprovides deeper insights into the question how stimulus properties influencethe selection and initiation of a response. SRC refers to the finding that hu-man performance is not only affected by characteristics of the stimulus setand the response set used in a task, but also by the combination of the sets(e.g., Kornblum, Hasbroucq, & Osman, 1990; Hommel & Prinz, 1997). Theterm SRC originally refers to the observation that a compatible mapping of


stimulus and response position leads to faster responses than an incompatiblemapping (Fitts & Deininger, 1954, Fitts & Seeger, 1953). Most recent modelsof SRC focus on the similarity between stimulus and response to explainSRC effects (Kornblum, Hasbroucq, & Osman, 1990; Hommel, Musseler,Aschersleben, & Prinz, 1999). Thus it is surprising that only very few com-patibility studies have investigated the question how movement observationaffects movement execution (Kerzel & Bekkering, in press; Sturmer, Ascher-sleben, & Prinz, in press; Brass, Bekkering, & Prinz, in press). However, thesestudies did not compare compatibility effects for different types of stimuli.

The present study was designed to address this issue by directly comparingideomotor-compatible and more symbolic stimuli, using an interference par-adigm similar to Kornblum and Lee (1995). In this paradigm, three differentconditions are contrasted: baseline, congruent, and incongruent conditions.A baseline condition established the standard reaction time (RT) for a single,relevant, imperative stimulus dimension. In the Kornblum and Lee study,for example, the letter ‘‘J’’ appeared as the imperative stimulus between thetwo hands (e.g., the ‘‘J’’ indicated a key press of the right middle finger).In the congruent condition, the imperative stimulus (e.g., the letter ‘‘J’’) wasshown in a congruent relation to a response-irrelevant stimulus dimension(e.g., the ‘‘J’’ appeared on the fingernail of the right middle finger). In theincongruent condition, however, the imperative stimulus appeared at a differ-ent, incongruent location (e.g., ‘‘J’’ appeared on the left index finger). Theintroduction of a baseline condition enabled the evaluation of the irrelevantstimulus information’s function in the congruent trials (i.e., whether the irrel-evant stimulus information facilitated response initiation). It also enabled theevaluation of the irrelevant dimension’s function in the incongruent trials(i.e., whether the irrelevant stimulus dimension interferes with response initi-ation). The exchange of the relevant and the irrelevant stimulus dimensionsmakes it possible to directly compare the influence of both stimulus dimen-sions.

Three experiments were conducted using this paradigm in order to com-pare ideomotor-compatible with symbolic stimuli. In Experiment 1, highlyideomotor-compatible videotaped finger movements (the lifting of the indexor the middle finger) were contrasted with symbolic cues (the number ‘‘1’’or ‘‘2’’). Experiment 2 contrasted the videotaped finger movements withspatial finger cues (an ‘‘x’’ on the fingernail of the index or middle finger).In Experiment 3, the same stimuli as in Experiment 2 were presented; butresponses (tapping with the index or middle finger) were no longerideomotor-compatible to the observed finger movement (lifting of the indexor middle finger).

EXPERIMENT 1

Two types of stimulus dimensions were presented in the first experiment:finger movements (lifting movement of index or middle finger) and symbolic

128 BRASS ET AL.

cues (the digits 1 and 2). Participants had to execute a finger movement inresponse to either the finger movements or the symbolic cues (1 required anindex-finger movement, whereas 2 required a middle finger movement). Asin a Stroop and Simon task, a relevant dimension was compared with anirrelevant dimension. Expressed in terms of the ideomotor theory, switchingfrom the symbolic-cues to the finger movements as the relevant dimensionshould lead to a RT advantage. In addition, in the symbolic cue condition,the irrelevant congruent movement is expected to exert a facilitative influ-ence. That is, observing a congruent finger movement should activate thecongruent response and, hence, should facilitate response initiation. Observ-ing an incongruent finger movement should activate the incorrect responseand, therefore, should lead to an interference effect. On the other hand, thefinger movement condition was expected to be unaffected by the congruencyof the symbolic cues.

Method

Participants

Eight students of the University of Munich participated in a 30-min-long session. All wereright-handed, had normal or corrected-to-normal vision, and were naive with respect to thepurpose of the study. They were paid for their participation.

Apparatus and Material

A Compaq Pentium 200-MHz computer controlled stimulus presentation and data acquisi-tion. Short, digitized video sequences of an animated hand were presented on a 17-in. screenwith a frame frequency of 60 Hz. The video sequences consisted of five frames. The animatedhand had a visual angle of approximately 6.5° vertically and 10° horizontally. Participantswere seated approximately 80 cm from the screen. The animated hand was positioned as toappear as a mirror image of the participant’s hand, which rested on a table. Movements wererecorded by an Optotrak system (Norton Digital). Six cameras were installed to sample theraw position data of active infrared markers (with a sampling rate of 250 Hz) located on thefingernails of the subject’s index and middle fingers. The system allows for a reconstructionof a movement’s trajectory in three-dimensional space. Movement onsets were computedbased on these trajectories and analyzed off-line, using MATLAB 5.0 routines. For detectionof the movement onset, a relative velocity threshold was used (10% of peak velocity). Move-ment onset was defined as the moment in which the velocity exceeded this threshold and afterwhich a minimum displacement (10% of movement amplitude) took place within a maximumamount of time (100 ms).

Design and Procedure

The experiment was divided into two blocks: one block with finger movements as the rele-vant dimension (finger movement condition) and one block with symbolic cues as the relevantdimension (symbolic cue condition). The finger-movement condition required that the partici-pants execute an index-finger movement in response to an observed index-finger movementand a middle-finger movement in response to an observed middle-finger movement. Threestimulus events were randomly (with a maximum of three stimulus-event repetitions) presented


baseline

congruent incongruent

FIG. 1. One of the two possible stimulus events that appeared in baseline, congruent, andincongruent trials (only the last frame of the video sequence is displayed). The baseline isdifferent in the symbolic-cue condition (right picture) and the finger-movement condition (leftpicture). Stimuli for congruent and incongruent trials were identical for both conditions.

in this block (see Fig. 1). In the baseline trials, only a finger movement was presented. In thecongruent trials, a congruent number—‘‘1’’ in the case of an index-finger movement and ‘‘2’’in the case of a middle-finger movement—was displayed between the index and the middlefingers. In the incongruent trials, an incongruent number—‘‘1’’ in the case of a middle-fingermovement and ‘‘2’’ in the case of an index-finger movement—was displayed. In the symboliccue condition (see Fig. 1), participants were instructed to execute an index-finger movementin response to the presentation of the number 1 or a middle-finger movement in response tothe presentation of the number 2. In the baseline trials, only the numbers were presented. Inthe congruent trials, a congruent finger movement accompanied the number; in incongruenttrials, an incongruent finger movement accompanied the presented number.

The video sequences began with a frame showing the hand in a resting position. This initialframe, which was identical for all stimuli, remained visible for about 560 ms. Then, threeframes (each lasting approximately 34 ms) presented the finger movement, the numbers, orboth. Finally, a fifth frame was presented for approximately 500 ms. The overall displacementof the presented finger movement was about 2°. Between the trials, the screen turned bluefor 1840 ms; hence, the total trial length was 3 s. Both blocks contained 120 trials (40 baselinetrials, 40 congruent trials, and 40 incongruent trials). Block order was counterbalanced acrossparticipants. Participants were instructed to execute their movement as quickly as possible.Before each block, 10 test trials were presented. Test trials were excluded from further analysis.

130 BRASS ET AL.

Data Analysis

Errors. Incorrect movements were defined to be errors when the amplitude was found tobe larger than 1/3 of the instructed finger movement. The mean error rate was 2.8%.

Response Times. Errors were excluded from the RT analysis. An ANOVA was computedwith two within-subject factors, using type of instruction (finger movement condition vssymbolic-cue condition) as the first factor and congruency (baseline, congruent, and incongru-ent) as the second factor.

Results

Error analysis revealed no significant results. A robust main effect (91ms) was, however, found for the type of instruction, F(1, 7) 5 92.9, p ,.001. Participants generally reacted more quickly to the finger movementthen when they reacted to a symbolic cue (273 ms versus 364 ms). Anothermain effect was found, this time regarding congruency: F(2, 14) 5 27.3,p , .001. In other words, participants’ reactions in congruent trials (304 ms)were faster, t(7) 5 3.5, p , .05, than in baseline trials (318 ms). Participants’reactions were, in turn, faster in the baseline trials, t(7) 5 6.1, p , .001,than in the incongruent trials (333 ms). The most interesting effect was foundin the two-way interaction between type of instruction and congruency, F(2,14) 5 19.9, p , .001 (see Fig. 2).

Congruency affected reaction times in the symbolic-cue condition (con-

congruency

congruent baseline incongruent

mov

emen

t ons

et (

ms)

260

280

300

320

340

360

380

400

420

symbolic cue condition

finger movement condition

FIG. 2. Movement onset (RT) as a function of congruency (congruent, baseline, and in-congruent) and type of instruction (finger-movement condition and symbolic cue condition).In the symbolic-cue condition the observed finger movement was the irrelevant dimension,while in the finger-movement condition the symbolic cue was the irrelevant dimension.


gruent: 336 ms, baseline: 363 ms, incongruent: 392 ms) but not in the finger-movement condition (congruent: 272 ms, baseline: 274 ms, incongruent: 274ms). If subjects responded to the symbolic cue, the irrelevant-congruent fin-ger movement led to a significant facilitation effect of 27 ms, t(7) 5 3.93,p , .01. In addition, the irrelevant-incongruent finger movement led to asignificant interference effect of 29 ms, t(7) 5 4.83, p , .012.

Discussion

The main effect found for type of instruction indicated that responses weremore quickly initiated in the finger-movement condition than in the sym-bolic-cue condition. In addition, responses to the symbolic cues were facili-tated when the irrelevant finger movement was congruent. An interferenceeffect was found when the irrelevant finger movement was incongruent. Boththe facilitation and the interference effects provided evidence for a strongand automatic influence of observed finger movements on movement execu-tion. On the other hand, the presentation of the irrelevant symbolic cue didnot influence the response in the finger-movement condition.

Although these results are quite consistent with our hypothesis, similarresults already have been obtained for a comparison between symbolic cuessimilar to the ones used in Experiment 1 and spatial finger cues (e.g., Korn-blum & Lee, 1995). However, the ideomotor theory predicts that the ob-served finger movement should be more ideomotor-compatible to the re-sponse than a spatial finger cue. We tested this assumption by comparingthe finger-movement condition with a spatial finger-cue condition.

EXPERIMENT 2

In Experiment 2 the symbolic cues were replaced by spatial finger cues(see Fig. 3). The finger was cued by an ‘‘x’’ drawn on the fingernail ofeither the index or the middle finger. Again, both stimulus dimensions werecontrasted in an interference paradigm in which each dimension was relevantin one block and irrelevant in the other block. The ideomotor theory wouldpredict that the finger movement be more similar to the sensory feedbackof its required response and, hence, more ideomotor-compatible to the re-sponse than the spatial finger cue. Thus, the presentation of finger movementshould affect response initiation more than the presentation of a spatial fingercue. As a consequence, faster responses are expected in the case that thefinger movement is the relevant dimension than in the case that the spatialfinger cue is the relevant dimension. In addition, the theory predicts thatthe irrelevant finger movement should affect performance in the finger-cuecondition. In other words, congruent-irrelevant finger movements should fa-cilitate response initiation, whereas incongruent-irrelevant finger movementsshould slow down response initiation in the spatial finger cue condition.

132 BRASS ET AL.

baseline

congruent incongruent

FIG. 3. One of two possible stimulus events that appeared in baseline, congruent, andincongruent trials (only the last frame of the video sequence is displayed). The baseline inthe finger-movement condition was identical to Experiment 1 (left picture), while in the finger-cue condition it is a cross on the fingernail (right picture). Congruent and incongruent trialswere again identical in both conditions.

Method

Participants

Eight students of the University of Munich participated in a 30-min session. All participantswere right-handed, had normal or corrected to normal vision, and were naive with respect tothe purpose of the study. They were paid for their participation.


Stimulus presentation and measurement procedure was identical to that in Experiment 1.The finger cue consisted of a black ‘‘x’’ which was drawn on a fingernail of either the indexor the middle finger. In the baseline trials of the spatial finger-cue condition, the cross appearedon a fingernail, but the hand appeared motionless. In the congruent trials, the cross appearedon the moving finger, in the incongruent trials the cross appeared on the nonmoving finger.


The experiment was again divided into two blocks. In one block (spatial finger-cue condi-tion), participants were required to lift the finger equivalent to the one marked by the cross.


In the other block (finger-movement condition) participants had to respond to the observedfinger movement, as they did in Experiment 1. The temporal order of the digitized videosequences was identical to Experiment 1. In the second frame (the first frame in which move-ment was displayed), the cross appeared. As in Experiment 1, 10 test trials were performedbefore each block. Test trials were excluded from further analysis.

Data Analysis

Errors. The mean error rate was 2.18 %.Response times. Errors were excluded from further RT analysis. An ANOVA was computed

using the factors type of instruction (finger-movement condition vs spatial finger-cue condi-tion) and congruency (baseline, congruent, and incongruent).

Results

Error analysis revealed only a significant main effect for congruency, F(2,14) 5 8.20, p , .01, which was in the same direction as the RT effect,indicating no speed-accuracy trade-off. A main effect was also found for thetype of instruction, F(1, 7) 5 118.9, p , .001. When participants respondedto the finger movement they were faster than with the spatial finger-cue con-dition (277 ms versus 318 ms). In addition a main effect was found for con-gruency, F(2, 14) 5 46.1, p , .001. In other words, participants respondedmore quickly in congruent trials (276 ms), t(7) 5 2.6, p , .05 than in baselinetrials (283 ms). Participants’ reactions in baseline trials were, in turn, faster,t(7) 5 7.1, p , .001, than in incongruent trials (334 ms). Again, the interac-tion (Fig. 4) between the type of instruction and congruency was significant,F(2, 14) 5 36.1, p ,. 001.

If subjects were instructed to respond to the spatial finger cue, an irrelevantcongruent finger movement led to a significant facilitation effect of 22 ms,t(7) 5 4.2, p , .01. In addition, the irrelevant-incongruent finger movementled to a significant interference effect of 70 ms, t( 7) 5 7.1, p , .001. Ifparticipants were instructed to respond to the finger movement, a congruentspatial finger cue had no influence on response time, but an incongruentspatial finger cue led to an interference effect of 33 ms, t(7) 5 4.4, p , .01.

Discussion

The results of Experiment 2 demonstrated that the ideomotor-compatiblefinger movements dominated the less ideomotor-compatible spatial fingercues. Again, it was found that participants were faster in the finger-movementcondition than in the spatial finger cue condition. Responses to the spatialfinger cues were facilitated when the finger movement was congruent; aninterference effect was found when the finger movement was incongruent.Both the facilitation and the interference effects described above providedevidence for a strong influence of movement observation on movement exe-cution. It is interesting to note that an interference effect—but not a facilita-tive effect—was evident within the finger-movement condition when the

134 BRASS ET AL.

congruency


mov

emen

t ons

et (

ms)

240

260

280

300

320

340

360

380


spatial finger cue condition

FIG. 4. RT as a function of congruency (congruent, baseline, and incongruent) and typeof instruction (finger-movement condition, and spatial finger cue condition). In the finger-cuecondition the observed finger movement was the irrelevant dimension, while in the finger-movement condition the finger cue was the irrelevant dimension.

spatial finger cue was used as the irrelevant stimulus dimension. In otherwords, a congruent spatial finger cue provides no additional benefit for re-sponse initiation. An incongruent spatial finger cue, however, appears to neg-atively influence response initiation processes. These results are consistentwith predictions derived from the ideomotor theory. In particular, they indi-cate that the stimulus’ similarity to the sensory feedback from the responsedetermines the degree of compatibility.

Thus far, we have assumed that observed finger movements’ strong influ-ence on movement execution could be attributed to the similarity of observedand executed actions (i.e., the degree of ideomotor compatibility). This as-sumption was, however, not explicitly tested in Experiments 1 and 2. Sincethe executed action was always kept constant, one might as well explain thefindings with a pure perceptual account. Such a perceptual account wouldstate that the observed finger movement is processed faster than the symbolicand spatial cues. In other words, not the relationship between observed andexecuted actions was causing the effects found, but the results of Experi-ments 1 and 2 can also be described at a perceptual level only. In Experiment3 these two predictions are directly tested against each other. In order to doso, the stimuli used in Experiment 3 were identical to the ones used in Experi-ment 2. However, the finger movement required did not match the fingermovement observed. If the similarity between observed and executed finger


movements underlie the effects observed so far, a substantial change in thepattern of results should occur. However, if the effects were caused by aperceptual difference only, similar results as in Experiment 2 should occur.

EXPERIMENT 3

In Experiment 3, the exact same stimuli were used as in Experiment 2(lifting finger movements and an ‘‘x’’ that appeared on the fingernail). Theparticipants’ required response to these stimuli was, however, different. Incontrast to the first two experiments, in which participants were required toraise their index or middle fingers, respectively, participants now had to initi-ate a tapping movement with the index or the middle finger. This manipula-tion reduced the degree of ideomotor compatibility with the observed fingermovement and therefore a substantial change was expected in the pattern ofresults. The facilitation effect—which had been caused by the ideomotor-compatible irrelevant movement—should disappear. Furthermore, the inter-ference effect should decrease, and the RT difference of the finger-movementcondition and the spatial finger-cue condition should decrease. The percep-tual interpretation, on the other hand, predicts no substantial change in thepattern of results.

Method

Participants

Eight students of the University of Munich participated in a 30-min session. All were right-handed, had normal or corrected-to-normal vision, and were naive with respect to the purposeof the study. They were paid for their participation.


The stimulus presentation and measurement procedure was identical to Experiment 2. How-ever, participants now were instructed to hold their index and middle fingers a few centimetersabove the surface of the table and to rest their ring and the little fingers on the table.


The experiment was divided into two blocks. In one block (spatial finger-cue condition)the instruction was to watch for the finger with an ‘‘x’’ and to tap on the table with this finger.In the other block (finger-movement condition), subjects were instructed to tap on the tablewith the finger that was equivalent to the lifted finger on the screen.

Data Analysis

Errors. The mean error rate was 2.5%.

Response times. Errors were excluded from further analysis. An ANOVA with two within-subject factors was computed. The first factor was type of instruction (finger movement condi-tion vs the spatial finger-cue condition) and the second factor was congruency (baseline, con-

136 BRASS ET AL.

gruent, and incongruent). In addition a between-experiment analysis was computed using datafrom Experiments 2 and 3.

Results

Error analysis revealed no significant results. In contrast to the results ofExperiments 1 and 2, no main effect was found regarding the type of instruc-tion, F(1, 7) 5 4.5, p 5 .07, although a small advantage for the finger-movement condition was still present (11 ms). However, the baseline condi-tions did not differ statistically, t(7) 5 .75, p , .45. The main effect forcongruency was once again significant, F(2, 14) 5 75.5, p , .001. Impor-tantly, responses in the baseline trials (282 ms) were no faster than responsesin the congruent trials (282 ms). But participants responses in both baselinetrials and congruent trials were faster, t(7) 5 8.9, p , .001, than their re-sponses in the incongruent trials (313 ms). Again, the two-way interaction(Fig. 5) between type of instruction and congruency was significant, F(2,14) 5 9.7, p , .01. In the spatial finger-cue condition, an incongruent fingermovement led an interference effect of 43 ms in comparison with baselineperformance, t(7) 5 10.4, p , .001. No facilitation effect was found forthe observed finger movement, t(7) 5 .5, p 5 .63. In the finger-movementcondition, a congruent finger cue had no significant influence on responsetime. An incongruent spatial finger cue, however, led to an interference effectof 18 ms, t(7) 5 3.7, p , .01.

congruency


mov

emen

t ons

et (

ms)

240

260

280

300

320

340

360

380


spatial finger cue condition

FIG. 5. RT as a function of congruency (congruent, baseline, and incongruent) and typeof instruction (finger-movement condition, spatial finger-cue condition). Stimuli were identicalto Experiment 2, the participants’ required response to these stimuli was, however, different(tapping with the index or middle finger).


In order to compare the effects of Experiments 2 and 3, both experimentswere combined in one ANOVA. In addition to the factors type of instructionand congruency the factor response mapping (ideomotor-compatible vs notideomotor-compatible ) was added. All within-subject effects were signifi-cant: type of instruction, F(1, 14) 5 49.0, p , .001, congruency, F(2, 28) 593.9, p , 001; and type of instruction 3 congruency, F(2, 28) 5 40.2, p, .001. The interaction between type of instruction and response mappingwas also significant, F(1, 14) 5 9.0, p , .01, which indicated that the maineffect for type of instruction decreased when response mapping was not ideo-motor-compatible. The interaction between congruency and response map-ping was also found to be significant, F(2, 28) 5 7.7, p , .01, as was thethree-way interaction between type of instruction, congruency, and responsemapping, F(2, 28) 5 12.1, p , .001. In addition, the interference effect ofthe incongruent finger movement decreased significantly (70 ms vs 43 ms),t(14) 5 2.5, p , .05.

Discussion

Experiment 3 showed that reducing ideomotor compatibility by changingthe relation of observed and executed action influenced the pattern of results.This became obvious in Experiment 3 as well as in a between-experimentcomparison with Experiment 2. Importantly, the interaction of experimentand type of instruction indicates that the advantage of movement observationdecreased significantly when the amount of ideomotor compatibility was re-duced. In addition, the advantage of observed finger movements was largelyreduced in Experiment 3 and even disappeared completely in the baselineconditions. This result indicated that a significant portion of the reaction-time advantage for finger movement compared to spatial finger cues could beattributed to the similarity of stimulus and response and not to the perceptualadvantage of finger movements compared to spatial finger cues. Observingan irrelevant finger movement had no facilitative effect in the finger-cuecondition and the interference effect of the incongruent finger movementwas substantially reduced compared to Experiment 2. Altogether, the dataof Experiment 3 showed that the match of observed and executed action wascritical for the strong influence of observed finger movement. This findingis in contradiction to the perceptual interpretation, which predicted no sub-stantial changes in the pattern of results.

GENERAL DISCUSSION

The aim of this study was to directly compare stimuli of different degreesof ideomotor compatibility. In the first experiment, it was found that partici-pants responded more quickly to finger movements than to symbolic cues.In addition, responses to the symbolic cues were facilitated when the irrele-vant finger movement was congruent and an interference effect was found

138 BRASS ET AL.

when the irrelevant finger movement was incongruent. Both the facilitationand the interference effects of the irrelevant finger movement describedabove provided evidence for an automatic influence of observed finger move-ments on movement execution.

The second experiment compared observed finger movements with spatialfinger cues. The results of Experiment 2 demonstrated that the finger move-ment had a stronger influence on movement execution than did the spatialfinger cue. Again, if the observed finger movement was the relevant stimulusdimension, participants reacted faster than if the spatial finger cue was therelevant dimension. In addition, responses to the finger cues were facilitatedwhen the irrelevant finger movement was congruent and an interference ef-fect was found when the irrelevant finger movement was incongruent. In thefinger-movement condition, however, an interference effect of the incongru-ent spatial finger cue was found. This indicated that a spatial finger cue alsoinfluenced the response. By reducing the similarity of observed and executedactions in Experiment 3, it was demonstrated that the strong influence ofobserved finger movement on movement execution was related to ideomotorcompatibility, which contradicts a perceptual interpretation of the results.

Ideomotor Compatibility for Visual Feedback from a Response

Thus far, ideomotor compatibility has been investigated, primarily withregard to sensory feedback that was not directly related to the movement. Forexample, researchers used auditorily presented words and verbal responses(Greenwald, 1970b; Morrin, Konick, Troxell, & McPherson, 1965; Davis,Moray, & Treisman, 1961) or printed digits and handwriting (Greenwald,1970b). Studies that investigated the compatibility of visual feedback from aresponse (Kerzel & Bekkering, 2000; Sturmer, Aschersleben & Prinz, 1999;Brass, Bekkering, & Prinz, in press) did not directly compare stimuli of dif-ferent degrees of ideomotor compatibility.

By using an interference paradigm, we demonstrated that stimuli with ahigh degree of ideomotor compatibility had a stronger influence on the re-sponse than less ideomotor-compatible stimuli. From the perspective of theideomotor theory, a symbolic cue (like the numbers in Experiment 1) doesnot share any properties with the sensory feedback of the response. Hence, noactivation of the response image is expected. A spatial finger cue, however,specifies the response position and hence should influence response initiationto some extent. Nevertheless, the observed finger movements were muchmore similar to the sensory feedback of the response than the spatial fingercues.

Imitation and Automatic Response Activation

Movement observation was found to exert a strong influence on movementexecution, both being the relevant stimulus dimension as well as being the


irrelevant stimulus dimension. When finger movement was the relevant stim-ulus dimension, the task was an imitative one in the sense that one does whatone sees. The results revealed that the finger movement condition led tofaster responses than did the nonimitative instructions (symbolic cue andspatial finger cue). The suggestion made at the beginning of this article, thatimitating a movement is an easier task than responding to an abstract instruc-tion, seems to be confirmed by the present results, at least at a very basicmotor control level.

But the influence of observed finger movement on movement executionwas not restricted to imitation of the finger movement. Even when the fingermovement was the irrelevant dimension it strongly affected movement exe-cution. This finding supports the idea that movement observation exerts anautomatic influence on movement execution. If the observed finger move-ment is congruent to the proper response, this response is activated. If thefinger movement is incongruent the incorrect response alternative is activatedand must be inhibited. Although automatic response activation has also beendemonstrated for spatial finger cueing (e.g., Kornblum & Lee, 1995), theresults of Experiments 2 and 3 indicate that the strength of the activationprocesses depends on the degree of its ideomotor compatibility.

Recently, Craighero et al. (in press) proposed a motor-visual primingmechansim which may serve as an alternative account for our results. Theyfound that the preparation of a specific response facilitates the detection anddiscrimination of objects that have an orientation which is congruent withthe prepared response. Such a facilitation effect was also found when sub-jects finally needed to respond with different effectors, indicating that theimproved object detection was due to the prepared congruent response andnot due to a priming of the executed response by the visual object, as waspreviously demonstrated by Craighero et al. (1996, 1998). A motor-visualpriming mechanism, however, is unlikely to be able to explain fully the RTpatterns of the present experiments. First of all, in the present experiments,subjects were not able to prepare only one response in advance. Second,even when motor-visual priming for both possible finger movements exists,how would this influence be able to explain the effects found for finger move-ments on the irrelevant dimension?

Another interesting aspect that emphasizes the special role of ideomotorcompatibility is the fact that the additional information provided by the ob-served finger movement was not relevant for the selection of the correctresponse. That is, in order to select the correct response, participants onlyhad to specify the finger to be moved. The type of movement (lifting ortapping) was preinstructed and remained constant within each experiment. Inother words, the observed finger movement was not informative for responseselection. For models that explain SRC effects with a controlled S-R transfor-mation operation and that cast doubt any on kind of automatic response acti-vation (e.g., Hasbroucq, Guiard, & Ottomani, 1990; Hasbroucq & Guiard,

140 BRASS ET AL.

1991), this finding is difficult to explain. Ideomotor theory, however, hasno difficulties explaining such a finding. It assumes that observing a fingermovement preactivates a response image of the corresponding response. Thisactivation process influences response initiation even if the response is prede-fined. Therefore, observing a finger movement should affect movement exe-cution even in a task containing no response uncertainty. In a recent study,Brass, Bekkering, and Prinz (in press) explicitly investigated this point usinga simple response task. Indeed, they found a substantial compatibility effectin a task in which participants always had to execute the same response.Their findings clearly support the view that highly ideomotor-compatiblestimuli influence response initiation independent of response-selection pro-cesses.

Ideomotor Compatibility and Recent Theoretical Developments

Ideomotor theory seems capable of explaining the results of our experi-ments. The results are, however, also consistent with more recent theoriesof compatibility, such as the model of dimensional overlap (Kornblum, Has-broucq & Osman, 1990). This model states that ‘‘. . . SRC is the directconsequence of the degree to which the stimulus and the response sets of astimulus–response (S-R) ensemble are perceptually, conceptually, or struc-turally similar’’ (p. 855, Kornblum & Lee, 1995). The notion of perceptualsimilarity is very similar to the notion of ideomotor compatibility. One’sperception of a body movement is similar to what one perceives when exe-cuting that movement. The theoretical advantage of the ideomotor theory isthat it provides a functional mechanism of how the similarity between stimu-lus and response can influence response initiation by emphasizing the roleof the response image in control of voluntary action.

This notion of ideomotor compatibility fits well in the more general frame-work of the relationship between perception and action as elaborated recentlyin the Theory of Event Coding (Hommel, Musseler, Aschersleben, & Prinz,submitted). This theory states that ‘‘. . . perceptual contents and action goalsare cognitively represented by distributed, yet integrated codes of their distalfeatures, that is, perceived and to-be-produced events are coded within acommon representational medium’’ (p. 3). Because an observed actionshares a lot of features with its distal representation, the perception–actionlink is very close for observed and executed actions. This view is also sup-ported by recent neurocognitive findings.

Neurocognitive Support for the Theory of Ideomotor Action

As described in the Introduction neuropsychological research supports theidea that a specific mechanism is involved in imitative action. Recently, someneurophysiological and brain-imaging studies also support this view. Forexample, so called ‘‘mirror neurons’’ were found in monkeys’ premotor cor-


tex (di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992). These neu-rons are active both during specific goal-directed behavior and during theobservation of the same behavior. This finding was interpreted as a neuralcorrelate of an observation/execution matching system. Such a system isthought to directly link performed actions with the recognition of observedactions. It has been argued that an action observation/execution matchingsystem should also exist in human primates (Gallese, Fadiga, Fogassi, &Rizzolatti, 1996). For instance, Fadiga, Fogassi, Pavesi, and Rizzolatti (1995)showed that the observation of goal-directed arm movements is followed byselective EMG activation if the primary motor cortex is concurrently stimu-lated by transcranial magnetic stimulation. It was concluded that movementobservation leads to an activation of response-relevant cortical structures.This view is also supported by a number of recent brain-imaging studies(Decety et al., 1994, 1997; Rizzolatti et al., 1996; Krams et al., 1998).

Even if it is difficult to transfer these neurocognitive findings to the behav-ioral level, the converging evidence of behavioral, neuropsychological, neu-rophysiological, and brain-imaging data support the assumption that there isa close relationship of action observation and action execution as proposedby ideomotor theory.

REFERENCES

Alexander, M. P., Baker, E., Naeser, M. A., Kaplan, E., & Palumbo, C. (1992). Neuropsycho-logical and neuroanatomical dimensions of ideomotor apraxia. Brain, 115, 87–107.

Brass, M., Bekkering, H., & Prinz, W. (in press). Movement observation affects movementexecution in a simple response task. Acta Psychologica.

Craighero, L., Fadiga, L., Rizzolatti, G., & Umilta, C. (1998). Visuomotor priming. VisualCognition, 5(1-2), 109–125.

Craighero, L., Fadiga, L., Rizzolatti, G., & Umilta, C. (in press). Action for perception: Amotor-visual attentional effect. Journal of Experimental Psychology: Human Perceptionand Performance.

Craighero, L., Fadiga, L., Umilta, C. A., & Rizzolatti, G. (1996). Evidence for visuomotorpriming effect. Neuroreport: An International Journal for the Rapid Communication ofResearch in Neuroscience, 8(1), 347–349.

Davis, R., Moray, N., & Treisman, A. (1961). Imitating responses and the rate of gain ofinformation. Quarterly Journal of Experimental Psychology, 13, 78–89.

De Renzi, E., Motti, F., & Nichelli, P. (1980). Imitating gestures: A quantitative approach toideomotor apraxia. Archives of Neurology, 37, 6–10.

Decety, J., Grezes, J., Costes, N., Perani, D., Jeannerod, M., Procyk, E., Grassi, F., & Fazio,F. (1997). Brain activity during observation of actions: Influence of action content andsubject’s strategy. Brain, 120(10), 1763–1777.

Decety, J., Perani, D., Jeannerod, M., Bettinardi, V., Tadary, B., Woods, R., Mazziotta,J. C., & Fazio, F. (1994). Mapping motor representations with positron emission tomo-graphy. Nature, 371, 600–602.

di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992). Understandingmotor events: A neurophysiological study. Experimental Brain Research, 91, 176–180.

142 BRASS ET AL.

Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. (1995). Motor facilitation during actionobservation: A magnetic stimulation study. Journal of Neurophysiology, 73(6), 2608–2611.

Fitts, P. M., & Deininger, R. L. (1954). S-R compatibility: Correspondence among pairedelements within stimulus and response codes. Journal of Experimental Psychology, 48,483–492.

Fitts, P. M., & Seeger, C. M. (1953). S-R compatibility: Spatial characteristics of stimulusand response codes. Journal of Experimental Psychology, 46, 199–210.

Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotorcortex. Brain, 119, 593–609.

Gibson, J. J. (Ed.). (1986). The ecological approach to visual perception. Hillsdale, NJ: Erl-baum.

Greenwald, A. G. (1970a). Sensory feedback mechanisms in performance control: With specialreference to the ideo-motor mechanism. Psychological Review, 77, 73–99.

Greenwald, A. G. (1970b). A choice reaction time test of ideomotor theory. Journal of Experi-mental Psychology, 80(1), 20–25.

Hasbroucq, T., Guiard, Y., & Ottomani, A. (1990). Principles of response determination: Thelist-rule model of SR compatibility. Bulletin of the Psychonomic Society, 28(4), 327–330.

Hasbroucq, T., & Guiard, Y. (1991). Stimulus response compatibility and the Simon effect:Toward a conceptual clarification. Journal of Experimental Psychology: Human Percep-tion and Performance, 17(1), 246–266.

Heilman, K. M., & Rothi, L. J. G. (1985). Apraxia. In K. M. Heilman & E. Valenstein (Eds.),Clinical neuropsychology . New York: Oxford Univ. Press.

Hommel, B., & Prinz, W. (1997). Theoretical issues in stimulus-response compatibility: Edi-tors’ Introduction. In B. Hommel & W. Prinz (Eds.), Theoretical issues in stimulus–response compatibility. Amsterdam: Elservier. Pp. 3–11.

Hommel, B., Musseler, J., Aschersleben, G., & Prinz, W. (2000). The theory of event coding(TEC). Submitted for publication.

Jeannerod, M. (1994). The representing brain: Neural correlates of motor intention and imag-ery. Behavioral & Brain Sciences, 17(2), 187–245.

Kerzel, D., & Bekkering, H. (1999). Motor activation from visible speech: Evidence fromstimulus response compatibility. Journal of Experimental Psychology: Human Perceptionand Performance, 26(2), 634–648.

Kornblum, S., & Lee, J.-W. (1995). Stimulus-response compatibility with relevant and irrele-vant stimulus dimensions that do and do not overlap with the response. Journal of Experi-mental Psychology: Human Perception & Performance, 21(4), 855–875.

Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis forstimulus–response compatibility: A model and taxonomy. Psychological Review, 97(2),253–270.

Krams, M., Rushworth, M. F. S., Deiber, M.-P., Frackowiak, R. S. J., & Passingham, R. E.(1998). The preparation, execution and supression of copied movements in the humanbrain. Experimental Brain Research, 120, 386–398.

Lhermitte, F., Pillon, B., & Serdaru, M. D. (1986). Human autonomy and the frontal lobes.Part I. Imitation and utilization behavior: A neuropsychological study of 75 patients.Annals of Neurology, 19, 335–343.

Mehler, M. F. (1987). Visuo-imitative apraxia. Neurology, 37, 129.Morrin, R. E., Konick, A., Troxell, N., & McPherson, S. (1965). Information and reaction

time for ‘‘naming’’ responses. Journal of Experimental Psychology, 70, 309–314.


Ochipa, C., Rothi, L. J. G., & Heilman, K. M. (1994). Conduction apraxia. Journal of Neurol-ogy, Neurosurgery & Psychiatry, 57(10), 1241–1244.

Prinz, W. (1987). Ideo-motor action. In H. Heuer & A. F. Sanders (Eds.), Perspectives onperception and action. Hillsdale: Lawrence Erlbaum. Pp. 47–76.

Prinz, W. (1990). A common coding approach to perception and action. In O. Neumann &W. Prinz (Eds.), Relationships between perception and action. Berlin: Springer-Verlag.

Rizzolatti, G., Fadiga, L., Matelli, M., Bettinardi, V., Paulesu, E., Perani, D., & Fazio, F.(1996). Localization of grasp representations in humans by PET. Experimental BrainResearch, 111, 246–252.

Rothi, L. J., Ochipa, C., & Heilman, K. M. (1991). A cognitive neuropsychological model oflimb praxis. Cognitive Neuropsychology, 8(6), 443–458.

Roy, E. A., & Hall, C. (1992). Limb apraxia: A process approach. In L. Proteau & D. Elliott(Eds.), Advances in psychology, Vol. 85: Vision and motor control. Amsterdam: North-Holland. Pp. 261–282.

Schnider, A., Hanlon, R. E., Alexander, D. N., & Benson, D. F. (1997). Ideomotor apraxia:Behavioral dimensions and neuroanatomical basis. Brain and Language, 58(1), 125–136.

Sturmer, B., Aschersleben, G., & Prinz, W. (in press). Effects of correspondence betweencomplex stimulus and response patterns. Journal of Experimental Psychology: HumanPerception and Performance.

Tucker, M., & Ellis, R. (1998). On the relations between seen objects and components ofpotential actions. Journal of Experimental Psychology: Human Perception & Perfor-mance, 24(3), 830–846.

Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and...

Documents

Transcript of Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and...