B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

Ava i l ab l e on l i ne a t www.sc i enced i r ec t . com

www.e l sev i e r . com/ loca te /b ra i n res

Research Report

Word and deed: A computational model ofinstruction following

Anand Ramamoorthy⁎, Tom VergutsDepartment of Experimental Psychology, Ghent University, Belgium

A R T I C L E I N F O

⁎ Corresponding author at: H. Dunantlaan 2, 9E-mail address: Anand.Ramamoorthy@ug

0006-8993/$ – see front matter © 2011 Elseviedoi:10.1016/j.brainres.2011.12.025

A B S T R A C T

Article history:Accepted 12 December 2011Available online 20 December 2011

Instructions are an inextricable, yet poorly understood aspect of modern human life. In thispaper we propose that instruction implementation and following can be understood as fastHebbian learning in prefrontal cortex, which trains slower pathways (e.g., cortical–basalganglia pathways). We present a computational model of instruction following that isused to simulate key behavioral and neuroimaging data on instruction following. Wediscuss the relationship between our model and other models of instruction following,the predictions derived from it, and directions for future investigation.

© 2011 Elsevier B.V. All rights reserved.

Keywords:Hebbian learningPrefrontal cortexBasal gangliaInstruction

1. Introduction

There are many types of learning. Human beings can learnthrough trial-and-error by interacting with the environment(Thorndike, 1911). This, however, is costly, time-consuming,and dangerous. Consider for example learning the conse-quences of touching fire or eating a strange berry, by trying itout. Learning by observation can circumvent the costs associatedwith trial-and-error (Bandura, 1977). With the advent of lan-guage, yet a third option became available. Verbal informationsharing promotes group cohesion while facilitating learning atreduced temporal cost. Understandably, learning from instruc-tions (often verbal) became an integral part of the learning reper-toire of the human brain.

From to do lists to softwaremanuals, instructions influencethe lives of modern humans on multiple levels. Yet, how theyare understood and implemented by the brain remains a mys-tery (Monsell, 1996). On the empirical side, a few recent studieshave explored the effects of instructions on performance. Afew themes emerge when the literature is surveyed. First, in-structions can be rapidly and accurately implemented without

000 Ghent, Belgium. Fax:ent.be (A. Ramamoorthy

r B.V. All rights reserved

explicit training (Ruge and Wolfensteller, 2010). Second, evenverbally instructed mappings that have never been applied,can interferewithwell-appliedmappings if the two sets ofmap-pings share stimulus dimensions (Cohen-Kdoshay and Meiran,2009; De Houwer et al., 2005; Waszak et al., 2008). A few recentstudies have also explored the neural correlates of instructionfollowing (Brass et al., 2009; Cole et al., 2010; Hartstra et al.,2011; Ruge and Wolfensteller, 2010). Third, dissociations mayarise between instructions and their implementation: In partic-ular, patients exhibiting goal neglect can verbally report the re-quired instructions without being able to implement them(Duncan et al., 1996; Luria, 1966).

On the theoretical front, instructions are crucial yet unex-plained components in theories of cognition. For example,models of cognitive control typically incorporate task represen-tations which implement the demands imposed by the task athand (Botvinick et al., 2001; Cohen et al., 1990; Verguts andNotebaert, 2008). However, investigations of instruction follow-ing itself are less common. One exception is the computationalmodeling work by Noelle and Cottrell (1995). More recently,Helie and Ashby (2009), Helie et al. (2010) proposed a

+32 92646496.).

.

55B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

computational account of learning that beginswith explicit rulesand ends with procedural knowledge. However, they do not ex-plore the acquisition of the rule itself. Doll et al. (2009) recentlyproposed a model of instruction control of reinforcement learn-ing. In their model prefrontal cortex (PFC) projects directly tomotor cortex and the basal ganglia (BG) to select responses con-sistent with the instructions.

Here, we follow up on the Doll et al. (2009) approach and con-sider instruction learning and implementation as instantiationsof Hebbian learning. For this purpose, we combine two comple-mentary models of learning and automatisation, namely SPEEDand COVIS. In SPEED (Ashby et al., 2007), learning occurs initiallyin the basal ganglia and is eventually transferred to cortex withan attendant increase in automaticity. In the COVIS framework(Ashby et al., 1998, 2011), performance is governed by two sys-tems — a rule-based one (dependent on prefrontal cortex) and aprocedural one (dependent on BG). We propose that a more gen-eral model that combines the main features of both SPEED andCOVIS is suited to explain various forms of learning, including in-struction following. In this framework,when instructionsarepro-vided, the prefrontal cortex learns them quickly, but executesthem slowly (Boureau and Dayan, 2010; Daw et al., 2005). Indeed,novel learning typically activates prefrontal cortex (e.g., Miller &Cohen, 2001; Toni et al., 2001). Upon repeated application, theBG (which learn more slowly but execute more quickly) pick upthe appropriate stimulus–response mapping by Hebbian learn-ing, where the appropriate response is provided by the prefrontalroute. Finally, after extensive application another cortical path-way would take over (hyperdirect pathway; Ashby et al., 2007).

With this general framework, we describe and test a modelthat focuses on the acquisition and transfer of instructedmappings. In the next section we describe the model and dis-cuss its biological plausibility. This is followed by the simula-tion studies. Theoretical considerations and empiricalpredictions are elaborated in the General discussion.

Fig. 1 – A schematic representation of the model. Weights from othe two layers. Plastic weights are represented by dotted arrows

1.1. The model

1.1.1. ArchitectureThe first route in the model is indirect (left part of Fig. 1); here,new instructions (such as “if you see a hexagon, press the leftarrow key” and “if you see a square, press the right arrow key”)can be rapidly learnt. The second one is the direct route (rightpart of Fig. 1); it gradually picks up the regularities implemen-ted by the indirect route.

In the indirect route, the instruction is represented interms of its components. One component contains stimulusrepresentations (e.g., “hexagon”) and the other, response rep-resentations (e.g., “press right key”). These two componentsare typically (but not necessarily) verbal, encoded by two dis-tinct layers (see Fig. 1) and related to sensory and motorareas via long-termmemory (temporal lobe (TL) and premotorcortex, respectively; see Fig. 1). By premotor cortex we refer tothe human analog of the dorsal premotor cortex (PMd) inmonkeys, which is a region associated with abstract motorplanning (Nakayama et al., 2008).

In the case of verbal instructions, stimuli and responsesare connected to their verbal equivalents (e.g., “hexagon”,“left key”). It is reasonable to assume that a tight associationbetween an object or attribute and its verbal analog comes tobe encoded during development (Fischer and Zwaan, 2008).Also, action verbs evoke activation of motor representations(Fischer and Zwaan, 2008; Hauk et al., 2008). The two compo-nents of the indirect route are linked together by PFC; in par-ticular, PFC subregion Inferior Frontal Junction (IFJ) appearsto be a candidate for this role given that it is active in circum-stances that require task-set switching or the loading ofnovel task sets (Derrfuss et al., 2005; Derrfuss et al., 2009),both of which require flexible verbal mapping. In the model,associating stimulus and response representations isachieved by fast Hebbian learning during the instruction

ne layer to another are represented by an arrow connecting. PFC = Prefrontal Cortex. TL = Temporal Lobe.

56 B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

phase. Neurally, the associative striatum is probably also partof the indirect path (Ashby et al., 2010), but we don't include ithere for simplicity.

The direct route includes, in addition to the stimulus andresponse areas, the basal ganglia. The circuitry of thecortico-striato-pallido-thalamo-cortical pathway (Mink, 1996)is approximated by a one-layer excitatory path. In particular,fronto-striatal loops are not included, and the direct pathcan be considered a simplified version of subcortical controlof action selection (e.g., Ashby et al., 2010; Dominey, 2005;Frank, 2005, 2006). The direct route gradually acquires stimulus–response associations by Hebbian learning, where the correctstimulus–response pairs are provided by the indirect route. Thehyperdirect cortical pathway mentioned in the introduction(e.g., Ashby et al., 2007) is not currently implemented.

It may be argued that the characterization of the indirectroute as one having more intermediate steps than the directone is contrary to the actual neural organization, where the BGhas more intermediate synapses (e.g., Mink, 1996). However,the layout is consistentwith the generally acknowledged findingthat the PFC route is a slowprocessing route (e.g.,Miller&Cohen,2001). More generally, the number of synapses between two pro-cessing layers may be an imperfect measure of processingspeed. Nevertheless, we report simulation studies that explorethe influence of varying the respective path lengths in the tworoutes.

1.1.2. DynamicsInformation flows along the directions indicated by arrows inFig. 1. In each trial, activation in the stimulus layer is clamped,i.e., set at a specific value as opposed to allowing the layer toreach the activation level over time. Stimuli are representedby localist coding in a vector with the element correspondingto the stimulus set to 1 and all other elements set to zero. Ac-tivation of other model units is described by standard differ-ence equations of the form (activation of input and outputunits denoted x and y, respectively):

yj tð Þ ¼ τyj t−1ð Þ þ 1−τð Þ∑ixiwij ð1Þ

where yj(t) is the activation of the jth output unit at time t inthe trial, xiwij is the net input from unit i to unit j, and τ is acascade rate parameter (set at 0.9). A response is chosen ifone of the response units reaches a threshold of 1. Reactiontimes (RTs) are calculated by counting the number of activa-tion cycles (within a trial) needed to reach this threshold.

We now describe how learning occurs in the model. Ini-tially, all weights are random values sampled from a uni-form distribution between 0 and 0.01. After response, acompetitive process selects the most active unit in the PFCand the most active unit in BG. Typically, the winner's acti-vation in a competitive model is a function of its original ac-tivation before the competition (e.g., Grossberg, 1973). As asimplified implementation of this process, we scaledthe winner's activation (stimulus j) after competition by12 1−e−Nrep jð Þ� �

, with Nrep (j) the repetition number of stimulusj in the trial at hand. In the instruction phase, learning oc-curs only in the connections between stimulus representa-tions, PFC and response representations. In the test phase,

learning occurs only in the BG. In both layers, learning fol-lows the rule given below from trial n – 1 to n:

wij nð Þ ¼ wij n−1ð Þ þ λ xi−dwij n−1ð Þ� �yj ð2Þ

where d is aweight decayparameter, set to 0.1. The termdwij(n−1)is subtracted fromthe input xi to constrain the learningprocess inthe direction of the input (as is typical in a Hebbian/competitivelearning algorithm, e.g., Fritzke, 1997).

The fact that only the PFC learned in the instruction phase,and the BG only in the test phase, implemented our assump-tion of fast learning in PFC. In their respective phases (instruc-tion and test, for PFC and BG respectively), the learning ratewas λ=9 for each layer. In the simulations, we also explorethe case when PFC learns in both instruction and test phases.

1.1.3. Testing the modelThe model provides a unified mechanism for rapidly imple-menting instructions as is required in a variety of contexts(e.g., cognitive control; Botvinick et al., 2001; Cohen et al.,1990). To provide points of contact with empirical data, the re-sults of two recent studies (Ruge and Wolfensteller, 2010;Waszak et al., 2008), dealing directlywith instructionswere sim-ulated. The first study (Ruge and Wolfensteller, 2010) describesthe transition from instructed to implemented mappings atboth behavioral and neural levels. To test different hypothesesregarding the architectural and parametric features of themodel, different versions of the model were built and testedusing the Ruge and Wolfensteller paradigm as detailed in themethods section (4.1.1, 4.1.2 and 4.1.3). The second (Waszak etal., 2008) describes a paradigm for investigating the influenceof merely instructed mappings on performance. This study de-scribes the basic interference and congruence effects reportedin other studies (Cohen-Kdoshay and Meiran, 2009; De Houweret al., 2005). Finally, a modified version of the Ruge andWolfen-steller instruction paradigm was used as an initial computa-tional exploration of goal neglect (Duncan et al., 1996).

1.2. Phenomena simulated

1.2.1. From instructed to pragmatic representations (Ruge &Wolfensteller, 2010)Instructions can be implementedwith ahighdegreeof accuracyon the very first trial (e.g., Cohen-Kdoshay and Meiran, 2009;Cole et al., 2010). With increasing practice, the mapping losesnovelty and becomes automatic as reflected, for example, inRT. Ruge and Wolfensteller (2010) studied the transition frominstructed to implemented stimulus–response mappings usingfMRI. They used a simple stimulus–response mapping task toidentify the neural correlates of mappings that had beeninstructed and subsequently applied. Each stimulus wasmapped onto one of two possible responses (press left or rightkey). Four stimuli (two for each key) and their mappings werefirst instructed (instruction phase). This was followed by 32practice trials inwhich each stimulus appeared 8 times in a ran-domized sequence (test phase). This procedure was repeatedover 20 blockswith new stimuli in each block to obtain accuratefMRI data.Within each block, responses became fasterwith rep-etition. Error rates also decreased with repetition. At the neural

Fig. 2 – Simulation 1.1. 2a, RT curve as a function of stimulus repetition (Ruge andWolfensteller, 2010). 2b, Model RT curve. Datapoints represent mean RT averaged across four stimuli and 16 subjects. Error bars indicate 95% confidence intervals. 2c, BOLDestimates for left-IFJ from Ruge and Wolfensteller (2010). 2d, activation levels in model PFC across repetitions. 2e, BOLDestimates for caudate nucleus from Ruge and Wolfensteller. 2f, activation levels in model basal ganglia across repetitions.

57B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

level, activation levels across repetitionsdecreased in the left IFJand increased in the BG (in particular, the caudate nucleus). Thereported changes in other areas (e.g., decrease in left posteriorintraparietal sulcus) are beyond the scope of the current studyand hence not discussed.

1.2.2. Crosstalk of instructed and applied arbitrary S-R map-pings (Waszak et al., 2008)Waszak et al. (2008) investigated the effect of merelyinstructed and applied visuomotor mappings. They usedstimuli varying on two dimensions (color and shape) and sub-jects were presented with color-task or shape-task trials inter-mixed. The tasks involved applying arbitrary mappings fromcolors and shapes to left and right responses (for instance,“if circle, then press left arrow key” or “if brown object, thenpress right arrow key”; see Fig. 6). A third of the stimulus–re-sponse associations in each task were merely instructed andthe other two-thirds were applied (i.e., trained). The irrelevantstimulus dimension allowed the stimulus in any given trial tobe categorized as univalent, bivalent, or instructed. In the caseof univalent stimuli, only the relevant stimulus dimensionhad a valid response mapping; for example, the stimuluswould be a shape in a particular color with only the shapebeing associated with a response. For bivalent stimuli, bothrelevant and irrelevant dimensions had valid response map-pings. The instructed stimuli were similar to the bivalentones, except that the irrelevant stimulus dimension and itsmapping were merely instructed and had never been applied.

The experiment consisted of an instruction phase, fol-lowed by five practice blocks of 96 trials each. The fourthand fifth practice blocks were preceded by two test blocks(each lasting for 36 trials) in which the instructed stimuliwere presented as valid targets. This rendered these stimulieffectively bivalent for the final two practice blocks.

Waszak et al. (2008) hypothesized that the presentation ofa stimulus with two dimensions having valid response map-pings (with one of the two mappings being valid for a secondtask) would lead to an interference effect. Interference effect re-fers to the delay in responding to bivalent or instructed stim-uli (congruent or incongruent) relative to univalent stimuli. Inaddition, the RT differences between incongruent and congru-ent stimuli for the bivalent and instructed types were alsocomputed (congruency effect).

Waszak et al. (2008) found an interference effect for bothbivalent and instructed stimuli, but it was larger for bivalentstimuli. Moreover, the interference effect was larger for thepractice blocks after the test blocks than before the test blocks(see Fig. 7a). They also found a congruency effect for bivalentstimuli across all practice blocks. In contrast, the instructedstimuli did not show a congruency effect in the first threepractice blocks but an effect was observed in the final twopractice blocks (see Fig. 7c).

1.2.3. Goal neglectGoal neglect refers to being able to describe an instructionwhile not being able to implement it. First reported in frontal

58 B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

lobe patients (Luria, 1966), Duncan et al. (1996) demonstratedthat goal neglect can also be observed in normal subjects if in-structions are sufficiently complex. In a recent study, Duncanet al. (2008) demonstrated that it is specifically the total numberof elements to be remembered in the task which determinesthe extent of goal neglect and its correlation with general intelli-gence. We hypothesize that the modeling framework presentedhere may yield insights into goal neglect. Currently, we applyan adapted version of the Ruge design as a first step towardmodeling goal neglect.

2. Results

2.1. From instructed to pragmatic representations

2.1.1. Simulation study 1.1The major behavioral findings of Ruge and Wolfensteller(2010) were replicated. First, mean error percentage was zero(in the empirical study, it was very low and decreased acrossrepetitions). Next, RTs decreased as a function of stimulusrepetition (empirical and simulated data in Figs. 2a and b,respectively).

In the empirical data, activation decreased in PFC acrossstimulus repetition (Fig. 2c). The same is observed in themodel (Fig. 2d). Also the activation increase across repetitionsin basal ganglia (caudate nucleus in Ruge and Wolfensteller,

Fig. 3 – Simulation 1.2. 3a, RT curve as a function of stimulus repepoints represent mean RT averaged across four stimuli and 16 suestimates for left-IFJ from Ruge and Wolfensteller (2010). 3d, actiestimates for caudate nucleus from Ruge and Wolfensteller. 3f, a

2010; Fig. 2e) was obtained in the model (Fig. 2f). The reasonwhy activation decreased in the PFC across trials is becausethe basal ganglia become faster with additional learning (i.e.,across trials), leaving less opportunity for the PFC to be highlyactive before response threshold is reached.

2.1.2. Simulation study 1.2: continuous learning in the PFCIn the current simulation, learning continued throughout thetask in PFC. Of the 50 simulations, 1 was excluded due to alarge number of errors (more than 25%).The analyses wereperformed with the remaining 49 simulations. The major be-havioral findings of Ruge and Wolfensteller (2010) were repli-cated. First, mean error percentage was zero (in theempirical study, it was low and decreased across repetitions).RTs decreased as a function of stimulus repetition (empiricaland simulated data in Figs. 3a and b, respectively).

Empirically, activation decreased in PFC across stimulusrepetition (Fig. 3c). The same trend appears in the model(Fig. 3d). Also the activation increase across repetitions in BG(caudate nucleus in Ruge and Wolfensteller, 2010; Fig. 3e)was obtained in the model (Fig. 3f).

2.1.3. Simulation study 1.3: model with equidistant pathsfrom stimulus to response in both routesIn this study the synapses from stimulus to response layersthrough both routes were made equal in number so that thetwo routes were of equal length. The major behavioral find-ings of Ruge and Wolfensteller (2010) were replicated. First,

tition (Ruge andWolfensteller, 2010). 3b, Model RT curve. Databjects. Error bars indicate 95% confidence intervals. 3c, BOLDvation levels in model PFC across repetitions. 3e, BOLDctivation levels in model basal ganglia across repetitions.

59B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

mean error percentage was very low and remained below0.01% (zero errors in the simulation). RTs decreased as a func-tion of stimulus repetition (empirical and simulated data inFigs. 4a and b, respectively).

As before, activation decreased in PFC across stimulus rep-etition (data, Fig. 4c; model, Fig. 4d). Again, as BG learn more,there is less opportunity for PFC to be as active as in the initialphase of responding, even though the paths are equidistant.Activation increases across repetitions in BG (caudate nucleusin Ruge and Wolfensteller, 2010, Fig. 4e; model, Fig. 4f).

Although the BG becomes increasingly active with learn-ing, a complete switch from PFC to BG does not occur in thiscase (see Fig. 4). For a complete switch to occur, it is requiredthat PFC learns earlier than BG, but also that PFC acts slowerthan BG. This is not the case with equidistant paths, inwhich case the two paths act at approximately the samespeed when both are well-trained.

2.1.4. Simulation study 1.4: model with equidistant pathsfrom stimulus to response in both routes, with PFC learningthroughout the simulationOf the 50 simulations, 2 were excluded due to a large numberof errors (25%). The analyses were performed with theremaining 48 simulations. Results were very similar to thoseobtained in study 1.3 (see Fig.5). Again, if the two paths actat approximately the same speed, there is no complete switchfrom PFC to BG.

Fig. 4 – Simulation 1.3. 4a, RT curve as a function of stimulus repetpoints represent mean RT averaged across four stimuli and 16 suestimates for left-IFJ from Ruge and Wolfensteller (2010). 4d, actiestimates for caudate nucleus from Ruge and Wolfensteller. 4f, a

2.2. Crosstalk of instructed and applied mappings

Of the 50 simulated subjects, 6 were excluded due to less than90% accuracy. The simulation results were very similar tothose of the original study (Waszak et al., 2008). First, therewas an interference effect for both bivalent and instructedstimuli, which increased with practice (Fig. 7b). Second, therewas a congruency effect for bivalent stimuli which remainedapproximately constant after practice. In contrast, for theinstructed stimuli, the congruency effect was initially verysmall for instructed stimuli (Fig. 7d) but became larger afterpractice. One discrepancy is that the Waszak et al. study didnot obtain a congruency effect for instructed stimuli whereasthe model did. However, the model is consistent with otherempirical studies that did find a congruence effect forinstructed stimuli (Cohen-Kdoshay and Meiran, 2009; DeHouwer et al., 2005).

2.3. Simulating goal neglect

Here, we applied the model to the important finding that in-struction understanding and following can be dissociated(i.e., goal neglect, Luria, 1966). For this purpose, the Ruge andWolfensteller (2010) paradigm was extended to include a ver-bal reporting task immediately after each S-R trial. We chosethis approach, rather than emulating the Duncan et al. designin great detail, because it allowed us to simulate the core

ition (Ruge andWolfensteller, 2010). 4b, Model RT curve. Databjects. Error bars indicate 95% confidence intervals. 4c, BOLDvation levels in model PFC across repetitions. 4e, BOLDctivation levels in model basal ganglia across repetitions.

Fig. 5 – Simulation 1.4. 5a, RT curve as a function of stimulus repetition (Ruge andWolfensteller, 2010). 5b, Model RT curve. Datapoints represent mean RT averaged across four stimuli and 16 subjects. Error bars indicate 95% confidence intervals. 5c, BOLDestimates for left-IFJ from Ruge and Wolfensteller (2010). 5d, activation levels in model PFC across repetitions. 5e, BOLDestimates for caudate nucleus from Ruge and Wolfensteller. 5f, activation levels in model BG across repetitions.

60 B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

phenomenon of goal neglect in the simplest way possible. Inline with typical behavioral paradigms, different responsedeadlines were used for the two tasks (S-R and verbal report),with the reporting task deadline much less strict. Percentageerror in the S-R task relative to the verbal task was used as ameasure of goal neglect (see Duncan et al., 2008) and

Fig. 6 – A representation of the task used in Waszak

evaluated across different PFC learning rates (λPFC ) and S-Rtask response deadlines (tmax).

A very low PFC learning rate (λPFC=1) in combination witha strict S-R task response deadline (tmax=20) resulted in agreater percentage of errors on the S-R task (30%), with respectto the verbal task (2.3%) (similar to goal neglect; Fig. 8a). This

et al. (2008), based on their original illustration.

Fig. 7 – Simulation 2. Interference effect before test blocks (BT) and after test blocks (AT) in data (7a) andmodel (7b). Congruencyeffect (BT and AT) in data (7c) and model (7d).

61B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

was ameliorated by increasing the S-R task response deadlinetmax from 20 to 40 (resulting in error rates of 8.3% and 1% forthe S-R and verbal tasks respectively) and from 40 to 60(resulting in errors rates of 0.5% and .3%, respectively) cycles(Figs. 8 d, g). In contrast, with moderately high PFC learningrates (λPFC=5, Figs. 8b, e, h), or with high PFC learning rate(λPFC=9, Figs. 8c, f, i) there was overall low error rate (and theerror rates were similar for the two tasks).

Fig. 8 – Simulation 3. Goal neglect. 3×3 plot of mean percentagerates (λPFC, different columns) and S-R task response deadlines (

3. General discussion

We presented a unified framework for instruction implemen-tation, with Hebbian learning at its core. We derived a dual-route model from this framework and applied it to instructionfollowing. The model was tested by simulating two recent ex-periments on instructions. The first simulation showed that

error in the S-R and verbal tasks across prefrontal learningtmax, different rows).

62 B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

sufficient practice can cause a switch from one route to an-other without need for a homunculus. However, a completeswitch only occurred if the faster-learning path also actedmore slowly. The second simulation study showed that merelyinstructed mappings can influence applied mappings (interfer-ence and congruency effects), but that they increase with prac-tice. Finally, we proposed that goal neglect can be interpreted inthis framework as emerging from a combination of an overalllow learning rate (perhaps due to brain damage or dual tasking)and a strict response deadline for the S-R task. Consistent withthis view, Czernochowski (2011) report evidence for impairedrule representations in older adults following short preparatoryintervals but not long ones.

Learning in the BG was implemented as Hebbian learning.In general, though, Hebbian learning may be modulated by re-ward. In particular, if Hebbian learning is coincident with re-ward (phasic dopamine), then the Hebbian learning processis more efficient (e.g., Reynolds et al., 2001). However, giventhe very high accuracy rates in the experimental paradigmsthat we modeled, and given that explicit reward is nevergiven, trial-to-trial variability in reward (phasic dopamine) isprobably small. Hence, these influences are captured by thelearning rate.

The model also yields a number of predictions for futureempirical work. One prediction issuing from the model isthat the connectivity between instruction-related areasshould change across repeated implementation of the in-struction. This prediction will be tested in future empiricalwork. Another prediction is that disruption to the PFC willabolish influences of merely instructed stimuli in the Waszaket al. paradigm. This can be tested using rTMS or workingmemory loading. In addition, when these merely instructedstimuli become practiced, the disrupting effects should disap-pear. Finally, Simulation 1 suggests that the relative speeds ofthe PFC and BG routes determine the extent to which the pro-cessing switches from the PFC route to the BG one. Althoughthis is difficult to test in humans, it may be testable usingsingle-unit recordings.

In the remainder, we first discuss the relation between ourapproach and earlier models. Finally, we place the model inthe broader framework of dual-route architectures and dis-cuss the related but ill-understood topic of suggestions.

3.1. Models of instruction following

Noelle and Cottrell (1995) described a connectionist model ofinstruction following based on the simple recurrent networkarchitecture (Elman, 1990; St.John and McClelland, 1990).Their approach to instruction following was activation-based,in the sense that a novel instruction was encoded as an acti-vation pattern (Botvinick and Plaut, 2006). For instance, an in-struction such as “if you see a hexagon, press the left key”would be encoded as a pattern of activation without changingthe connection weights in the system. To be able to do this forany novel (untrained) instruction, without any learning(weight changes) during the presentation of these instruc-tions, their model needed a long training phase with errorbackpropagation in which it acquired a set of instruction+stimulus-to-response mappings that could then be general-ized to novel mappings of this type. Learning in our model

is, by contrast, Hebbian, weight-based (cf. Botvinick and Plaut,2006) and it occurred in the instruction phase (in the prefron-tal connections) as well as in the practice phase (in the directroute).

In one sense, the long training phase of the Noelle et al.model corresponds to our hypothesized developmental acqui-sition of verbal and pragmatic associations. There are impor-tant differences, however. First, the Noelle et al. trainingregime used backpropagation, whereas ours can emergefrom the simpler and more biologically plausible Hebbianlearning. Second, the associative structure emerging fromthe developmental process in our model can be more broadlyrecruited by other parts of the cognitive system, because itsimply consists of associations between corresponding con-cepts (e.g., color red and word red).

As mentioned in the Introduction, Doll et al. (2009) alsopresented a model of instructional control based on thewell-known model of reinforcement learning in BG advancedby Frank (2005, 2006), which is implemented in a simplifiedform as the direct route in our model. The Doll et al. modelalso employed fast learning in PFC to implement instructions.It was applied to a probabilistic selection task in which a thirdof the instructions were incorrect with regard to the rewardprobabilities associated with a particular pair of stimuli.Their model was applied to a case of conflict between instruc-tions and contingencies. While it was not inconsistent withour own approach, we think that the current model adds sev-eral features of interest, in particular, a demonstration of howthe Hebbian framework can deal with different “instructionfollowing” phenomena described above; a theoretical accountof when and why switching should occur (complementarysystems are either fast-learning and slow-executing or viceversa); and an integration of the influential COVIS and SPEEDmodels.

3.2. Dual-route architectures

Dual-route architectures are cognitive systems where oneprocessing route is explicit, propositional, and effortful, andthe other is implicit, associative (contingency-based), and ef-fortless (e.g., Sloman, 1996). They have been claimed to under-lie diverse neurocognitive phenomena from reasoning(Sloman, 1996) to self-regulation (Carver et al., 2008). Ourmodel belongs to this general class with a propositional indi-rect route (of the form, “if …then…”) and a contingency-based direct route. Despite clear and important differencesbetween the two routes, we have proposed that a similarlearning mechanism may underlie both. This similarity be-tween explicit and implicit routes may have an evolutionarybasis, given that selection tends to recycle previously success-ful structures and processes (Dehaene and Cohen, 2007).Moreover, the modeling process demonstrated that sufficientpractice can cause a switch from one route to another withoutany need for a homunculus.

Additionally, the model yielded the prediction that shiftingfrom instructed to procedural knowledge consists of a gradualtransition of response selection from PFC to BG. If the modelwere to be extended to the development of automaticity (e.g.Ashby et al., 2007), the dual-route architecture would be mod-ified by the addition of a hyperdirect pathway which

63B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

associates stimuli with responses without a mediating region.In our framework, learning in such a hyperdirect route wouldbe even slower while it would be the fastest to execute thelearnt mapping. Empirical (fMRI) tests of this prediction willbe reported later on.

3.3. Instructions and verbally mediated phenomena

Suggestions are propositions used to influence an individual'sbeliefs and behavior. Examples include alterations of autobio-graphical memories (Mazzoni et al., 2001), eyewitness ac-counts (Frenda et al., 2011), modification of Stroop effects(Raz and Campbell, 2011), and the placebo effect (Benedettiet al., 2005). While several studies have aimed at uncoveringthe neural correlates of phenomena mediated by suggestion(Craggs et al., 2008; Petrovic et al., 2010), its nature remainspoorly understood.

We propose that suggestions can be understood from themodeling framework proposed here. A suggestion would berapidly learnt by the indirect route (specifically, the PFC) andapplied immediately. This suggestion can be either cooperativeor competitive with other processing routes (direct paths).Theindirect route is expected to influence regions associated withattentional and evaluative processing, such as the anterior cin-gulate cortex (e.g., Ploghaus et al., 2003) or ventromedial PFC(Hare et al., 2011). Consistently, recent studies have identified in-creased PFC activation in suggestion-mediated analgesic effects(Derbyshire et al., 2009). Hence, the core principles proposedhere can be used to generate a model to account forsuggestion-mediated phenomena. Currently, however, this re-mains an area for future investigation.

4. Experimental procedure

4.1. Simulation study 1.1: Ruge and Wolfensteller (2010)

There were four units in the stimulus layer, corresponding tothe four stimuli. Left and right responses were encoded bytwo units in the response layer. The same coding schemewas applied to the (verbal) stimulus representations and (ver-bal) response representations (indirect route), with the samenumber of units in the respective layers. The PFC and BGlayers had 200 units each.

The design of Ruge andWolfensteller (2010) was replicatedexactly, with 32 trials in the test phase. During the instructionphase, the stimuli and responses were presented as activationpatterns in the stimulus representation and response represen-tation layers (indirect route, left part of Fig. 1) and the associa-tion was acquired by the PFC (Eq. (2)). During the test phase,activation and learning obeyed Eqs. (1) and (2), respectively.PFC and BG activation vectors were collected before responsecompetition for analysis. Mean RTs andmeanerror percentageswere calculated across the 50 simulated subjects (as inRugeandWolfensteller, 2010).

4.1.1. Simulation study 1.2: continuous learning in the PFCModel parameters and settings were the same as in study 1.1,except that PFC was allowed to learn throughout the

experiment. Behavioral and activation data were collectedand analyzed as above.

4.1.2. Simulation study 1.3: model with equidistant pathsfrom stimulus to response in both routesThe intermediate layers in the indirect pathway (temporallobe and premotor cortex in the original model) and the verbalrepresentations (verbal stimulus representation and responserepresentation) were removed. As a consequence, the PFC andBGpathswere rendered equidistant. Otherwise,model parame-ters were the same as in study 1.1. Behavioral and activationdata were collected and analyzed as above.

4.1.3. Simulation study 1.4: model with equidistant pathsfrom stimulus to response in both routes and PFC learningthroughout the simulationThe architecture used in study 1.3 was used with one modifi-cation, namely the continuation of learning in the PFCthroughout the simulation (as in study 1.2).

Model parameters were the same as in study 1.1. Behavioraland activation data were collected and analyzed as above.

4.2. Simulation study 2: Waszak et al. (2008)

Model parameters were the same as in Simulation 1.1. Atten-tion to the relevant stimulus dimension was encoded by a gat-ing parameter (set at 0.6) by which the activation of therelevant stimulus dimension was multiplied (Verguts andNotebaert, 2008). The activation of the irrelevant stimulus di-mension was scaled by 0.4. Color and shape stimuli wereencoded using two vectors of equal length (six elementseach). Therewere two stimulus layers (6 nodes/layer), each pro-viding input to the PFC and BG layers. The PFC and BG layerswere divided into task-relevant sub-populations (S and C, forshapes and colors, respectively). Both PFC sub-populationsand BG sub-populations had 1000 units each. Responses werethe same across tasks and were encoded using two units.

The model was instructed in two consecutive phases (onefor the color task, one for the shape task), before the start ofthe experiment, similar to Simulation 1. After the instruc-tions, the stimuli (composed of two dimensions as describedabove) were presented. The model was given exactly thesame task as the subjects in Waszak et al. (2008). It was ex-posed to 5 practice blocks of 96 trials each in which univalent,bivalent and instructed stimuli were randomized. The 4th and5th practice blocks were preceded by two test blocks of 36 tri-als each (following the design ofWaszak et al., 2008). RTs werecalculated as before. Stimulus-specific RTs were averagedacross the different subjects (simulations).

4.3. Simulation study 3: simulating goal neglect

For a population of 100 models, the Ruge and Wolfensteller(2010) paradigm was adapted to include a verbal reportingtask immediately following each of the 32 S-R trials. In theverbal reporting task, a stimulus probe was presented (thesame stimulus presented for the S-R task) and the activationof the (verbal) response representation layer (on the left-hand side of Fig. 1) was read out directly, corresponding tothe production of a verbal response by the subject.

64 B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

To simulate the rapid presentation of stimuli as in Duncanet al. (2008) , the time to respond in the S-R task was limited(temporal deadline, tmax=20, 40, or 60 cycles), and the timeto respond to the verbal task held constant (temporal deadline200 cycles). To implement the hypothesis that goal neglect is afunction of the integrity of prefrontal learning, the learningrate of the PFC was set at 3 different levels (λPFC=1, 5 and 9).The learning rate of the BG was kept at its original value of9; however, similar results as those reported here wereobtained when we varied this learning rate too. Goal neglectwas quantified as the discrepancy between themean percent-age errors on the S-R and verbal task.

Acknowledgments

We thank Kevin Diependaele, Kevin Gurney and SebastienHelie for their valuable comments on a previous version ofthis manuscript. AR and TV were supported by Research Pro-ject 3G005909 awarded by FWO-Flanders (Belgium). Addresscorrespondence to Anand.Ramamoorthy@UGent.be.

R E F E R E N C E S

Ashby, F.G., Alfonso-Reese, L.A., Turken, A.U., Waldron, E.M., 1998.A neuropsychological theory of multiple systems in categorylearning. Psychol. Rev. 105, 442–481.

Ashby, G.F., Ennis, J.M., Spiering, B.J., 2007. A neurobiologicaltheory of automaticity in perceptual categorization. Psychol.Rev. 114 (3), 632–656.

Ashby, G.F., Turner, B.O., Horvitz, J.C., 2010. Cortical and basalganglia contributions to habit learning and automaticity.Trends Cogn. Sci. 14 (5), 208–215.

Ashby, G.F., Paul, E.J., Maddox, T.W., 2011. COVIS. In: Pothos, E.M.,Wills, A.J. (Eds.), Formal Approaches in Categorization.Cambridge University Press, New York.

Bandura, A., 1977. Social Learning Theory. Prentice Hall, NJ.Benedetti, F., Mayberg, H.S., Wager, T.D., Stohler, C.S., Zubieta, J.-K.,

2005. Neurobiological mechanisms of the placebo effect. J.Neurosci. 25 (45), 10390–10402.

Botvinick, M.M., Plaut, D.C., 2006. Short-term memory for serialorder: a recurrent neural network model. Psychol. Rev. 2,201–233.

Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S., Cohen, J.D.,2001. Conflict monitoring and cognitive control. Psychol. Rev.108, 624–652.

Boureau, Y.-L., Dayan, P., 2010. Opponency revisited: competitionand cooperation between dopamine and serotonin.Neuropsychopharmacolog. Rev. 1–24.

Brass, M., Wenke, D., Spengler, S., Waszak, F., 2009. Neuralcorrelates of overcoming interference from instructed andimplemented stimulus–response associations. J. Neurosci. 29(6), 1766–1772.

Carver, C.S., Johnson, S.L., Joorman, J., 2008. Serotonergic function,two-mode models of self regulation, and vulnerability todepression: what depression has in common with impulsiveaggression. Psychol. Bull. 134 (6), 912–943.

Cohen, J.D., Dunbar, K., McClelland, J.L., 1990. On the control ofautomatic processes: a parallel distributed processing accountof the stroop effect. Psychol. Rev. 97 (3), 332–361.

Cohen-Kdoshay, O., Meiran, N., 2009. The representation ofinstructions operates like a prepared reflex. Exp. Psychol. 56(2), 128–133.

Cole, M.W., Bagic, A., Kass, R., Schneider, W., 2010. Prefrontaldynamics underlying rapid instructed task learning reversewith practice. J. Neurosci. 30 (42), 14245–14254.

Craggs, J.G., Price, D.D., Perlstein, W.M., Verne, G.N., Robinson,M.E., 2008. The dynamic mechanisms of placebo inducedanalgesia: evidence of sustained and transient regionalinvolvement. Pain 139, 660–669.

Czernochowski, D., 2011. ERP evidence for scarce rulerepresentations in older adults following short, but not longpreparatory intervals. Front. Psychol. 2:221.

Daw, N.D., Niv, Y., Dayan, P., 2005. Uncertainty-based competitionbetween prefrontal and dorsolateral striatal systems forbehavioral control. Nature Neurosci. 8, 1704–1711.

De Houwer, J., Beckers, T., Vandorpe, S., Custers, R., 2005. Furtherevidence for the role of mode-independent short-termassociations in spatial Simon effects. Percept. Psychophys. 67(4), 659–666.

Dehaene, S., Cohen, L., 2007. Cultural recylcing of cortical maps.Neuron 56, 384–398.

Derbyshire, S.W., Whalley, M.G., Oakley, D.A., 2009. Fibromyalgiapain and its modulation by hypnotic and non-hypnoticsuggestion: an fMRI analysis. Eur. J. Pain 13, 542–550.

Derrfuss, J., Brass, M., Neumann, J., von Cramon, D., 2005.Involvement of the inferior frontal junction in cognitivecontrol: meta-analyses of switching and stroop studies. Hum.Brain Mapp. 25, 22–34.

Derrfuss, J., Brass, M., von Cramon, D., Lohmann, G., Amunts, K.,2009. Neural activation at the junction of the inferior frontalsulcus and the inferior precentral sulcus: interindividualvariability, reliability, and association with sulcal morphology.Hum. Brain Mapp. 30 (1), 299–311.

Doll, B.B., Jacobs, W.J., Sanfey, A.G., Frank, M.J., 2009. Instructionalcontrol of reinforcement learning: a behavioural andneurocomputational investigation. Brain Res. 1299, 74–94.

Dominey, P.F., 2005. From sensorimotor sequence to grammaticalconstruction: evidence from simulation and neurophysiology.Adapt. Behav. 13 (4), 347–361.

Duncan, J., Emslie, H., Williams, P., 1996. Intelligence and thefrontal lobe: the organization of goal-directed behaviour. Cogn.Psychol. 30, 257–303.

Duncan, J., Parr, A., Woolgar, A., Thompson, R., Bright, P., Cox, S.,et al., 2008. Goal neglect and Spearman's g: competing parts ofa complex task. J. Exp. Psychol. 137 (1), 131–148.

Elman, J.L., 1990. Finding structure in time. Cogn. Sci. 14 (2),179–211.

Fischer, M.H., Zwaan, R.A., 2008. Embodied language: a review ofthe tole of the motor system in language comprehension. Q. J.Exp. Psychol. 61 (6), 825–850.

Frank, M.J., 2005. Dynamic dopamine modulation in the basalganglia: a neurocomputational account of cognitive deficits inmedicated and nonmedicated Parkinsonism. J. Cogn. Neurosci.17 (1), 51–72.

Frank, M.J., 2006. Hold your horses: a dynamic computational rolefor the subthalamic nucleus in decision making. Neural Netw.19, 1120–1136.

Frenda, S.J., Nichols, R.M., Loftus, E.F., 2011. Current issues andadvances in misinformation research. Curr. Dir. Psychol. Sci.20 (1), 20–23.

Fritzke, B., 1997. Some Competitive Learning Methods.Ruhr-Universitat Bochum.

Grossberg, S., 1973. Contour enhancement, short-term memory,and constancies in reverberating neural networks. Stud. Appl.Math. 52, 213–257.

Hare, T.A., Malmaud, J., Rangel, A., 2011. Focusing attention on thehealth aspects of foods changes value signals in vmPFC andimproves dietary choice. J. Neurosci. 31 (30), 11077–11087.

Hartstra, E., Kuhn, S., Verguts, T., Brass, M., 2011. Theimplementation of verbal instructions: an fMRI study. Hum.Brain Mapp. 32, 1811–1824.

65B R A I N R E S E A R C H 1 4 3 9 ( 2 0 1 2 ) 5 4 – 6 5

Hauk, O., Shtyrov, Y., Pulvermuller, F., 2008. The time course ofaction and action-word comprehension in the human brain asrevealed by neurophysiology. J. Physiol. Paris 102 (1–3), 50–58.

Helie, S., Ashby, F.G., 2009. A neurocomputational model ofautomaticity and maintenance of abstract rules. Proccedingsof International Joint Conference on Neural Networks, pp.1192–1198. Atlanta, Georgia, USA.

Helie, S., Roeder, J.L., Ashby, F.G., 2010. Evidence for corticalautomaticity in rule-based categorization. J. Neurosci. 30 (42),14225–14234.

Luria, A.R., 1966. Higher Cortical Functions inMan. Tavistock, London.Mazzoni, G.A., Loftus, E.F., Kirsch, I., 2001. Changing beliefs about

implausible autobiographical events: a little plausbility goes along way. J. Exp. Psychol., Appl. 7 (1), 51–59.

Miller, E.K., Cohen, J.D., 2001. An integrative theory of prefrontalcortex function. Annu. Rev. Neurosci. 24, 167–202.

Mink, J., 1996. The basal ganglia: focused selection and inhibitionof competing motor programs. Prog. Neurobiol. 50, 381–425.

Monsell, S., 1996. Control of mental processes. In: Bruce, V. (Ed.),Unsolved Mysteries of the Mind. Hove, England: Erlbaum, pp.93–148.

Nakayama, Y., Yamagata, T., Tanji, J., Hoshi, E., 2008.Transformation of a virtual action plan into a motor plan inthe premotor cortex. J. Neurosci. 28 (41), 10287–10297.

Noelle, D.C., Cottrell, G.W., 1995. A connectionist model ofinstruction following. In: Moore, J.D., Lehman, J.F. (Eds.),Proceedings of the Seventeenth Annual Conference of theCognitive Science Society. Lawrence Erlbaum Associates,Hillsdale, NJ, pp. 369–374.

Petrovic, P., Kalso, E., Petersson, K.M., Andersson, J., Fransson, P.,Ingvar, M., 2010. A prefrontal non-opioid mechanism inplacebo analgesia. Pain 150 (1), 59–65.

Ploghaus, A., Becerra, L., Borras, C., Borsook, D., 2003. Neuralcircuitry underlying pain modulation: expectation, hynposis,placebo. TRENDS Cogn. Sci. 7 (5), 197–200.

Raz, A., Campbell, N.K., 2011. Can suggestion obviate reading?Supplementing primary Stroop evidence with exploratorynegative priming analyses. Conscious. Cogn. 20 (2), 312–320.

Reynolds, J.N., Hyland, B.I., Wickens, J.R., 2001. A cellularmechanism of reward-related learning. Nature 413, 67–70.

Ruge, H., Wolfensteller, U., 2010. Rapid forumlation of pragmaticrule representations in the human brain durigninstruction-based learning. Cereb. Cortex 20, 1656–1667.

Sloman, S.A., 1996. The empirical case for two systems ofreasoning. Psychol. Bull. 119 (1), 3–22.

St.John, M.F., McClelland, J.L., 1990. Learning and applyingcontextual constraints in sentence comprehension. Artif.Intell. 46 (1–2), 217–257.

Thorndike, E.L., 1911. Animal Intelligence. MacMillan, New York.Toni, I., Rushworth, M.F., Passingham, R.E., 2001. Neural

correlates ofvisuomotor associations. Exp. Brain Res. 141,359–369.

Verguts, T., Notebaert, W., 2008. Hebbian learning of cognitivecontrol: dealing with specific and nonspecific adaptation.Psychol. Rev. 115 (2), 518–525.

Waszak, F., Wenke, D., Brass, M., 2008. Cross-talk of instructedand applied arbitrary visuomotor mappings. Acta Psychol. 127,30–35.