“What” and “Where” in Visual Working Memory: A Computational Neurodynamical Perspective for...

lsquolsquoWhatrsquorsquo and lsquolsquoWherersquorsquo in Visual Working MemoryA Computational Neurodynamical Perspective for

Integrating fMRI and Single-Neuron Data

Gustavo Deco1 Edmund T Rolls2 and Barry Horwitz3

Abstract

amp Single-neuron recordings functional magnetic resonanceimaging (fMRI) data and the effects of lesions indicatethat the prefrontal cortex (PFC) is involved in some types ofworking memory and related cognitive processes Based onthese data two different models of the topographical andfunctional organization of the PFC have been proposedorganization-by-stimulus-domain and organization-by-processIn this article we utilize an integrate-and-fire network tomodel both single-neuron and fMRI data on short-termmemory in order to understand data obtained in topologi-cally different parts of the PFC during working memory tasksWe explicitly model the mechanisms that underlie working-memory-related activity during the execution of delay tasksthat have a lsquolsquowhatrsquorsquo-then-lsquolsquowherersquorsquo design (with both objectand spatial delayed responses within the same trial) Themodel contains different populations of neurons (as foundexperimentally) in attractor networks that respond in thedelay period to the stimulus object the stimulus positionand to combinations of both object and position informationThe pools are arranged hierarchically and have global inhi-bition through inhibitory interneurons to implement competi-

tion It is shown that a biasing attentional input to define thecurrent relevant information (object or location) enables thesystem to select the correct neuronal populations duringthe delay period in what is a biased competition model ofattention The processes occurring at the AMPA and NMDAsynapses are dynamically modeled in the integrate-and-fireimplementation to produce realistic spiking dynamics It isshown that the fMRI data characteristic of the dorsal PFCand linked to spatial processing and manipulation of items canbe reproduced in the model by a high level of inhibitionwhereas the fMRI data characteristic of the ventral PFC andlinked to object processing can be produced by a lower level ofinhibition even though the network is itself topographicallyhomogeneous with no spatial topology of the neurons Thisarticle thus not only presents a model for how spatial versusobject short-term memory could be implemented in thePFC but also shows that the fMRI BOLD signal measuredduring such tasks from different parts of the PFC could reflecta higher level of inhibition dorsally without this dorsal regionnecessarily being primarily spatial and the ventral regionobject-related amp

INTRODUCTION

The aim of this article is to integrate via a large-scaleneuronal network model that generates the dynamicsand synaptic processing of neurons in an integrate-and-fire implementation single-neuron and functional mag-netic resonance imaging (fMRI) measurements of theprefrontal cortex (PFC) associated with visual workingmemory processing One of the aims of this integration isto help interpret the fMRI signals recorded in topograph-ically separate parts of the PFC and more generally toprovide a fundamental approach to understanding fMRIsignals

The PFC is involved in at least some types of workingmemory and related processes such as planning (Fuster2000 Goel amp Grafman 1995 Goldman-Rakic 1995

Goldman-Rakic 1996) as shown by single-neuron (Fu-nahashi Bruce amp Goldman-Rakic 1989) neuroimaging(Ungerleider Courtney amp Haxby 1998) and lesionstudies (Levy amp Goldman-Rakic 1999 Goldman-Rakic1987) Working memory refers to an active system formaintaining and manipulating information in mind heldduring a short period of time (usually seconds) (Badde-ley 1986)

Two models of the topographical and functionalorganization of the PFC have been proposed (seeMiller 2000 for a review) The first model proposesorganization-by-stimulus-domain with spatial (lsquolsquowherersquorsquo)working memory supported by the dorsolateral PFC inthe neighborhood of the principal sulcus (Brodmannrsquosarea [BA] 469 in the middle frontal gyrus [MFG]) andobject (lsquolsquowhatrsquorsquo) working memory supported by theventrolateral PFC on the lateral convexity (BA 45 inthe inferior frontal gyrus [IFG]) Some but not all fMRIstudies in humans and single-cell data in primates

1Institucion Catalana de Recerca Estudis Avancats (ICREA) andUniversitat Pompeu Fabra 2University of Oxford 3NationalInstitutes of Health

copy 2004 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 164 pp 683ndash701

support this topographical organization (but see below)(Leung Gore amp Goldman-Rakic 2002 Wilson OrsquoSca-laidhe amp Goldman-Rakic 1993 Goldman-Rakic 1987Fuster Bauer amp Jervey 1982) A second model pro-poses a hierarchical organization of the PFC with non-mnemonic higher order functions (eg manipulation ofitems in memory) ascribed to dorsolateral prefrontalareas and short-term memory maintenance functionsascribed to inferior prefrontal areas (Owen et al 1999DrsquoEsposito et al 1998 Petrides 1994) Consistent withthis second model some event-related fMRI studies(Postle amp DrsquoEsposito 1999 2000) with a what-then-where design failed to find evidence for anatomicalsegregation of spatial and object visual working memoryin the PFC of humans

Analysis of how the PFC implements working memoryhas been extended by analyzing neuronal activity whena monkey performs a delay task with a what-then-wheredesign (ie the monkey performs an object and spatialdelayed response within the same trial) With this kindof experimental paradigm Rao Rainer and Miller(1997) found some neurons that showed either object-tuned (what) or location-tuned (where) delay activityHowever a large percentage of the measured PFCneurons (52) showed both what and where tuningSimilarly in delay conditional response and delayedspatial response tasks some neurons respond in thedelay period to the stimulus object some to the stim-ulus position some to combinations of the stimulusobject and its position and some to the responserequired with no clear topographical separation re-ported (White amp Wise 1999 Asaad Rainer amp Miller1998 2000 Hoshi Shima amp Tanji 1998)

Taken together all these results imply that thetopographical and functional organization of the PFCin relation to different types of working memory is stillinsufficiently understood To elucidate the underlyingmechanisms and the fMRI signals that can result inhumans we describe here a computational model ofthe PFC that can perform what-then-where and where-then-what memory tasks of the type illustrated inFigure 1 (see Methods for details) and has spikingneurons whose simulated activities are directly compa-rable to the single-neuron data recorded in the PFC(Rao et al 1997) The integrate-and-fire neurons areorganized into a hierarchical set of attractor or autoas-sociation networks each implementing a short-termmemory as illustrated in Figure 2 (see Methods fordetails and Rolls amp Treves 1998 Rolls amp Deco 2002)There are separate attractors for lsquolsquowhatrsquorsquo stimulus wasshown for lsquolsquowherersquorsquo the stimulus was shown and acombined lsquolsquowhatndashwherersquorsquo attractor pool of neuronsThe relative activity of the different attractor poolsrequired to perform the what-then-where and where-then-what short-term memory tasks is provided by anexternal attentional signal that biases the differentpools of neurons within the general framework of the

biased competition model of attention (Rolls amp Deco2002 Reynolds amp Desimone 1999 Chelazzi MillerDuncan amp Desimone 1993 Chelazzi 1998 MillerGochin and Cross 1993 Motter 1993 Spitzer Desi-mone amp Moran 1988 Moran amp Desimone 1985) Rollsand Deco (2002) added to this framework by introduc-ing a neurodynamical theoretical framework for biasedcompetition which enables the whole framework tooperate from the level of integrate-and-fire neurons upto global phenomena such as short-term memory andattention Because the model is implemented at theintegrate-and-fire single-neuron level with the biophys-ical properties of the synapses part of the simulationwe are then able as described in this article tointegrate over the total synaptic activity within an areaand convolve with the hemodynamic response functionillustrated in Figure 3 (see Methods for details andRolls and Deco 2002 Horwitz Tagamets amp McIntosh1999 Horwitz Friston amp Taylor 2000) in order tomodel the fMRI blood oxygen level-dependent (BOLD)signal and simulate human neuroimaging experimentsBasing the link to the BOLD signal on the total synapticactivity (in fact current) within a region is appropriatebecause channels are opened by synaptic activity thatrequire ions to be pumped back against electrochem-ical gradients and this is the very energy-intensiveaspect of neuronal activity Experimental support forthis idea has been provided by Logothetis PaulsAugath Trinath and Oeltermann (2001) (for reviewssee Lauritzen 2001 Jueptner amp Weiller 1995)

RESULTS

Single-Neuron Recordings

In this subsection we present a theoretical analysis ofneuronal activity in the primate PFC underlying theexecution of a what-then-where working memory taskThe neuronal recordings of Rao et al (1997) demonstrat-ed the existence of neurons showing stimulus-domain-specific sensitivity that is object-tuned activity in thewhat-delay and location-tuned activity in the where-delay but they found also a large proportion of neuronsshowing integrated what-and-wherendashtuned activity dur-ing both what- and where-delays During each trial oftheir experiment while the monkey maintained fixationon a center spot a sample object was briefly presentedon the screen After a first delay (what-delay) two testobjects were briefly presented at two of four possibleextrafoveal locations One of the test objects matchedthe sample and the other was a nonmatch After asecond delay (where-delay) the monkey had to respondwith a saccade to the remembered location of the matchFigures 4A and 5A illustrate the neuronal recordings ofRao et al

The simulation of this experiment starts with a precueperiod of 1000 msec in which the network exhibitsspontaneous activity (3 Hz for the excitatory pools and

684 Journal of Cognitive Neuroscience Volume 16 Number 4

9 Hz for the inhibitory pools as in the experimentalrecordings of Wilson Scalaidhe amp Goldman-Rakic1994) A target stimulus with a feature characteristic Fi

and at a location Sj is presented next during the first cueperiod of 1500 msec (ie during this period the objectpool Fi and the spatial pool Sj receive external Poissonspikes with an increased rate from next to next + linput)After the first cue period the stimulus is removed andonly the feature characteristics of the target object haveto be encoded and retained during a what-delay periodof 6500 msec We modeled the attentional what-bias by

assuming that all feature-specific pools (ie all pools Fn

for all n) receive Poisson spikes with an increased rate(next + latt) This is followed by a second cue period of1500 msec where a matched object (identical to thecued object) reappeared at another new location differ-ent from the one originally cued during the first cueperiod After that only the location of the matchedtarget has to be encoded and the feature informationcan be ignored during this second where-delay period of6500 msec Again we modeled the attentional where-bias by assuming that all location-specific pools (ie all

Figure 1 Schematic diagram of a typical what-then-where and where-then-what trial Following an instructional cue the behavioral task beginswith a fixation period (Tfix) followed by an initial cueing stimulus presentation (Tcue1) followed by a first delay period (Tdelay1) followed by thepresentation of a matching stimulus and a distractor (Tcue2) followed by a second delay period (Tdelay2) followed by a probe stimulus (Tresponse)that elicited a response In what-then-where trials an object delay task is followed by a spatial delayed response task In where-then-what trials aspatial delay task is followed by an object delayed response task

Deco Rolls and Horwitz 685

pools Sn for all n) receive Poisson spikes with an in-creased rate (next + latt) This second delay is followedby a period of 1500 msec where the final probe ispresented and a response has to be made

Figures 4B and 5B show the results of the simula-tions which can be directly compared with the corre-sponding experimental observations of Rao et al(1997) shown in Figures 4A and 5A Figures 4A and4B plot the responses of prefrontal neurons showing

either object-tuned (top part of A and B) or location-tuned (bottom part of A and B) delayed activity lsquolsquoGoodobjectrsquorsquo and lsquolsquopoor objectrsquorsquo refer to the objects used assamples (lsquolsquogoodrsquorsquo means the preferred object or featurefor that neuron and lsquolsquopoorrsquorsquo refers to the nonpreferredobject or feature for that neuron) lsquolsquoGood locationsrsquorsquoand lsquolsquopoor locationsrsquorsquo refer to the locations cued by thematching object during the second cue (lsquolsquogoodrsquorsquo meansthe preferred spatial location for that neuron and

Figure 2 Prefrontal cortical module The synaptic current flows into the cell are mediated by four different families of receptors The recurrentexcitatory postsynaptic currents are given by two different types of EPSP respectively mediated by AMPA and NMDA receptors These twoglutamatergic excitatory synapses are on the pyramidal cells and interneurons The external inputs (background sensory input or externaltop-down interactions from other areas) are mediated by AMPA synapses on pyramidal cells and interneurons Inhibitory GABAergic synapses onpyramidal cells and interneurons yield corresponding IPSPs Each neuron receives also Next excitatory AMPA synaptic connections from outside thenetwork These connections provide three different type of external interactions (1) a background noise due to the external spontaneous firingactivity (2) a sensory input and (3) an attentional stimulus-domain-specific bias


lsquolsquopoorrsquorsquo refers to the nonpreferred spatial location forthat neuron) The experimental and numerical binwidths were 20 msec In the case of the simulations(Figure 4B) we present the calculated temporal evolu-tion of the averaged population activity (over all neu-rons in each specific pool during the temporal bin of20-msec period) of specific pools The spatiotemporalspiking activity shows that during the first short-termmemory what-delay period only the what-specific neu-rons representing the feature characteristics of the firstcue maintain persistent activity and build up a stableglobal attractor in the network that maintains the firingduring the delay period (Figure 4B top) On the otherhand during the second short-term memory where-delay period only the where-specific neurons repre-senting the location of the matched cue during thesecond stimulus presentation maintain persistent activ-ity and build up a stable global attractor in the networkthat maintains the firing during the second delayperiod (Figure 4B bottom)

Figure 5 shows the responses of prefrontal neuronswith best responses to a combination of a particularobject and a particular response in the second delayperiod These neurons show object-tuned activity in thefirst what-delay (left panel) and location-tuned activityduring the second where-delay (middle panel) Theright panel shows activity that is tuned to both objectand location during the second where-delay Figure 5Ashows neuronal data from the recordings of Rao et al(1997) Figure 5B shows the averaged activity of thepopulation of neurons in the simulation that respondbest to a combination of what and where information Inthe simulation as well as in the neurophysiologicalexperiments cueing a good location with a good object

elicited more activity than cueing a good location witha poor object A poor location elicited less activity thana good location regardless of which object cued itThese specific global attractors which correspond to aspecific condition of attention to a particular locationtriggered by a particular object condition in the taskincorporate several single-condition attractors includingobject-specific location-specific and object- and loca-tion-specific attractor populations of neurons The cuestimulus and the biasing attentional top-down informa-tion applied to the sensory neurons drive the systeminto the corresponding global attractor according to thebiased competition mechanism

The numerical simulations thus show that the as-sumed microcircuits in the PFC shown in Figure 2 areconsistent with the empirical microscopic measure-ments (single-neuron recording) of Rao et al (1997)and instantiate therefore a concrete microscopic (neu-ron level) organization of the PFC (which is stimulus-domain-specific) that incorporates sensory pools withwhat-specific and where-specific sensitivity with combi-nation what-and-where pools The specific underlyingwiring could be established by Hebbian learning

Event-Related fMRI Data What-Then-Where

In this section we describe simulations of the event-related fMRI investigation of Postle and DrsquoEsposito(1999) in which they investigated the time course ofthe activations in the dorsolateral PFC (Areas 9 and 46)and ventrolateral PFC (Areas 44 45 47) during thewhat-then-where and where-then-what tasks describedabove (see Figure 1) Figure 6A and C plot the tem-poral evolution of the trial-averaged fMRI signal ex-tracted from the ventrolateral and dorsolateral PFCrespectively Delay-period activity in the ventrolateraland dorsolateral PFC was observed during both what-and where-delay periods Because of the similarity ofthe observed fMRI signal evolution under both thewhat-then-where and where-then-what conditions es-pecially during both delay periods Postle and DrsquoEspo-sito concluded that the hypothesis that ventrolateraland dorsolateral PFC regions may differentially supportworking memory for object and spatial stimuli respec-tively could not be confirmed suggesting a morefunctional organization of the PFC Note in Figure 6Aand B the 5- to 6-sec delay of the fMRI signal due tothe hemodynamical response

In order to understand better the neurodynamicalsubstrate underlying these fMRI results which suggesta more functional organization of the PFC than a stim-ulus-domain-specific organization (object vs spatial)and especially to harmonize these facts with the topo-graphical stimulus-domain-specific organization sug-gested by some single- neuron recording experiments(Rao et al 1997 Goldman-Rakic 1987) we ran ourmodel for the setup of Postle and DrsquoEsposito (1999) and

Figure 3 Standard hemodynamic response function utilized for theconvolution with synaptic activity to produce simulated event-relatedfMRI activity from the neuronal network model of the PFC


simulated the temporal evolution of the fMRI signal Wesimulated with our model both the what-then-whereand where-then-what conditions

For the what-then-where condition the simulationstarts with a precue period of 1000 msec during whichthe network exhibits spontaneous activity Next a targetstimulus with a feature characteristic Fi and at a locationSj is presented during the first cue period of 1500 msec(ie during this period object pool Fi and spatial pool Sj

receive external Poisson spikes with an increased ratefrom next to next + linput) After the first cue period thestimulus is removed and only the feature character-istics of the target object have to be encoded andretained during a what-delay period of 6500 msec Wemodel the attentional what-bias by assuming that allfeature-specific pools receive Poisson spikes with anincreased rate (next + latt) This is followed by asecond cue period of 1500 msec where a matchedobject (identical to the cued object) reappears atanother new location different from the one originallycued during the first cue period After that only the

location of the matched target has to be encoded andthe feature information can be now ignored duringthis second where-delay period of 6500 msec Againwe model the attentional where-bias by assuming thatall location-specific pools (ie all pools Sn for all n)receive Poisson spikes with an increased rate (next +latt) This second delay is followed by a period of1500 msec where the final probe is presented and aresponse has to be performed

For the where-then-what condition after the precueperiod of 1000 msec during which the network exhibitsspontaneous activity a target stimulus with a featurecharacteristic Fi and at a location Sj is presented duringthe first cue period of 1500 msec (During this periodobject pool Fi and spatial pool Sj receive externalPoisson spikes with an increased rate from next tonext + linput) After the first cue period the stimulusis removed and only the spatial location of the targetobject has to be encoded and retained during a where-delay period of 6500 msec We model the attentionalwhere-bias by assuming that all spatial-specific pools

Figure 4 Responses of prefrontal neurons showing either object-tuned (top of part A and B) or location-tuned (bottom of part A and B) delayedactivity (A) Experimental recordings of Rao et al (1997) (B) Model simulations presenting the calculated temporal evolution of the averagedactivity of the population of feature-specific (top) and spatial or location-specific (bottom) neurons lsquolsquoGood objectrsquorsquo and lsquolsquopoor objectrsquorsquo refer towhether the object shown as a sample was effective in producing a response from the neuron or not lsquolsquoGood locationsrsquorsquo and lsquolsquopoor locationsrsquorsquo referto whether the location cued by the second cue was effective for the neuron or not The experimental and numerical bin widths were 20 msec (Aafter Rao et al 1997 with permission)


receive Poisson spikes with an increased rate (next +latt) This is followed by a second cue period of1500 msec where an object different from the targetreappeared at the first cued location After that onlythe feature characteristics of the object presentedduring the second cue period have to be encodedand the spatial information can now be ignored duringthis second what-delay period of 6500 msec Again wemodel the attentional what-bias by assuming that allfeature-specific pools (ie all pools Fn for all n) receivePoisson spikes with an increased rate (next + latt) Thissecond delay period is followed by a period of1500 msec where the final probe is presented and aresponse has to be performed

We found that the event-related fMRI data of Postleand DrsquoEsposito (1999) could be modeled by varying the

parameters that regulate the dynamics of the networkand that no spatial topology had to be introduced intothe prefrontal network Specifically we had to assumethat the network associated with the dorsolateral PFChas a higher level of inhibition than the networkassociated with the ventrolateral PFC The level ofinhibition was increased by increasing by a factor of1025 the maximal GABA conductivity constants gGABA

specified in Appendix A The evidence for this finding isdescribed next

Figure 6B and D presents the simulated fMRI signal forboth conditions and for both ventrolateral (low inhibi-tion top right figure) and dorsolateral (high inhibitionbottom right figure) PFC The simulations compare fa-vorably with the results of Postle and DrsquoEsposito (1999)shown in Figure 6A and C The important result of the

Figure 5 Responses of prefrontal neurons with best responses to a particular combination in the second delay period of a particular object and aparticular response These neurons show object-tuned activity in the first what-delay ( left panel) and location-tuned activity during the secondwhere-delay (middle panel) The right panel shows activity that is tuned to both object and location during the second where-delay (A)Experimental single-neuron recordings of Rao et al (1997) (B) Model simulations presenting the calculated temporal evolution of the averagedactivity of the population of neurons responding best to a combination of what and where information In the case illustrated cueing a goodlocation with a good object elicited more activity than cueing a good location with a poor object and poor location elicited less activity than a goodlocation regardless of which object cued it (After Rao et al 1997 with permission)


simulations is that fMRI activations of the type describedby Postle and DrsquoEsposito for the ventrolateral PFC(Figure 6A) can be obtained by using a low level ofinhibition (Figure 6B) and the fMRI activations of thetypes described for the dorsolateral PFC (Figure 6C) canbe obtained by using a high level of inhibition (Figure 6D)

Unsurprisingly because there was no topology ofobject versus spatial neurons in the simulated networkno differences are detected macroscopically (ie at thefMRI level) between the what-then-where and where-then-what conditions for the low-inhibition (ventrolater-al) network model (top right) Nor are any differencesdetected macroscopically between the what-then-where

and where-then-what conditions for the high-inhibition(dorsolateral) network model (bottom right) The simi-larities between the results of the simulations at themacroscopic level in the what-then-where and where-then-what task conditions are as found experimentally inthe fMRI signal However this fact does not mean ofcourse that the underlying neuronal responses are iden-tical during the what-then-where and where-then-whatconditions Figures 7 and 8 plot rastergrams for a pop-ulation of single neurons in the simulations for bothconditions for the ventrolateral network model with lowinhibition (Figure 7) and for the dorsolateral networkmodel with high inhibition (Figure 8) The spatiotempo-

Figure 6 Temporal evolution of the trial-averaged measured f MRI signal extracted from the ventrolateral (A) and dorsolateral (C) PFC(after Postle and DrsquoEsposito 1999) and simulated f MRI signals for both the what-then-where and where-then-what conditions and for boththe low-inhibition (ventrolateral) (B) and high-inhibition (dorsolateral) models (D) PFC Delay period activity in the ventrolateral anddorsolateral PFC is observed during both what- and where-delay periods Two conditions were compared what-then-where andwhere-then-what Because of the similarity of the development of the fMRI signal under both conditions and specially during both delayperiods the hypothesis that the ventrolateral PFC region may differentially support working memory for objects and the dorsolateral PFCfor spatial stimuli could not be confirmed suggesting a more functional organization of the PFC Note in the figure the 5- to 6-sec delay

of the fMRI signal due to the hemodynamical response


ral spiking activity shows that at the neuronal (micro-scopic) level topographically organized in Figures 7 and8 by what (Object 1 or 2) where (Space 1 or 2) and what-and-where (Ob-Sp) specific neurons strong differencesin the evolution and the structure of the successivelyelicited attractors for each temporal period are evidentThis fine-grain microscopic (neuron-level) structure islost at the macroscopic level of coarser spatial resolutionmeasured by MRI In fact during the short-term memorydelay period associated with a what (or where) taskonly the neurons representing the object feature char-acteristics (or spatial location) of the cue maintain per-sistent activity and build up a stable global attractor inthe network that maintains the firing during the delayperiod These specific global attractors each corre-

sponding to a specific stimulus-domain-attention condi-tion incorporate several single-pool attractors formedfrom the group of sensory pools or neuronal population(object-specific or space-specific neuronal populations)and from the group of combination tuned what-and-where neuronal pools The cue stimulus and the biasingattentional top-down information applied to the sensoryneurons drive the system into the corresponding globalattractor according to the biased competition mechanism

To test the alternative hypothesis that the dorsolat-eral PFC is more associated with spatial working mem-oryand the ventrolateral PFC is more associated withobject working memory we ran simulations for thesame tasks but now assuming that in the dorsolateralPFC there are more spatial sensory neurons (a factor

Figure 7 Rastergrams of simulations of the what-then-where (top) and where-then-what (bottom) task after the experimental paradigm ofPostle and DrsquoEsposito (1999) for the case of a network model with a low level of inhibition which produces results like those from theventrolateral region of the PFC The spiking activity of different populations of neurons is shown some with lsquolsquowhatrsquorsquo tuning (to Object 1 or 2)some with lsquolsquowherersquorsquo tuning to spatial location (labeled Space 1 or 2) and some with what-and-where tuning to different combinations ofobjects and locations (labeled Ob-Sp)


105 more than object sensory neurons) and in theventrolateral PFC there are more object sensory neu-rons (also a factor 105) For this set of simulationswe set the level of inhibition to be low for both theventrolateral and dorsolateral PFC Figure 9 showsthe simulation results An asymmetric behavior of thefMRI signal response is observed between the what-then-where (dashed line) and where-then-what (contin-uous line) conditions In the dorsolateral PFC (bottomof the figure) more activity is observed during the firstwhere-delay period during the where-then-what condi-tion whereas in the what-then-where conditionmore activity is observed during the second where-delay period In the ventrolateral PFC (top of thefigure) more activity is observed during the second

what-delay period during the where-then-what condi-tion whereas in the what-then-where condition moreactivity is observed during the first what-delay periodThese simulated fMRI signals are not consistent with theempirical findings of Postle and DrsquoEsposito (1999)We emphasize that Figure 9 is the only figure in thisarticle in which the spatial and object neurons are treatedas being topographically organized into separable pop-ulations realized in the simulations performed by run-ning the simulations separately with more object ormore spatial neurons to represent the ventral and dorso-lateral PFC

In summary our simulations show that single-celland fMRI data are consistent with the hypothesis thatdifferences between the dorsal and ventral PFC in the

Figure 8 Rastergrams of simulations of the what-then-where (top) and where-then-what (bottom) task after the experimental paradigm ofPostle and DrsquoEsposito (1999) for the case of a network model with high level of inhibition which produces results like those from thedorsolateral region of the PFC The spiking activity of different populations of neurons is shown some with lsquolsquowhatrsquorsquo tuning (to Object 1 or 2)some with lsquolsquowherersquorsquo tuning to spatial location (labeled Space 1 or 2) and some with what-and-where tuning to different combinations ofobjects and locations (labeled Ob-Sp)


fMRI signal can be accounted for by a higher level ofinhibition in the dorsal PFC relative to the ventral PFCThe simulations also show that the imaging results areconsistent with an architecture that is stimulus-domain-specific at the microscopic or neuronal level (withdifferent what where and what-and-where sensitiveneurons) but with the different neurons intermixedso that there is no separate topography at the macro-scopic level with separate regions for objects (ventral)versus space (dorsal) Indeed the simulations shown inFigure 9 with networks for objects and locations thatare even minimally spatially segregated (by 10) donot reproduce the fMRI results of Postle and DrsquoEspo-sito (1999) Of course this latter point does not meanthat empirical evidence for object versus location to-pology in the PFC will not be revealed in future

DISCUSSION

In the present study we investigated two differenthypotheses concerning the functional organization ofthe PFC during the delay period of a working memorytask One the organization-by-stimulus-domain hy-pothesis posits that the ventrolateral PFC contains alarge number of neurons that maintain active represen-tations of the visual features of objects during workingmemory delay periods and that the dorsolateral PFCcontains a large number of neurons that maintain rep-resentations of the spatial locations of objects duringsuch delays Second the organization-by-process hy-pothesis asserts that the main functional differencebetween the ventrolateral and dorsolateral PFC is thatthe ventrolateral PFC is concerned with maintenance of

Figure 9 Simulations with spatial topology in the network to simulate data according to the hypothesis that the dorsolateral PFC is moreassociated with spatial working memory and the ventrolateral PFC is more associated with object working memory The simulations for thedorsolateral PFC model have more spatial than object neurons (by a factor of 105) The simulations for the ventrolateral PFC model havemore object than spatial neurons (by a factor of 105) An asymmetry in the behavior of the f MRI signal was observed between the what-then-where (dashed line) and where-then-what (continuous line) conditions In the dorsolateral PFC condition (bottom of the figure) more activityis observed during the where (first) delay period during the where-then-what condition and in the what-then-where condition more activity isobserved during the where (second) delay period Consistently in the ventrolateral PFC (top of the figure) more activity is observed duringthe what (second) delay period during the where-then-what condition and in the what-then-where condition more activity is also observedduring the what (first) delay period


the information during the delay period of a workingmemory task whereas the dorsolateral PFC is involvedin manipulation of stored information Our main findingwas that our model could account for the neurophysi-ological activity seen in both the ventrolateral anddorsolateral PFC during the delay periods of workingmemory tasks (as shown by the empirical results of Raoet al 1997 see Figure 4) and at the same time couldprovide simulated fMRI patterns that matched experi-mental findings during a what-then-where short-termmemory task for both PFC sectors (as shown by thefMRI findings of Postle amp DrsquoEsposito 1999 see Figure 6)However we could not do this if we assumed that thedifference between ventrolateral and dorsolateral PFCfollowed the organization-by-stimulus-domain hypothe-sis (see Figure 9) Rather we had to assume that thedifferences between these two prefrontal regions re-sulted from assigning a greater amount of inhibition tothe dorsolateral portion of the PFC Our modeling thussuggests that different levels of competition of the net-works associated with the ventrolateral and dorsolateralPFC could be the neural basis of the different fMRIsignals associated with these brain regions In additionthe network model suggests that one important func-tional difference between the dorsolateral PFC and theventrolateral PFC is related to greater inhibition in thedorsolateral PFC than in the ventrolateral PFC Bothbrain areas show maintenance capabilities related totheir capacities to maintain stable attractors during delayperiods but the increased level of inhibition assumed inthe dorsolateral PFC may be associated with the capacityof this brain region to support more complex functionsExactly what those more complex functions may be isnot revealed by the present studies but higher inhibi-tion in the dorsolateral PFC might be useful for main-taining several separate representations and preventingthe formation of a global attractor which could beuseful if several items must be held in memory formanipulation Another possibility is that the informationcoming into the dorsolateral PFC might be more distrib-uted (which might be consistent with a spatial ascompared to an object representation cf Rolls Stringeramp Trappenberg 2002) and thus might require moreinhibition to prevent an overdistributed representationwhich does have disadvantages (Rolls amp Treves 1998Rolls amp Deco 2002)

Previous studies using such large-scale models torelate neural activity to functional brain imaging datahave employed leaky integrator-type neuronal units(eg Tagamets amp Horwitz 1998 employed WilsonndashCow-an units) One novel aspect of our model and of thework described in this article is that our architecturewhich has multiple attractor networks each composedof a separate population of neurons interconnected toform a separate or local attractor is implemented withintegrate-and-fire neurons (in the theoretical framework

of Brunel and Wang 2001) so that the details of thespiking and synaptic mechanisms involved can be under-stood and so that predictions can be made about theeffects for example of neurotransmitters and pharma-cological agents that have particular effects on synaptictransmission The processes occurring at the AMPA andN-methyl-D-aspartate (NMDA) synapses are dynamicallymodeled in the integrate-and-fire implementation toproduce realistic spiking dynamics In addition thearchitecture described is an extension beyond attractorarchitectures in that different neuronal pools or popula-tions connected hierarchically are simulated and in thatan attentional bias is applied that allows the network toselect the correct response given the current stimulusand attentional bias Specifically in this article we haveformulated the biased competition hypothesis for thefirst time in the framework of a very detailed and bio-physically realistic spiking neuronal and synaptic dy-namics This has enabled us to integrate the effects ofrecurrent maintenance associated with short-termmemory with the biasing attentional effects associatedwith a cognitive task As a result we have been able toprovide not only a quantitative and qualitative descriptionof different experiments but also are able to makeconcrete predictions that are testable experimentallyfor example about the effects of dopamine receptorblockade on the operation of the working memorysystems described here (in preparation)

It was pointed out in the Introduction that amongexperimentalists there are two major hypothesesconcerning the organization of PFC the organization-by-stimulus-processing-domain hypothesis (GoldmanRakic 1987) and the organization-by-functional-pro-cessing hypothesis (Owen et al 1999 DrsquoEsposito et al1998) For the latter the main distinction is centered onthe difference between a manipulation function (dorso-lateral PFC) and a maintenance function (ventrolateralPFC) during the delay period of a working memorytask Our findingmdashthat the distinction between dorso-lateral and ventrolateral PFC is that the former has agreater level of inhibition than does the lattermdashis moreconsistent with the organization-by-functional-process-ing hypothesis than it is with the organization-by-stim-ulus-domain hypothesis A test for the future wouldbe to explicitly employ a task requiring active manipu-lation of the items in working memory and comparesimulated and experimental fMRI activity However weshould mention that our results do not rule out entirelythe organization-by-stimulus-domain hypothesis Itmay be that spatial processing of visual informationitself requires a greater level of inhibition in the PFCA way to test this would be to expand our model toexplicitly model the preprocessing steps in posteriorcortex required for object and spatial vision (cf Corchsand Deco 2002 Rolls and Deco 2002 Tagamets ampHorwitz 1998)


Overall the results described here demonstrate howthe use of large-scale neural modeling that allows oneto relate microscopic and macroscopic neurophysiolog-ical activity enables one to make explicit hypothesesabout the neural substrates for high-level cognitiveconcepts that can be tested empirically in both humansand nonhumans

METHODS

What-Then-Where and Where-Then-What DelayedResponse Experiments

What-then-where- and where-then-what delayed re-sponse tasks have been used to study the organizationof the PFC at the single-neuron (Rao et al 1997) andfMRI levels (Postle amp DrsquoEsposito 1999 2000) Figure 1shows two examples of these designs Following aninstructional cue the behavioral task begins with afixation period (Tfix) followed by an initial cueingstimulus presentation (Tcue1) followed by a first delayperiod (Tdelay1) followed by the presentation of amatching stimulus and a distractor (Tcue2) followed bya second delay period (Tdelay2) followed by a probestimulus (Tresponse) which indicates that a response canbe made In what-then-where trials one has to encodethe featural (object-based) characteristics of the initialcue ignoring the spatial location on the screen and toretain in short-term memory this what-based informa-tion during the first delay Following this first delayduring an intermediate period two stimuli appeared onthe screen a target (identical to the cued object) and adistracting stimulus with featural characteristics thatdistinguish it from the target The location of the targetstimulus to be matched is different from the position atwhich the initial cueing target appeared After that onlythe location of the matched target has to be encodedand the feature information can now be ignored Dur-ing the second delay period only the where-basedinformation (location) has to be retained Upon pre-sentation of the final probe one has to judge whether itoccupied the same location as the location of theretained matched target during the intermediate stim-ulus presentation In where-then-what trials one has toencode the spatial location of the initial cue ignoringthe featural characteristics of the object presented onthe screen and to retain in short-term memory only thewhere-based information (location) during the firstdelay Following this first delay during an intermediateperiod two stimuli appear on the screen one at theinitially cued location (target location) and the other atanother location Both stimuli have different featuralcharacteristics from each other The featural character-istics of the stimulus at the target location have to beencoded and retained during the second delay periodthat is only the what-based information (object) has to

be retained Upon presentation of the final probe onehas to judge whether the probe has the same featuralcharacteristics of the retained matched target duringthe intermediate stimulus presentation

We simulated this task In all our simulations we usedthe following parameters for the task (following Postleand DrsquoEsposito 1999 2000) Tfix = 1000 msec Tcue1 =1500 msec Tdelay1 = 6500 msec Tcue2 = 1500 msecTdelay2 = 6500 msec and Tresponse = 1500 msec

The Neurodynamical Model of thePrefrontal Cortex

We follow the theoretical framework for modeling asingle integrate-and-fire attractor network introducedand studied by Brunel and Wang (2001) and extend itto multiple hierarchically organized networks organizedinto a biased competition architecture introduced bythe authors (Corchs amp Deco 2002 Rolls amp Deco 2002Deco amp Lee 2002 Deco amp Zihl 2001) We incorporateshunting inhibition (Battaglia amp Treves 1998 Rolls ampTreves 1998) and inhibitory-to-inhibitory cell synapticconnections (Brunel amp Wang 2001) which are useful inmaintaining stability of the dynamical system and incor-porate appropriate currents to achieve low firing rates(Brunel amp Wang 2001 Amit amp Brunel 1997) Accordingto the experimental neurophysiological evidence of Raoet al (1997) we assume the existence of different typesof neuronal populations or pools that show either object-tuned (what) or location-tuned (where) or both whatand where tuned activity in the delay period We showthat local synaptic connections (which could be set up byassociative learning) between these neuronal pools aresufficient for operation of the model In this section wedescribe the architecture and operation of the modeland the neuronal parameters and equations used aregiven in Appendix A

The Neurons

We use leaky integrate-and-fire neurons for modelingthe excitatory pyramidal cells and the inhibitory inter-neurons The synaptic inputs to an integrate-and-fireneuron are basically described by a capacitor Cm

connected in parallel with a resistor Rm through whichcurrents are injected into the neuron These currentinjections produce excitatory or inhibitory postsynapticpotentials EPSPs or IPSPs respectively

These potentials are integrated by the cell and if athreshold u is reached a d-pulse (spike) is fired andtransmitted to other neurons and the potential of theneuron is reset The incoming presynaptic d-pulsecurrent from another neuron is first low-pass filteredby the synaptic and membrane time constants and isthen realized as an EPSP or IPSP in the one-compart-


ment neuronal model We use biologically realisticparameters (McCormick Connors Lighthall amp Prince1985) We take for both excitatory and inhibitory neu-rons a resting potential VL = iexcl70 mV a firing thresholdu = iexcl50 mV and a reset potential Vreset = iexcl55 mVThe membrane capacitance Cm is 05 nF for the pyra-midal neurons and 02 nF for the inhibitory interneur-ons The membrane leak conductance gm is 25 nS forpyramidal cells and 20 nS for interneurons The refrac-tory period tref is 2 msec for pyramidal cells and 1 msecfor interneurons Hence the membrane time constanttm = Cmgm is 20 msec for pyramidal cells and 10 msecfor interneurons respectively

The synaptic current flows into the cells are mediatedby three different families of receptors The recurrentexcitatory postsynaptic EPSPs are mediated by AMPAand NMDA receptors These two glutamatergic excit-atory synapses are on the pyramidal cells and on theinterneurons The external inputs (background sensoryinput or external top-down interaction from otherareas) are mediated by AMPA synapses on pyramidalcells and interneurons Inhibitory GABAergic synapseson pyramidal cells and interneurons yield the corre-sponding IPSPs The mathematical descriptions of eachsynaptic channel are provided in Appendix A and thecorresponding parameters are also specified there Weconsider that the NMDA currents have a voltage depen-dence that is controlled by the extracellular magnesiumconcentration (Jahr amp Stevens 1990) CMg2 + = 1 mMWe neglect the rise time of both AMPA and GABAsynaptic currents because they are typically extremelyshort (lt1 msec) The rise time for NMDA synapses istNMDArise = 2 msec (Spruston Jonas amp Sakmann 1995Hestrin Sah amp Nicoll 1990) All synapses have a latency(time delay) of 05 msec The decay time for AMPAsynapses is tAMPA= 2 msec (Spruston et al 1995 Hestrinet al 1990) for NMDA synapses tNMDAdecay = 100 msec(Spruston et al 1995 Hestrin et al 1990) and for GABAsynapses tGABA = 10 msec (Xiang Huguenard amp Prince1998 Salin amp Prince 1996) The synaptic conductivitiesfor each receptor type were taken from Brunel andWang (2001) and were adjusted using a mean fieldanalysis to be approximately 1 nS in magnitude thesewere consistent with experimentally observed values(Destexhe et al 1998) (see Appendix A) As was notedby Brunel and Wang (2001) Wang (1999) and LismanFellous and Wang (1998) the recurrent excitation wasassumed to be largely mediated by the NMDA receptorsin order to provide more robust persistent activityduring the short-term memory related delay periodand the amplitude of recurrent excitation was smallerthan that of inhibition therefore the net recurrent inputto a neuron was hyperpolarizing during spontaneousactivity (ie without external inputs) (Brunel amp Wang2001 Amit amp Brunel 1997) Figure 2 shows schematicallythe synaptic structure assumed in the prefrontal corticalnetwork

The Network Architecture

The network is composed of NE (excitatory) pyramidalcells and NI inhibitory interneurons In our simulationswe use NE = 1600 and NI = 400 consistent with theneurophysiologically observed proportion of 80 pyra-midal cells versus 20 interneurons (Abeles 1991) Theneurons are fully connected (with synaptic strengths asspecified below) Neurons in the prefrontal corticalnetwork shown in Figure 2 are clustered into popula-tions or pools Each pool of excitatory cells containsfNE neurons (in our simulations f = 005) There aretwo different types of pool excitatory and inhibitoryThere are four subtypes of excitatory pool namelyobject-tuned (what pools) space-tuned (where pools)object-and-space-tuned (what-and-where pools) andnonselective Object pools are feature specific encodingfor example the identity of an object (eg form coloretc) The spatial pools are location specific and encodethe spatial position of a stimulus The integrated object-and-space-tuned pools encode both specific feature andlocation information The remaining excitatory neuronsdo not have specific sensory response or biasing inputsand are in a nonselective pool (The neurons in thenonselective pool have some spontaneous firing andhelp to introduce some noise into the simulation whichaids in generating the almost Poisson spike firing pat-terns of neurons in the simulation that are a property ofmany neurons recorded in the brain (Brunel amp Wang2001) All the inhibitory neurons are clustered into acommon inhibitory pool so that there is global compe-tition throughout the network

We assume that the synaptic coupling strengths be-tween any two neurons in the network act as if theywere established by Hebbian learning that is the cou-pling will be strong if the pair of neurons have corre-lated activity and weak if they are activated in anuncorrelated way Because of this neurons within a spe-cific excitatory pool are mutually coupled with a strongweight ws = 21 Neurons in the inhibitory pool aremutually connected with an intermediate weight w = 1(forming the inhibitory to inhibitory connections thatare useful in achieving nonoscillatory firing) They arealso connected with all excitatory neurons with the sameintermediate weight w = 1 The connection strengthbetween two neurons in two different specific excitatorypools is weak and given by ww= 1 iexcl 2f(ws iexcl 1)(1 iexcl 2f )unless otherwise specified (see next paragraph) Neu-rons in a specific excitatory pool are connected toneurons in the nonselective pool with a feedforwardsynaptic weight w = 1 and a feedback synaptic connec-tion of weight ww

The connections between the different pools are setup so that specific integrated what-and-where pools areconnected with the corresponding specific what-tunedand where-tuned pools as if they were based on Heb-bian learning of the activity of individual pools while the


different tasks are being performed The forward con-nections (input to integrated what-and-where pools)are wf = 165 The corresponding feedback synapticconnections are symmetric (ie identical to the corre-sponding feedforward synaptic connections)

The strengths of the feedforward and feedbackconnections between different pools are indicated inTable 1 The neurons in the stimulus-domain-specificsensory pools have forward and backward strongweight connections with neurons with which they areassociated in the what-and-wherendashtuned pools In oursimulations we consider two feature-tuned pools F1 andF2 (corresponding to two possible objects eg object Aand B) two space-tuned pools S1 and S2 (correspondingto two locations eg left and right) and thecorresponding four integrated what-and-where pools(FSij) for each of the possible combinations of what (Fi)and where (Sj) sensory information coming from thedomain-specific pools

Each neuron (pyramidal cells and interneurons) re-ceives Next = 800 excitatory AMPA synaptic connectionsfrom outside the network These connections providethree different types of external interactions (1) abackground noise due to the spontaneous firing activityof neurons outside the network (2) a sensory-relatedinput (object- or space-specific) and (3) an attentionalbias that specifies the task (what- or where-delayedresponse) The external inputs are given by a Poissontrain of spikes In order to model the backgroundspontaneous activity of neurons in the network (Brunelamp Wang 2001) we assume that Poisson spikes arrive at

each external synapse with a rate of 3 Hz consistentwith the spontaneous activity observed in the cerebralcortex (Rolls amp Treves 1998 Wilson et al 1994) Inother words the effective external spontaneous back-ground input rate of spikes to each cell is next = Next pound3 Hz = 24 kHz The sensory input is encoded byincreasing the external input Poisson rate next to next +linput to the neurons in the appropriate specific sensorypools (Brunel amp Wang 2001) For example if thestimulus is defined as an object with a feature charac-teristic Fi and at a spatial location Sj then the neuronsin the sensory object pool Fi and in the spatial sensorypool Sj will receive the increased external Poisson inputjust defined We used linput = 85 Hz Finally theattentional biasing specification of the task (ie whichdimension is relevant) is modeled by assuming that eachneuron in each of the pools associated with the relevantstimulus-domain (object or space) receives externalPoisson spikes with an increased rate from next tonext + latt throughout the trial We used latt = 85 HzThis external top-down domain-specific input probablycomes from the external prefrontal neurons that directlyencode abstract rules (Wallis Anderson amp Miller 2001)which in turn are influenced by the reward system (in theorbitofrontal cortex and amygdala [Rolls 1999]) whichenables the correct context to be selected During thelast 100 msec of the response period the external rate toall neurons is increased by a factor 15 in order to takeinto account the increase in afferent inputs due tobehavioral responses and reward signals (Brunel ampWang 2001)

The cortical architecture introduced above has thecharacteristic that its different global attractors corre-sponding to the different domain-specific relevant senso-ry situations are each composed of a set of single-poolattractors where the single pools (each pool is a pop-ulation of neurons) that are active represent a particularcombination of what-tuned where-tuned and what-and-wherendashtuned pools The cue stimulus and the biasingtop-down attentional information drive the system intothe corresponding attractor In fact the system is dy-namically driven according to the biased competitionhypothesis (Reynolds amp Desimone 1999 Chelazzi et al1993 Chelazzi 1998 Miller et al 1993 Motter 1993Spitzer et al 1988 Moran amp Desimone 1985) Multipleexcitatory pools of neurons activated by the sensory cuestimulus engage in competitive interactions using theinterneurons to implement the global competition Theexternal top-down interactions bias this competitionin favor of specific pools resulting in the buildup ofthe global attractor that corresponds to the stimulus-domain-specific situation required In this way irrele-vant sensory information will be suppressed by theunderlying neurodynamics implementing a form ofinternal prefrontal attentional system which is the basisof the attentional top-down bias transmitted to posteriorperceptual areas (Rolls amp Deco 2002)

Table 1 Neuronal Connectivity Between Different NeuronalPools in the Architecture of Figure 2

Pools F1 F2 S1 S2 FS1 1 FS1 2 FS2 1 FS2 2 Unsp Inh

F1 ws ww ww ww wf wf ww ww 1 1

F2 ww ws ww ww ww ww wf wf 1 1

S1 ww ww ws ww wf ww wf ww 1 1

S2 ww ww ww ws ww wf ww wf 1 1

FS1 1 wf ww wf ww ws ww ws ww 1 1

FS1 2 wf ww ww wf ww ws ww ww 1 1

FS2 1 ww wf wf ww ww ww ws ww 1 1

FS2 2 ww wf ww wf ww ww ww ws 1 1

Unsp ww ww ww ww ww ww ww ww 1 1

Inh 1 1 1 1 1 1 1 1 1 1

F1 and F2 object-tuned pools (corresponding to two possible objectseg Object A and B) S1 and S2 space-tuned pools (correspondingto two locations eg left and right) FSij combination-tuned what-and-here neuronal pools for each of the possible combinationsof lsquolsquowhatrsquorsquo (Fi) and lsquolsquowherersquorsquo (Sj) sensory information coming from thedomain-specific pools Unsp = nonspecific neuronal pool Inh =inhibitory neuron pool


All neuronal and synaptic equations were integratedusing the second-order RungendashKutta method with anintegration step of dt = 01 msec Checks were per-formed to show that this was sufficiently smallFor the neural membrane potential equations interpo-lation of the spike time and their use in the synapticcurrents and potentials were taken into account fol-lowing the prescription of Hansel Mato Meunier andNeltner (1998) in order to avoid numerical problemsdue to the discontinuity of the membrane potentialand its derivative at the spike firing time The externaltrains of Poisson spikes were generated randomly andindependently

Simulation of the Event-related fMRI ResponseHemodynamic Convolution of Synaptic Activity

The links between neural and synaptic activity andfMRI measurements are still not fully understood ThefMRI signal is unfortunately strongly filtered and per-turbed by the hemodynamic delay inherent in theBOLD contrast mechanism (Buxton amp Frank 1997)The fMRI signal is only a secondary consequence ofneuronal activity and yields therefore a blurred distor-tion of the temporal development of the underlyingbrain processes Regionally increased oxidative metab-olism causes a transient decrease in oxyhemoglobinand increase in deoxyhemoglobin as well as an in-crease in CO2 and NO This provokes over severalseconds a local dilatation and increased blood flow inthe affected regions that leads by overcompensation toa relative decrease in the concentration of deoxyhe-moglobin in the venules draining the activated regionthe alteration of deoxyhemoglobin which is paramag-netic can be detected by changes in T2 or T2 in theMRI signal as a result of the decreased susceptibilityand thus decreased local inhomogeneity which in-creases the MR intensity value (Glover 1999 Buxtonamp Frank 1997 Buxton Wong amp Frank 1998) Glover(1999) demonstrated that a good fit of the hemody-namical response can be achieved by the followinganalytic function

hhelliptdagger ˆ c1t nieiexcl tt1 iexcl a2c2t n2eiexcl t

t2

ci ˆ maxhellipt ni eiexcl tt1 dagger

where t is the time and c1 c2 a2 n1 and n2 areparameters that are adjusted to fit the experimentalmeasured hemodynamical response

Recently the putative neural basis of the BOLD signalhas been explored by Logothetis et al (2001) Theyanalyzed simultaneously recorded neuronal and fMRIresponses from the visual cortex of monkeys and

observed that the largest magnitude changes wereobserved in local field potentials (LFPs) which at re-cording sites characterized by transient responses werethe only signal that significantly correlated with thehemodynamic responses These findings suggest thatthe BOLD contrast mechanism reflects the synapticinput and intracortical processing of a given area ratherthan its spiking output

Based on these results and similar to Horwitz andTagamets (1999) we simulate the temporal evolutionof fMRI signals by convolving the total synaptic activitywith the standard hemodynamic response formulationof Glover (1999) presented above The total synapticcurrent (Isyn) is given by the sum of the absolutevalues of the glutamatergic excitatory components(implemented through NMDA and AMPA receptors)and inhibitory components (GABA) (Tagamets amp Hor-witz 1998) As described above we consider thatexternal excitatory contributions are produced throughAMPA receptors (IAMPAext) while the excitatory recur-rent synapses are produced through AMPA and NMDAreceptors (IAMPArec and INMDArec) The GABA inhibitorycurrents are denoted by IGABA (see Appendix A fordetails) Consequently the fMRI signal activity is cal-culated by the following convolution equation

Sf MRIhelliptdagger ˆZ 1

0hhellipt iexcl t 0daggerIsynhellipt 0daggerdt0

In our simulation we calculated numerically theconvolution by sampling the total synaptic activity every01 sec and introducing a cutoff at a delay of 25 sec Theparameters utilized for the hemodynamic standard re-sponse h(t) were taken from the article of Glover (1999)and were n1 = 60 t1 = 09 sec n2 = 120 t2 = 09 secand a2 = 02 Figure 3 plots the hemodynamic standardresponse h(t) for this set of parameters

APPENDIX A

In this appendix we give the mathematical equationsthat describe the spiking activity and synapse dynamicsin the network following in general the formulationdescribed by Brunel and Wang (2001) Each neuron isdescribed by an integrate-and-fire model The subthresh-old membrane V(t) potential of each neuron evolvesaccording to the following equation

CmdVhelliptdagger

dtˆ iexclgmhellipVhelliptdagger iexcl VLdagger iexcl Isynhelliptdagger hellip1dagger

where Isyn(t) is the total synaptic current flow into thecell When the membrane potential V(t) reaches thethreshold u a spike is generated and the membranepotential is reset to Vreset The neuron is unable to


spike during the first tref which is the absolute refrac-tory period

The total synaptic current is given by the sum ofglutamatergic excitatory components (NMDA and AMPA)and inhibitory components (GABA) As we describedabove we consider that external excitatory contributionsare produced through AMPA receptors (IAMPAext) whilethe excitatory recurrent synapses are produced throughAMPA and NMDA receptors (IAMPArec and INMDArec) Thetotal synaptic current is therefore given by

Isynhelliptdagger ˆ IAMPAexthelliptdagger Dagger IAMPArechelliptdagger Dagger INMDArechelliptdaggertDagger IGABA helliptdagger

hellip2dagger

where

IAMPAexthelliptdagger ˆ gAMPAexthellipVhelliptdagger iexcl VEdaggerXNext

jˆ1

sAMPAextj helliptdagger

hellip3dagger

IAMPArechelliptdagger ˆ gAMPArechellipVhelliptdagger iexcl VEdaggerXNE

jˆ1

wj

sAMPArecj helliptdagger hellip4dagger

INMDArechelliptdagger ˆgNMDArechellipV helliptdagger iexcl VEdagger

hellip1 Dagger CMgDaggerDagger exphellipiexcl0062Vhelliptdaggerdagger=357dagger

poundXNE

jˆ1

wjsNMDArecj helliptdagger

hellip5dagger

IGABAhelliptdagger ˆ gGABAhellipV helliptdagger iexcl VIdaggerXN1

jˆ1

sGABAj helliptdagger hellip6dagger

In the preceding equations VE = 0 mV and VT =iexcl70 mV The synaptic strengths wj are specified inMethods and in Table 1 The fractions of openchannels are given by

dsAMPAextj helliptdagger

dtˆ iexcl


tAMPADagger

X

k

dhellipt iexcl t kj dagger hellip7dagger

dsAMPArecj helliptdagger

dtˆ iexcl

sAMPArecj helliptdagger

tAMPADagger

X

k


dsNMDArecj helliptdagger

dtˆ iexcl

sNMDArecj helliptdagger

tNMDAdecayDagger axjhelliptdagger

pound hellip1 iexcl sNMDArecj helliptdaggerdagger hellip9dagger

dxjhelliptdaggerdt

ˆ iexclxjhelliptdagger

tNMDAriseDagger

X

k


dsGABAj helliptdagger

dtˆ iexcl

sGABAj helliptdaggertGABA

DaggerX

k


where the sums over k represent a sum over spikesemitted by presynaptic neuron j at time tj

k The valueof a = 05 mseciexcl1

The values of the conductances for pyramidal neu-rons were gAMPAext = 208 gAMPArec = 0052 gNMDArec =0164 and gGABA = 067 and for interneurons weregAMPAext = 162 gAMPArec = 00405 gNMDArec = 0129and gGABA = 049

Acknowledgments

Gustavo Deco acknowledges support through the German Mi-nistry for Research BMBF Grant 01IBC01A and through theEuropean Union grant IST-2001-38099

Reprint requests should be sent to Edmund Rolls via e-mail(EdmundRollspsyoxacuk httpwwwcnsoxacuk) or toBarry Horwitz Brain Imaging and Modeling Section NationalInstitute on Deafness and other Communication DisordersNational Institutes of Health Bethesda MD 20892 USA

REFERENCES

Abeles A (1991) Corticonics New York CambridgeUniversity Press

Amit D amp Brunel N (1997) Model of global spontaneousactivity and local structured activity during delayperiods in the cerebral cortex Cerebral Cortex 7237ndash252

Asaad W F Rainer G amp Miller E K (1998) Neural activity inthe primate prefrontal cortex during associative learningNeuron 21 1399ndash1407

Asaad W F Rainer G amp Miller E K (2000) Task-specificneural activity in the primate prefrontal cortex Journal ofNeurophysiology 84 451ndash459

Baddeley A (1986) Working memory New York OxfordUniversity Press

Battaglia F amp Treves A (1998) Stable and rapid recurrentprocessing in realistic autoassociative memories NeuralComputation 10 431ndash450

Brunel N amp Wang X (2001) Effects of neuromodulationin a cortical networks model of object working memorydominated by recurrent inhibition Journal ofComputational Neuroscience 11 63ndash85

Buxton R B amp Frank L R (1997) A model for the coupling


between cerebral blood flow and oxygen metabolism duringneural stimulation Journal of Cerebral Blood Flow andMetabolism 17 64ndash72

Buxton R B Wong E C amp Frank L R (1998) Dynamics ofblood flow and oxygenation changes during brain activationThe balloon model Magnetic Resonance in Medicine 39855ndash864

Chelazzi L (1998) Serial attention mechanisms in visualsearch A critical look at the evidence PsychologicalResearch 62 195ndash219

Chelazzi L Miller E Duncan J amp Desimone R (1993)A neural basis for visual search in inferior temporal cortexNature (London) 363 345ndash347

Corchs S amp Deco G (2002) Large-scale neural model forvisual attention Integration of experimental single cell andfMRI data Cerebral Cortex 12 339ndash348

Deco G amp Lee T (2002) A unified model of spatial andobject attention based on inter-cortical biased competitionNeurocomputing 44ndash46 775ndash781

Deco G amp Zihl J (2001) Top-down selective visualattention A neurodynamical approach Visual Cognition8 119ndash140

DrsquoEsposito M Aguirre G K Zarahn E Ballard D Shin RK amp Lease J (1998) Functional MRI studies of spatialand nonspatial working memory Cognitive Brain Research7 1ndash13

Destexhe A Mainen Z amp Sejnowski T (1998) Kineticmodels of synaptic transmission In C Koch amp I Segev(Eds) Methods in neural modeling From ions to networks(2nd ed pp 1ndash25) Cambridge MIT Press

Funahashi S Bruce C amp Goldman-Rakic P (1989)Mnemonic coding of visual space in the monkeyrsquosdorsolateral prefrontal cortex Journal of Neurophysiology61 331ndash349

Fuster J (2000) Executive frontal functions ExperimentalBrain Research 133 66ndash70

Fuster J M Bauer R H amp Jervey J P (1982) Cellulardischarge in the dorsolateral prefrontal cortex of themonkey in cognitive tasks Experimental Neurology 77679ndash694

Glover G H (1999) Deconvolution of impulse response inevent-related BOLD fMRI Neuroimage 9 416ndash429

Goel V amp Grafman J (1995) Are the frontal lobes implicatedin lsquolsquoplanningrsquorsquo functions Interpreting data from the Towerof Hanoi Neuropsychologia 33 632ndash642

Goldman-Rakic P (1987) Circuitry of primate prefrontalcortex and regulation of behavior by representationalmemory In F Plum amp V Mountcastle (Eds) Handbook ofphysiologymdashThe nervous system (pp 373ndash417) BethesdaMD American Physiological Society

Goldman-Rakic P (1995) Cellular basis of working memoryNeuron 14 477ndash485

Goldman-Rakic P (1996) Regional and cellular fractionationof working memory Proceedings of the National Academyof Sciences USA 93 13473ndash13480

Hansel D Mato G Meunier C amp Neltner L (1998) Onnumerical simulations of integrate-and-fire neural networksNeural Computation 10 467ndash483

Hestrin S Sah P amp Nicoll R (1990) Mechanisms generatingthe time course of dual component excitatory synapticcurrents recorded in hippocampal slices Neuron 5247ndash253

Horwitz B amp Tagamets M-A (1999) Predicting humanfunctional maps with neural net modeling Human BrainMapping 8 137ndash142

Horwitz B Friston K J amp Taylor J G (2000) Neuralmodeling and functional brain imaging An overview NeuralNetworks 13 829ndash846

Horwitz B Tagamets M-A amp McIntosh A R (1999) Neuralmodeling functional brain imaging and cognition Trendsin Cognitive Sciences 3 85ndash122

Hoshi E Shima K amp Tanji J (1998) Task-dependentselectivity of movement-related neuronal activity in theprimate prefrontal cortex Journal of Neurophysiology 803392ndash3397

Jahr C amp Stevens C (1990) Voltage dependence ofNMDA-activated macroscopic conductances predicted bysingle-channel kinetics Journal of Neuroscience 103178ndash3182

Jueptner M amp Weiller C (1995) Does measurement ofregional cerebral blood flow ref lect synapticactivitymdashImplications for PET and fMRI Neuroimage 2148ndash156

Lauritzen M (2001) Relationship of spikes synapticactivity and local changes of cerebral blood flowJournal of Cerebral Blood Flow and Metabolism 211367ndash1383

Leung H Gore J amp Goldman-Rakic P (2002) Sustainedmnemonic response in the human middle frontal gyrusduring on-line storage of spatial memoranda Journal ofCognitive Neuroscience 14 659ndash671

Levy R amp Goldman-Rakic P (1999) Executive frontalfunctions Journal of Neuroscience 19 5149ndash5158

Lisman J E Fellous J M amp Wang X J (1998) A role forNMDA-receptor channels in working memory NatureNeuroscience 1 273ndash275

Logothetis N K Pauls J Augath M Trinath T ampOeltermann A (2001) Neurophysiological investigation ofthe basis of the fMRI signal Nature 412 150ndash157

McCormick D Connors B Lighthall J amp Prince D (1985)Comparative electrophysiology of pyramidal and sparselyspiny stellate neurons in the neocortex Journal ofNeurophysiology 54 782ndash806

Miller E Gochin P amp Gross C (1993) Suppression of visualresponses of neurons in inferior temporal cortex of theawake macaque by addition of a second stimulus BrainResearch 616 25ndash29

Miller E K (2000) The prefrontal cortex and cognitivecontrol Nature Reviews Neuroscience 1 59ndash65

Moran J amp Desimone R (1985) Selective attention gatesvisual processing in the extrastriate cortex Science 229782ndash784

Motter B (1993) Focal attention produces spatially selectiveprocessing in visual cortical areas V1 V2 and V4 in thepresence of competing stimuli Journal of Neurophysiology70 909ndash919

Owen A M Herrod N J Menon D K Clark C J DowneyS P M J Carpenter T A Minhas P S Turkheimer F EWilliams E J Robbins T W Sahakian B J Petrides Mamp Pickard J (1999) Redefining the functional organizationof working memory processes within human lateralprefrontal cortex European Journal of Neuroscience 11567ndash574

Petrides M (1994) Frontal lobes and behaviour CurrentOpinion in Neurobiology 4 207ndash211

Postle B R amp DrsquoEsposito M (1999) lsquolsquoWhatrsquorsquo-then-lsquolsquoWherersquorsquoin visual working memory An event-related fMRI studyJournal of Cognitive Neuroscience 11 585ndash597

Postle B R amp DrsquoEsposito M (2000) Evaluating models of thetopographical organization of working memory function infrontal cortex with event-related fMRI Psychobiology 28132ndash145

Rao S Rainer G amp Miller E (1997) Integration of what andwhere in the primate prefrontal cortex Science 276821ndash824

Reynolds J amp Desimone R (1999) The role of neural


mechanisms of attention in solving the binding problemNeuron 24 19ndash29

Rolls E T (1999) The brain and emotion Oxford OxfordUniversity Press

Rolls E T amp Deco G (2002) Computational neuroscienceof vision Oxford Oxford University Press

Rolls E T amp Treves A (1998) Neural networks and brainfunction Oxford Oxford University Press

Rolls E T Stringer S M amp Trappenberg T P (2002)A unified model of spatial and episodic memoryProceedings of the Royal Society of London Series B269 1087ndash1093

Salin P amp Prince D (1996) Spontaneous GABA-A receptormediated inhibitory currents in adult rat somatosensorycortex Journal of Neurophysiology 75 1573ndash1588

Spitzer H Desimone R amp Moran J (1988) Increasedattention enhances both behavioral and neuronalperformance Science 240 338ndash340

Spruston N Jonas P amp Sakmann B (1995) Dendriticglutamate receptor channel in rat hippocampal CA3 andCA1 pyramidal neurons Journal of Physiology 482325ndash352

Tagamets M amp Horwitz B (1998) Integrating electrophysicaland anatomical experimental data to create a large-scalemodel that simulates a delayed match-to-sample humanbrain study Cerebral Cortex 8 310ndash320

Ungerleider L Courtney S amp Haxby J (1998) A neuralsystem for human visual working memory Proceedingsof the National Academy of Sciences USA 95883ndash890

Wallis J Anderson K amp Miller E (2001) Single neurons inprefrontal cortex encode abstract rules Nature 411953ndash956

Wang X (1999) Synaptic basis of cortical persistent activityThe importance of NMDA receptors to working memoryJournal of Neuroscience 19 9587ndash9603

White I amp Wise S (1999) Rule-dependent neuronal activityin the prefrontal cortex Experimental Brain Research 126315ndash335

Wilson F A W OrsquoScalaidhe S P amp Goldman-Rakic P S(1993) Dissociation of object and spatial processingdomains in primate prefrontal cortex Science 2601955ndash1958

Wilson F A W OrsquoScalaidhe S P amp Goldman-Rakic P (1994)Functional synergism between putative gamma-aminobutyrate-containing neurons and pyramidal neuronsin prefrontal cortex Proceedings of the National Academyof Sciences USA 91 4009ndash4013

Xiang Z Huguenard J amp Prince D (1998) GABA-Areceptor mediated currents in interneurons and pyramidalcells of rat visual cortex Journal of Physiology 506715ndash730


support this topographical organization (but see below)(Leung Gore amp Goldman-Rakic 2002 Wilson OrsquoSca-laidhe amp Goldman-Rakic 1993 Goldman-Rakic 1987Fuster Bauer amp Jervey 1982) A second model pro-poses a hierarchical organization of the PFC with non-mnemonic higher order functions (eg manipulation ofitems in memory) ascribed to dorsolateral prefrontalareas and short-term memory maintenance functionsascribed to inferior prefrontal areas (Owen et al 1999DrsquoEsposito et al 1998 Petrides 1994) Consistent withthis second model some event-related fMRI studies(Postle amp DrsquoEsposito 1999 2000) with a what-then-where design failed to find evidence for anatomicalsegregation of spatial and object visual working memoryin the PFC of humans

Analysis of how the PFC implements working memoryhas been extended by analyzing neuronal activity whena monkey performs a delay task with a what-then-wheredesign (ie the monkey performs an object and spatialdelayed response within the same trial) With this kindof experimental paradigm Rao Rainer and Miller(1997) found some neurons that showed either object-tuned (what) or location-tuned (where) delay activityHowever a large percentage of the measured PFCneurons (52) showed both what and where tuningSimilarly in delay conditional response and delayedspatial response tasks some neurons respond in thedelay period to the stimulus object some to the stim-ulus position some to combinations of the stimulusobject and its position and some to the responserequired with no clear topographical separation re-ported (White amp Wise 1999 Asaad Rainer amp Miller1998 2000 Hoshi Shima amp Tanji 1998)

Taken together all these results imply that thetopographical and functional organization of the PFCin relation to different types of working memory is stillinsufficiently understood To elucidate the underlyingmechanisms and the fMRI signals that can result inhumans we describe here a computational model ofthe PFC that can perform what-then-where and where-then-what memory tasks of the type illustrated inFigure 1 (see Methods for details) and has spikingneurons whose simulated activities are directly compa-rable to the single-neuron data recorded in the PFC(Rao et al 1997) The integrate-and-fire neurons areorganized into a hierarchical set of attractor or autoas-sociation networks each implementing a short-termmemory as illustrated in Figure 2 (see Methods fordetails and Rolls amp Treves 1998 Rolls amp Deco 2002)There are separate attractors for lsquolsquowhatrsquorsquo stimulus wasshown for lsquolsquowherersquorsquo the stimulus was shown and acombined lsquolsquowhatndashwherersquorsquo attractor pool of neuronsThe relative activity of the different attractor poolsrequired to perform the what-then-where and where-then-what short-term memory tasks is provided by anexternal attentional signal that biases the differentpools of neurons within the general framework of the

biased competition model of attention (Rolls amp Deco2002 Reynolds amp Desimone 1999 Chelazzi MillerDuncan amp Desimone 1993 Chelazzi 1998 MillerGochin and Cross 1993 Motter 1993 Spitzer Desi-mone amp Moran 1988 Moran amp Desimone 1985) Rollsand Deco (2002) added to this framework by introduc-ing a neurodynamical theoretical framework for biasedcompetition which enables the whole framework tooperate from the level of integrate-and-fire neurons upto global phenomena such as short-term memory andattention Because the model is implemented at theintegrate-and-fire single-neuron level with the biophys-ical properties of the synapses part of the simulationwe are then able as described in this article tointegrate over the total synaptic activity within an areaand convolve with the hemodynamic response functionillustrated in Figure 3 (see Methods for details andRolls and Deco 2002 Horwitz Tagamets amp McIntosh1999 Horwitz Friston amp Taylor 2000) in order tomodel the fMRI blood oxygen level-dependent (BOLD)signal and simulate human neuroimaging experimentsBasing the link to the BOLD signal on the total synapticactivity (in fact current) within a region is appropriatebecause channels are opened by synaptic activity thatrequire ions to be pumped back against electrochem-ical gradients and this is the very energy-intensiveaspect of neuronal activity Experimental support forthis idea has been provided by Logothetis PaulsAugath Trinath and Oeltermann (2001) (for reviewssee Lauritzen 2001 Jueptner amp Weiller 1995)

RESULTS

Single-Neuron Recordings

In this subsection we present a theoretical analysis ofneuronal activity in the primate PFC underlying theexecution of a what-then-where working memory taskThe neuronal recordings of Rao et al (1997) demonstrat-ed the existence of neurons showing stimulus-domain-specific sensitivity that is object-tuned activity in thewhat-delay and location-tuned activity in the where-delay but they found also a large proportion of neuronsshowing integrated what-and-wherendashtuned activity dur-ing both what- and where-delays During each trial oftheir experiment while the monkey maintained fixationon a center spot a sample object was briefly presentedon the screen After a first delay (what-delay) two testobjects were briefly presented at two of four possibleextrafoveal locations One of the test objects matchedthe sample and the other was a nonmatch After asecond delay (where-delay) the monkey had to respondwith a saccade to the remembered location of the matchFigures 4A and 5A illustrate the neuronal recordings ofRao et al

The simulation of this experiment starts with a precueperiod of 1000 msec in which the network exhibitsspontaneous activity (3 Hz for the excitatory pools and





















































DISCUSSION










METHODS







The Neurons




























Inh 1 1 1 1 1 1 1 1 1 1







t2









APPENDIX A


CmdVhelliptdagger







hellip2dagger

where


jˆ1


hellip3dagger


jˆ1

wj




poundXNE

jˆ1


hellip5dagger


jˆ1




dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl




dxjhelliptdaggerdt


tNMDAriseDagger

X

k



dtˆ iexcl


DaggerX

k





Acknowledgments



REFERENCES
























































































































DISCUSSION










METHODS







The Neurons




























Inh 1 1 1 1 1 1 1 1 1 1







t2









APPENDIX A


CmdVhelliptdagger







hellip2dagger

where


jˆ1


hellip3dagger


jˆ1

wj




poundXNE

jˆ1


hellip5dagger


jˆ1




dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl




dxjhelliptdaggerdt


tNMDAriseDagger

X

k



dtˆ iexcl


DaggerX

k





Acknowledgments



REFERENCES











































































METHODS







The Neurons




























Inh 1 1 1 1 1 1 1 1 1 1







t2









APPENDIX A


CmdVhelliptdagger







hellip2dagger

where


jˆ1


hellip3dagger


jˆ1

wj




poundXNE

jˆ1


hellip5dagger


jˆ1




dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl




dxjhelliptdaggerdt


tNMDAriseDagger

X

k



dtˆ iexcl


DaggerX

k





Acknowledgments



REFERENCES




























































































Inh 1 1 1 1 1 1 1 1 1 1







t2









APPENDIX A


CmdVhelliptdagger







hellip2dagger

where


jˆ1


hellip3dagger


jˆ1

wj




poundXNE

jˆ1


hellip5dagger


jˆ1




dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl




dxjhelliptdaggerdt


tNMDAriseDagger

X

k



dtˆ iexcl


DaggerX

k





Acknowledgments



REFERENCES









































































t2









APPENDIX A


CmdVhelliptdagger







hellip2dagger

where


jˆ1


hellip3dagger


jˆ1

wj




poundXNE

jˆ1


hellip5dagger


jˆ1




dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl




dxjhelliptdaggerdt


tNMDAriseDagger

X

k



dtˆ iexcl


DaggerX

k





Acknowledgments



REFERENCES








































































hellip2dagger

where


jˆ1


hellip3dagger


jˆ1

wj




poundXNE

jˆ1


hellip5dagger


jˆ1




dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl


tAMPADagger

X

k



dtˆ iexcl




dxjhelliptdaggerdt


tNMDAriseDagger

X

k



dtˆ iexcl


DaggerX

k





Acknowledgments



REFERENCES





































































“What” and “Where” in Visual Working Memory: A Computational Neurodynamical Perspective for...

Documents

Transcript of “What” and “Where” in Visual Working Memory: A Computational Neurodynamical Perspective for...