Reward modulates attention independently of action value inposterior parietal cortex

Christopher J. Peck1,*, David C. Jangraw1,*, Mototaka Suzuki1, Richard Efem1, andJacqueline Gottlieb1,21Department of Neuroscience, Columbia University2Department of Psychiatry, Columbia University

AbstractWhile numerous studies explored the mechanisms of reward-based decisions (the choice of actionbased on expected gain), few asked how reward influences attention (the selection of informationrelevant for a decision). Here we show that a powerful determinant of attentional priority is theassociation between a stimulus and an appetitive reward. A peripheral cue heralded the delivery ofreward (RC+) or no reward (RC−); to experience the predicted outcome monkeys made a saccadeto a target that appeared unpredictably at the same or opposite location relative to the cue. Althoughthe RC had no operant associations (did not specify the required saccade) they automatically biasedattention, such that the RC+ attracted attention and RC− repelled attention from their location.Neurons in the lateral intraparietal area (LIP) encoded these attentional biases, maintaining sustainedexcitation at the location of an RC+ and inhibition at the location of an RC−. Contrary to thehypothesis that LIP encodes action value, neurons did not encode the expected reward of the saccade.Moreover, the cue-evoked biases were maladaptive, interfering with the required saccade, and theybiases increased rather than abating with training, strikingly at odds with an adaptive decision process.After prolonged training valence selectivity appeared at shorter latencies and automaticallytransferred to a novel task context, suggesting that training produced visual plasticity. The resultssuggest that reward predictors gain automatic attentional priority regardless of their operantassociations, and this valence-specific priority is encoded in LIP independently of the expectedreward of an action.

Keywordsparietal cortex; attention; reward; plasticity; eye movements; learning

INTRODUCTIONA central question in neuroscience concerns the mechanisms by which animals make reward-based decisions (Sutton and Barto, 1998; Sugrue et al., 2005; Bogacz, 2007). A system of

Address for correspondence: Jacqueline Gottlieb, PhD, Department of Neuroscience, Columbia University, 1051 Riverside Drive, Unit87, New York, NY 10032, Phone: 212-543-6931, ext. 500, Fax: 212-543-5816, jg2141@columbia.edu.*These authors contributed equally to this reportAUTHOR CONTRIBUTIONSC.J.P. and D.C.J conducted the experiment and analyzed the data. M.S. contributed to data collection and analysis. R.E. performed initialbehavioral training. J.G. conceived the experiments and wrote the manuscript.COMPETING INTERESTS STATEMENTThe authors declare that they have no competing financial interests.Note: Supplementary information is available on the Journal of Neuroscience website.

NIH Public AccessAuthor ManuscriptJ Neurosci. Author manuscript; available in PMC 2010 March 9.

Published in final edited form as:J Neurosci. 2009 September 9; 29(36): 11182–11191. doi:10.1523/JNEUROSCI.1929-09.2009.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

choice for the study of decision making has been the oculomotor system, in particular themechanisms guiding rapid eye movements (saccades). Many experiments have focused on thelateral intraparietal area (LIP), a cortical area that has a spatiotopic visual representation andis implicated in attention and saccade planning. LIP neurons encode the direction of anupcoming saccade, and their pre-saccadic responses are scaled by expected reward, suggestingthat LIP encodes a representation of action value that specifies the metrics and expected gainof a potential saccade (Platt and Glimcher, 1999; Sugrue et al., 2004).

However, saccades are closely related with attention, and prior studies did not distinguishwhether reward modulates attention or saccade decisions. This distinction is significantbecause attention is important for monitoring informative or salient objects that may beunrelated to immediate action. In natural behavior, decisions are forged in the presence ofmultiple extraneous stimuli that have no immediate operant significance. If these stimuli biasattention automatically by virtue of their reward associations, this may impair rather thanfacilitate a desired action. LIP neurons respond robustly to covertly attended objects (Oristaglioet al., 2006; Balan et al., 2008) and encode the ability of salient distractors to interfere with anongoing task (Bisley and Goldberg, 2003; Balan and Gottlieb, 2006; Ipata et al., 2006). Thus,it is important to understand whether neurons specifically encode the expected reward of anaction or whether they encode a visual signal that automatically encodes the salience of rewardpredictors independently of the operant significance of those predictors. These possibilitiescould not be disambiguated in prior studies, because in those studies all the visual stimuli inthe display were part of the decision set (each stimulus represented a saccade alternative),confounding an attentional and decisional interpertation (Platt and Glimcher, 1999; Sugrue etal., 2004).

To address this question, we used a novel task in which a peripheral visual cue (RC) predictedthe trial’s outcome and, to experience the expected outcome monkeys made a saccade toseparate target whose location was independent of the cue. We report that, even though the RChad no operant significance, they automatically biased attention in valence-specific manner,and these biases were encoded in LIP. Cues predicting reward attracted attention to theirlocation and evoked sustained excitation in LIP, while cues predicting no reward repulsedattention and evoked sustained inhibition in LIP. These biases were maladaptive, as theyinterfered with the required (optimal) saccade to the target. And yet, strikingly at odds with anadaptive decision process, these biases grew rather than abating after prolonged training. Theresults suggest that LIP encodes the power of reward predictors to bias attention in valence-specific manner whether or not these biases reflect the expected reward of an action.

METHODSGeneral Methods

Data were collected from two adult male rhesus monkeys (Macaca mulatta) using standardbehavioral and neurophysiological techniques as described previously (Oristaglio et al.,2006). Visual stimuli were presented on a SONY GDM-FW9000 Trinitron monitor (30.8 by48.2 cm viewing area) located 57 cm in front of the monkey. The precise timing of stimuluspresentation was measured accurately using a diode fixed to the top left corner of the monitorto detect the onset of a refresh cycle. Licking was measured by means of an infrared beam thatwas projected between the monkey’s mouth and the reward spout, and produced a TTL pulseeach time it was interrupted by protrusions of the monkey’s tongue. Eye position was recordedusing an eye coil system and digitized at 500 Hz. All methods were approved by the AnimalCare and Use Committees of Columbia University and New York State Psychiatric Instituteas complying with the guidelines within the Public Health Service Guide for the Care and Useof Laboratory Animals.

Peck et al. Page 2

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Behavioral TaskDuring the task, two placeholders were continuously present, positioned so they fell in the RFand at the opposite location when the monkey achieved central fixation (Fig. 1a). After themonkey achieved fixation, a reward cue (RC) was then presented for 300 ms either at the RFplaceholder or at the opposite location. Some RC indicated that the trial will end in juice reward(designated as RC+), while others indicated that the trial, even if correctly performed, will endin no reward (designated as RC−). The RC was followed by a 600 ms delay period duringwhich monkeys had to maintain fixation. At the end of the delay the fixation point was removedand one of the two placeholders (randomly selected) brightened; monkeys had to make asaccade to this placeholder in order to complete the trial. On RC+ trials a reward of constantsize (250 ms solenoid open time) was delivered at 350 ms after the end of a correct saccade.On an unrewarded (RC−) trial there was no juice reward, but a 600 ms post-saccade delay wasapplied to these trials in order to equate the total trial length across RC conditions. Error trials(premature or late saccades, or saccades away from the target) were immediately repeated untilcorrectly performed.

Reward cues (RC) were abstract computer-generated wireframe figures of distinct shape andcolor, approximately equated for size and luminance. Stimuli were scaled with retinaleccentricity to range from 1.5° to 3.0° in height and 1.0° to 2.0° in width. The fixation pointwas a 0.5° × 0.5° square and fixation was enforced within 2.5° of the fixation point and 3° ofthe saccade target. The fixation and saccade windows were constant across trials, so thataccuracy requirements did not differ according to reward condition. Eight RC (2 of each, over-learned RC+, over-learned RC−, newly-learned RC+ and newly-learned RC−) were presentedin random order for a total of 32 correct trials per RC or 256 trials per block.

Neural recordingsElectrode tracks were aimed to the lateral bank of the intraparietal sulcus based on stereotacticcoordinates and structural MRI. Neurons were tested on the task if they had spatially-tunedvisual, delay or pre-saccadic activity on a standard memory-guided saccade task. A total of 58neurons (21 from monkey C and 37 from monkey S) provided a full data set and are includedin the analysis.

Statistical AnalysisAll analyses were initially performed for each monkey and, since there were no significantdifferences between monkeys, the pooled data are presented here. Values in the text representmean ± standard error unless otherwise noted. Statistical comparisons were performed withnon-parametric 2 sample tests (paired or unpaired Wilcoxon test) or with 2-way analysis ofvariance (ANOVA) and evaluated at p = 0.05 unless otherwise indicated.

Analysis of Behavioral DataAll analyses are based on correct trials, with the exception of the analyses of saccade accuracy(Fig. 2a,b), which included correct and error trials. Although error trials were immediatelyrepeated, sometimes multiple times, we only included trials which the monkey successfullycompleted on the first repetition in our analysis to avoid biasing the results by long runs ofrepeated trials. Saccade onset was detected offline using velocity and acceleration criteria;saccade latency was measured from target onset to saccade onset. Saccade accuracy wasdefined as (180-d)/180, where d is the absolute angular distance, in degrees, between the vectorsrepresenting the target and the saccade endpoint relative to the fixation position.

We measured anticipatory licking in a window extending from 20 ms before to 50 ms afterreward delivery for RC+ trials and between 300 ms before and 350 ms after the time when

Peck et al. Page 3

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

juice would be delivered on a rewarded trial for RC− trials. The more generous time windowwas used to allow for the possibility that monkeys may be inaccurate at estimating the expectedtime of reward. Because monkeys licked by default for all newly-learned RC, we defined thelearn-point by examining the extinction of licking on RC− trials. A trial was considered a ‘non-lick’ trial if the monkey was licking for less than 20% of the time during the measurementwindow, and the learn point was defined as the first of 3 consecutive non-lick RC−presentations.

Analysis of Neural DataWe calculated receiver operating (ROC) indices (Green and Swets, 1968; Oristaglio et al.,2006) in five separate analyses: 1) to measure neural reward selectivity, by comparing firingrates on RC+ and RC− trials, pooled across individual RC (Fig. 4); 2) to measure visualresponse latency, by comparing post-RC with pre-RC firing rates, pooled across all RC; 3) tomeasure the post-RC spatial bias, by comparing firing rates after RC inside and opposite theRF, 600–900 ms after RC onset (Fig. 5); 4) to measure selectivity for saccade direction, bycomparing activity before saccades toward and opposite the RF, 100–200 ms after target onset,excluding all spikes that occurred after saccade initiation (Fig. 6); and 5) to compare selectivityfor probe valence, by comparing activity evoked by RC+ and RC− probes. To measure rewardselectivity (Fig. 4b–d) we calculated ROC values in consecutive non-overlapping 10 ms binsaligned on RC onset. Raw trial-by-trial spike trains were smoothed using a half-Gaussian filter(half Gaussian standard deviation = 20 ms), which smeared the signal only forward in time,thus avoiding an underestimation of the latency. For each bin, the firing rates distributionevoked by the 2 RC+ was compared with that evoked by the 2 RC−, so that ROC valuesrepresent preference for RC+ or RC− regardless of feature selectivity. The statisticalsignificance of each value was assessed using a permutation test (n = 1000). A neuron wasdeemed valence selective if it showed significant ROC indices (p < 0.001) for 12 consecutivebins, and the latency of reward selectivity was marked at the beginning of the first of these 12bins. The same criterion was applied to standard trials (Fig. 4) and probe trials. To calculatethe latency of the visual response, we calculated ROC values comparing RC-evoked firingrates (pooled across all RC types for inside RF presentations) with baseline firing rates 0–30ms after RC onset. The visual latency was defined as the first of 12 consecutive significanttime steps (p < 0.001, 1 ms bin width).

In order to confirm that differences in pre-saccadic activity (Fig. 6) between congruent andincongruent trials were not simply a correlate of different saccade metrics for each RC, werepeated the analysis on a subset of trials matched for reaction times. For each RC group (newly-learned RC+, over-learned RC+, newly-learned RC−, over-learned RC−), we selected a subset(90% of the trials in each original data set) of congruent and incongruent trials that did notdiffer significantly in (p > 0.5) their distributions of reaction times.

RESULTSBehavioral task

To examine the impact of reward predictors on spatial attention we used a method borrowedfrom visual cueing tasks. In these tasks a peripheral visual cue is first presented and is followed,after a delay period, by the appearance of a saccade target either at the cued location or at theopposite location. Although cues are not informative regarding saccade direction, theyautomatically bias attention by virtue of their bottom-up salience. Attentional biases aremeasured by comparing saccades directed toward and opposite the cue. At short cue-targetonset asynchronies, salient cues facilitate same-direction (congruent) relative to oppositedirection (incongruent) saccades, suggesting that they capture attention; at longer asynchronies

Peck et al. Page 4

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

the cues impair congruent relative to incongruent saccades, suggesting that they repel attentionfrom their location (Klein, 2000; Fecteau et al., 2004; Fecteau and Munoz, 2005).

In the present task the peripheral cue validly signaled the expected reward of the trial (Figure1a). Positive cues (RC+) signaled that a correct saccade will result in reward, while negativecues (RC−) signaled that, even if correctly completed, the trial will result in no reward. An RCappeared for 300 ms in the neurons’ receptive field (RF) or at the opposite location, and wasfollowed, after a 600 ms delay period, by a saccade target either at the same location or at theopposite location as the cue. Monkeys had to make a saccade to the target in order to experiencethe outcome predicted by the RC. Error trials were immediately repeated until it correctlycompleted, discouraging monkeys from immediately aborting unrewarded trials. Thus, theoptimal strategy in the task was to make a saccade to the target regardless of the location orvalence of the RC.

The RCs were abstract colored shapes that were initially novel to the monkeys. Each neuronwas tested with a set of newly-learned RC that were introduced and trained for one sessiononly. In addition, neurons were tested with a set of familiar (overlearned) RC, which had beenassociated with a constant outcome (reward or no reward) for at least 13 sessions before neuralrecordings began (at least 2,600 correct trials per stimulus). Two distinct stimuli were assignedto each RC type (RC+, RC−, newly-learned and overlearned) in order to control for stimulus-specific effects. All conditions (RC type, RC location and target location) werecounterbalanced and randomly interleaved within a session.

Behavioral performanceWe used three measures of performance. First, to ascertain that monkeys learned the rewardvalence of the cue, we measured anticipatory licking, a conditioned response that is a reliablemeasure of appetitive learning (Schultz, 2006). For over-learned RC, monkeys lickedselectively for RC+ but not RC− from the very first cue presentation indicating their familiaritywith the stimuli (Figure 1b, two right panels). For novel RC, monkeys began by licking for allstimuli, but, within the first few presentations ceased licking for a RC− (two left panels). Theaverage learn point (Figure 1c; the first presentation of an RC− at which licking reliablyextinguished) was 8.41 ± 0.54 presentations for newly-learned RC (mode: 5 presentations, n= 58) and 2.03 ± 0.19 presentations for over-learned RC (mode: 1 presentation, n = 58). Trialsprior to the learn point were excluded from subsequent analysis. Thus, in comparing newly-learned and over-learned RC we capture the differential effects of stimuli with known rewardvalence that had been trained for short or long periods of time. (We use the term “overlearned”where others may use “well-learned” or “long-learned”, to indicate that training proceededlong past the behavioral learn point as defined by anticipatory licking.)

Second, to assess how reward impacted motivation we compared saccade metrics on rewardedand unrewarded trials. Reward expectation improved saccade performance, as shown by ahigher fraction of correctly completed trials (91% vs 76%), higher saccade accuracy (compareleft and right panels in Figure 2a; p < 10−4, Wilcoxon signed rank test) and shorter reactiontimes (compare left and right panels of Figure 2b; p < 10−79) on RC+ relative to RC− trials.This replicates the well-known effects of motivation on operant behavior (Watanabe et al.,2001; Bendiksby and Platt, 2006; Kobayashi et al., 2006; Roesch and Olson, 2007).

Our third and primary measure determined whether the RC biased attention in spatially specificmanner, a question not previously addressed in the literature. To answer this question examinedwhether the RC differentially affected saccades that happened to be directed toward or oppositethe RC location. We found that the RC spatially biased saccades in valence-specific manner,such that the RC+ slightly attracted (facilitated) saccades toward its location while the RC−strongly repulsed (impaired) saccades away from its location relative to saccades directed

Peck et al. Page 5

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

away. Figure 2 compares congruent and incongruent saccades with respect to accuracy (Figure2a) and RT (Figure 2b). On RC+ trials saccade accuracy was slightly higher on congruent thanon incongruent trials (Figure 2a, left), revealing a slight attractive effect of the RC+ (2-wayANOVA, p = 0.049 for effect of congruence; p > 0.05 for effect training length and p = 0.455for interaction). On RC− trials, in contrast, accuracy was markedly lower (Figure 2a, right)and RT was higher (Figure 2b, right) on congruent relative to incongruent trials, suggesting astrong repulsive effect. Figure 2c illustrates the impairment in accuracy produced by an RC−in a representative session. While on RC+ trials saccade endpoints were tightly clustered aroundthe target (top), on RC− trials saccades had large endpoint scatter, and many fell outside thetarget window. This was not simply an effect of motivation, as this large scatter was only seenfor congruent saccades (Figure 2a, right). Moreover, the effect was maladaptive, as it causedmany erroneous saccades. Strikingly, however, the effect grew rather than abating withtraining, becoming worse for the overlearned relative to newly-learned RC. A 2-way ANOVAacross the data set revealed, for RC− trials, significant effects of congruence, training lengthand interaction for both accuracy (p < 10−9 , p = 0.025 and 0.001, respectively; Figure 2a, right)and RT (p = 0.0001, 0.002, 0.005, Figure 2b, right). Thus, the spatial repulsion generated byan RC− increased with training despite its detrimental impact on performance.

LIP neurons show valence selectivity that increases with long-term trainingTo see whether and how LIP neurons encoded the reward effects in this task we tested theirresponses to the RC and saccade target. In the following, we first compare neuronal valenceselectivity for newly-learned and overlearned RC appearing in the RF. We next examinewhether neurons encoded the RC or expected reward by comparing trials in which the RCappeared inside and opposite the RF. Finally, we analyze the impact of reward on responsesencoding saccade direction.

When cues appeared in the RF, neurons had a fast transient response to cue onset, which soondifferentiated according to cue valence, becoming stronger after RC+ relative to RC−.Responses of a representative neuron are shown in Figure 3, and population responses (n = 58neurons) are shown in Figure 4a. Valence selectivity did not simply reflect neural preferencefor RC shape or color, as it clearly persisted when distinct RC were averaged within eachreward category (i.e., averaged histograms in Figure 3 and Figure 4a). Of the neurons showingsignificant modulation by reward, some had additional stimulus selectivity or interactionbetween stimulus and reward (14/25 for newly-learned RC, 24/41 for over-learned RC; 2-wayANOVA, 100–500 ms after RC onset). However, among neurons that were modulated eitherby reward or by stimulus identity, the former were significantly more prevalent (25/27 for over-learned RC, 14/26 for newly-learned RC; Chi-square test, p = 0.01). Thus, although shape/color preference was present in a subset of cells (Sereno and Maunsell, 1998), it could notaccount for the stronger and more prevalent sensitivity to reward.

As shown in Figure 3 and Figure 4, reward effects were larger for over-learned relative tonewly-learned cues. Because monkeys had clear discriminatory licking to all stimuli, thisrepresents a long-term effect that develops with a time course longer than the fast acquisitionof the conditioned response (e.g., Figure 1). We note, however, that neurons also showed ashort-term learning effect, acquiring valence selectivity for newly-learned RC within the firstfew trials. A population analysis of this fast learning component is shown in Fig. S1.

We quantitatively measured valence selectivity using an ROC analysis (Green and Swets,1968) comparing the distribution of firing rates evoked by RC+ and RC−. This yielded an ROCindex ranging between 0 and 1, where values of 0.5 indicate no reward selectivity, and valuesbelow or above 0.5 indicate preference, respectively, for RC− and RC+. Significant valenceselectivity, predominantly preference for RC+, was found in a vast majority of neurons (Figure4b). Long-term training increased the fraction of selective neurons from 85% for newly-learned

Peck et al. Page 6

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

to 95% for over-learned RC (p = 0.023, Chi-square test) and significantly strengthened overallpreference for the RC+ (Figure 4d). Training also decreased the latency of reward selectivity.Median latencies of the reward effect were, across individual neurons (Figure 4c) 155 ms forover-learned RC versus 245 ms for newly-learned RC (p = 0.0018), and in the populationresponse, 95 ms versus 235 ms (Fig. 4d). To directly examine whether valence selectivity waspresent in the early visual response, we calculated selectivity in a 70 ms window aligned oneach neuron’s visual response latency (see Methods). The fraction of neurons with significantselectivity in this early time window increased from 15% for newly-learned RC to 41% forover-learned RC (p = 0.0020, Chi-square test). Thus, long-term training increased theprevalence and magnitude, and decreased the latency of valence coding in LIP.

Measurement of firing rates showed that the training-related increase in selectivity wasassociated with a decline in visually evoked responses. Visual responses (100–500 ms afterRC onset) declined by 4.90 ± 0.86 sp/s for over-learned vs. newly-learned RC+,. Responsesshowed a larger decline for RC−, of 8.32 ± 1.37 sp/s for RC− (in normalized units, differenceswere 0.07 ± 0.02 vs 0.14 ± 0.03; all p < 10−5 relative to 0 and p < 10−7 for RC+ versus RC−).Thus, prolonged training produced a global decline in firing rates for both RC+ and RC−, whichmay have indicated an effect of stimulus familiarity, and an additional a valence-specific effect- an especially pronounced decline for stimuli predicting no reward.

Reward cues produce spatial biases in sustained activityThe observation that neurons maintained valence selectivity throughout the delay period(Figure 4) was surprising given that the RC were uninformative regarding saccade direction.One possibility is that these sustained responses reflect global effects of motivation – a generalincrease in activity in rewarded relative to unrewarded trials regardless of RC location - asreported in the frontal lobe (Roesch and Olson, 2004;Kobayashi et al., 2006) and LIP itself(Bendiksby and Platt, 2006). Alternatively, the cue-evoked responses may be spatially specific,evoking relatively higher or lower activity at the cue location, consistent with a spatialattentional bias toward or away from the cue (Bisley and Goldberg, 2003). To distinguish thesepossibilities we compared sustained activity evoked by the RC when these appeared inside andopposite the RF (Figure 5a, black vs. gray traces).

Sustained RC-evoked responses were spatially specific, appearing only if the RC werepresented in the RF. Following presentation of an RC+ in the RF (Figure 5a, top panels) neuronsgenerated sustained excitation. However, if the RC+ appeared opposite the RF there was noresponse during the delay period, although there was a transient decline in firing during thevisual epoch, suggestive of a push-pull mechanism. Table 1 provides detailed comparisons ofvisual and delay firing rates with the pre-RC baseline. If an over-learned RC− appeared in theRF (bottom right panel) the transient visual response was followed by sustained inhibitionduring the delay period; again, there was no response if the RC− appeared at the oppositelocation (bottom right). We measured these spatial biases using ROC analysis, comparingdelay-period firing rates on inside-RF and opposite-RF trials (Figure 5b; note that these ROCindices reflect neural selectivity for RC location not valence). For RC+, spatial ROC indiceswere above 0.5, indicating an attractive bias toward the cue’s location (0.63 ± 0.03 for newly-learned RC, 0.61 ± 0.02 for over-learned RC, both p < 10−4 relative to 0.5). For newly-learnedRC−, there was no significant bias (0.53 ± 0.02, p = 0.25), but for over-learned RC−, indiceswere lower than 0.5, indicating a repulsive bias away from the RC− location (0.43 ± 0.02, p <10−3). Thus the RC set up a spatial bias across the topographical representation in LIP, whichwas attractive, toward the location of an RC+ and repulsive, away from an over-learned RC−,consistent with the spatial biases exerted by the RC on saccades (compare with Figure 2).

In principle it is possible that neurons encode RC valence and location during the delay periodand encode expected reward only after saccade direction is specified – i.e., after presentation

Peck et al. Page 7

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

of the saccade target (Platt and Glimcher, 1999; Sugrue et al., 2004). Therefore we analyzedactivity between target presentation and saccade onset. We computed saccade directionselectivity as the difference in firing rates prior to saccades toward and opposite the RF (Figure6a), and quantitatively measured this selectivity using ROC analysis (comparing firing ratesprior to saccades into and opposite the RF, 100–200 ms after target onset) (Figure 6b). Neuronsreliably encoded saccade direction in all trials, as shown by the positive response differencesin Figure 6a and ROC values greater than 0.5 in Figure 6b. However, directional responseswere not affected by expected reward. Wilcoxon signed rank test shows no significantdifferences in ROC indices between rewarded and unrewarded trials (p = 0.132 for congruentsaccades and p = 0.105 for incongruent saccades; compare right and left panels in Figure 6b).On the other hand, pre-saccadic responses did reflect congruence on RC− trials, being strongerfor incongruent relative to congruent saccades (a 2-way ANOVA on ROC values revealed, forRC−, p = 0.026 for effect of congruence and p > 0.7 for effect of training and interaction; forRC+, p > 0.3 for all effects). Thus, neural activity continued to reflect the RC-induced bias butwas not modulated by expected reward even while monkeys were planning the saccade.

A potential concern raised by these observations is that reward may have affected saccademetrics by a mechanism outside of LIP, and LIP neurons merely reflected this change insaccade metrics. However, this concern can be ruled out with respect to saccade accuracy,because, while the analysis of saccade accuracy in Figure 2 included correct and error saccades,the neuronal analysis in Figure 5 and Figure 6 were based only on correct trials. In correcttrials, saccade accuracy was high and statistically equivalent for all RC and congruence classes,so that neuronal valence effects could not be a secondary effect of saccade endpoint scatter.To rule out that neural effects were attributed to differences in RT (which were significant evenfor correct trials; Figure 2b, right) we recalculated saccade ROC indices in a subset of trialsthat were equated for RT (see Methods). Significant effects of congruence persisted in thissubset (RC−: p = 0.0463 for effect of congruence, p > 0.52 for training and interactions; RC+:p > 0.33 for all; 2-way ANOVA). Thus, neurons encoded RC valence and not a spurious effectof saccade accuracy or RT.

Learned salience and visual plasticityProlonged training lowered the latency of reward selectivity so that, for overtrained RC valenceselectivity was present in the early visual response for approximately 50% of neurons. Thisshorter latency has two potential explanations. First, training may have increased the speed ofreward evaluation and of the reward-related feedback being relayed to LIP. Alternatively,training may have induced a plastic - hardwired - change in the bottom-up visual response tothe RC, which no longer required active evaluation of reward valence. We reasoned that, iftraining produced visual plasticity, valence selectivity should be present in the visual responseeven in a novel context in which the RC no longer predict reward.

We tested a subset of neurons in a new behavioral context, probe trials, which were presentedafter a block of standard trials. Probe trials were identical with standard trials in that they startedwith a valid RC followed by a delay period and by presentation of a saccade target. However,in contrast with standard trials, the RC and saccade target always appeared opposite the RF,distracting attention as much as possible away from the RF (Figure 7a). Simultaneous withillumination the saccade target opposite the RF, a task-irrelevant probe was briefly flashed inthe RF. The RF probe was chosen from the set of RC trained on the previous block of standardtrials, but its valence was unpredictive of reward.

In environments containing multiple potential predictors animals use the earliest and mostreliable predictor to infer expected reward (Fanselow and Poulos, 2005). Thus, we expectedthat the first, informative, RC would block the monkeys’ interpretation of probe valence. Thiswas confirmed by measurements of anticipatory licking (Figure 7b). Even though reward was

Peck et al. Page 8

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

delivered 500–600 ms after probe onset (350 ms after the end of the saccade) allowing ampletime to generate anticipatory licking in response to the probe, licking depended solely on theinformative cue and was entirely unaffected by the probe (p < 10−7 for main effect of first RCvalence; p > 0.89 for main effect of probe valence and interaction, 2-way ANOVA). Thissuggests that monkeys actively evaluated the reward valence of the first cue but not that of theprobe.

Despite their lack of relevance for reward, the probes were still salient visual stimuli and wereexpected to elicit visual responses and bottom-up shifts of attention. If the visual responsesand/or the attentional weight of the stimuli were permanently modified by reward training,valence-specific effects in bottom-up attention should persist even on probe trials.

Examination of neural responses and saccade latencies confirmed this result with respect toovertrained probes (Figure 8). For over-learned probes, significant valence selectivity waspresent across the population (Figure 8a, top; ROC analysis, 130–230 ms after probe onset, p= 0.0064, n = 34 neurons) and individually in 15 of the 34 neurons tested (40%; Figure 8b,main panel). Despite the differences in task conditions, selectivity on probe trials was positivelycorrelated with that on standard trials (r = 0.44, p = 0.0099, n = 34). There was no interactionbetween probe responses and reward expectation (the valence of the informative RC; Fig. S2,all p > 0.71, 2-way ANOVA), showing that these effects could not be due to expected reward.Because visual responses in LIP correlate with the distracting power of a task-irrelevantstimulus (Balan and Gottlieb, 2006;Ipata et al., 2006), we wondered whether an RC+ probewould produce stronger interference with the saccade relative to an RC−. Indeed, saccadelatencies were longer if the saccade was performed in the presence of an RC+ relative to anRC− (Figure 8c, left panel; blue vs. red traces; p = 0.058 for rewarded trials, and p = 0.0023for unrewarded trials). Note that probes did not affect motivation, as this effect would havehad the opposite sign (i.e., shorter RT in the presence of an RC+ relative to RC−).

In contrast to over-learned probes, the visual responses evoked by newly-learned probe werenot valence-selective (Fig. 8a, bottom; ROC analysis, p > 0.71) even though the neuronsshowed clear selectivity in the corresponding time window on standard trials (inset).Significant selectivity was present in only 7/34 individual neurons, was not correlated betweenprobe and standard trials (r = 0.14, p = 0.4442) and was absent even in the subset (n = 15) thatwas selective for over-learned probes (Figure 8c, inset). Consistent with these neural results,newly-learned probes did not differentially affect saccade RT (Figure 8c, right, red vs bluelines, p = 0.65 and p = 0.13). Thus, the spatial attentional effects of a newly-learned RC areonly present on standard trials, suggesting that these effects depend on active evaluation ofstimulus valence.

DISCUSSIONWhile multiple studies have explored the factors that govern saccade decisions, much less isknown about how the brain determines the attentional priority or salience of informativestimuli. Here we show that a powerful determinant of attentional priority is the learnedassociation between a stimulus and an appetitive reward. Stimuli associated with reward gainenhanced representation in LIP and attract attention to their location; stimuli associated withno-reward evoke lower or inhibitory responses in LIP and repel attention from their location.This valence-dependent priority is assigned automatically even when it is objectively non-optimal – when a reward predictor is spatially separate from, and interferes with a requiredaction. The results suggest that associations between a stimulus and an appetitive reward, evenwhen established independently of an operant association, are important determinants of thepower of the stimulus to attract attention.

Peck et al. Page 9

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Multiple effects of rewardThe estimation of potential gains or losses is critical for survival, and it is not surprising thatreward computations produce diverse behavioral effects mediated by multiple neuralmechanisms. In the present task, we identified three distinct behavioral effects of reward;however, only one of these effects was encoded in LIP. First, reward expectation engendereda conditioned response, anticipatory licking. The properties of anticipatory licking weredissociated from firing rates in LIP. While monkeys acquired discriminatory licking after thefirst few stimulus presentations, valence effects in LIP continued to grow on much longer timescales. In addition, on probe trials neurons encoded the valence of task-irrelevant probe stimuli,even though these stimuli did not affect licking. These dissociations suggest that anticipatorylicking did not depend on LIP, in line with a wealth of evidence showing that conditionedbehaviors depend on subcortical, not cortical mechanisms (Fanselow and Poulos, 2005).Secondly, expected reward affected motivation, such that saccade performance was superioron trials that culminated with reward. LIP neurons did not encode motivation either, as firingrates did not differ between rewarded and unrewarded trials unless the RC was in the RF.Motivation may have been mediated by prefrontal and premotor areas that are sensitive toexpected outcome (Kobayashi et al., 2006; Roesch and Olson, 2007). A final, and previouslyuninvestigated effect, was the ability of the RC to bias attention. Positive reward predictorsattracted attention while negative predictors repelled attention from their location, and thesespatial biases were encoded in LIP through sustained excitatory of inhibitory responses specificto the RC location. Thus, LIP neurons did not reliably encode non-spatial aspects of rewardcomputations indexed by conditioned behaviors or motivation, but only encoded a very specificeffect – the valence-specific attentional weight of a stimulus associated with reward.

The specificity of these responses speaks to a longstanding question regarding the role of LIPin reward-based behaviors (Maunsell, 2004). Our findings provide direct evidence that LIP isnot a critical part of the neural systems that monitor or evaluate potential gains or losses, anetwork that appears to include subcortical structures along with cortical areas such as theorbitofrontal or cingulate cortices (Schultz, 2006). However, LIP does appear important forallowing the expression of reward-dependent behaviors, in particular the biasing attention andeye movements according to the valence of external stimuli. Consistent with this, we recentlyshowed that reversible inactivation of LIP does not affect reward evaluation processesthemselves, but affects the ability to use reward (or other sources of information) in a spatiallyunbiased manner (Balan and Gottlieb, 2009). Thus, LIP provides a visuo-spatial map that readsout the outcome of reward computations in spatial terms for the purpose of guiding attentionand action.

Distinguishing between the reward effects on attention and actionWhile the proposal that LIP represents a pragmatic reward-modulated spatial representationhas been advanced before (Sugrue et al., 2004), prior studies concluded that this representationencodes the expected reward of an action (a saccade) (Platt and Glimcher, 1999; Dorris andGlimcher, 2004; Sugrue et al., 2004). However, the present results suggest that thisinterpretation may not provide a general description of expected reward influence on LIPresponses. In the present task, neurons encoded the reward valence of a stimulus even whenthis valence was not aligned with the reward of the saccade.

Four central properties distinguish the reward modulations in our task from a code of actionvalue. First, the action value hypothesis predicts that after the initial visual response, neuronsshould weight each potential target position equally (i.e., have equal delay period firing ratesregardless of RC location) given the equal probability that the target will appear there.However, we find that neural responses (Figure 5) and subsequent saccades (Figure 2) remainedbiased by RC valence and location, even following a delay after the RC itself is extinguished.

Peck et al. Page 10

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Secondly, the action value hypothesis predicts that neural responses should differ accordingto RC RC-induced valence when the target appears in the neuron’s RF and the ‘action-value’contingency has been established. Contrary to this prediction, pre-saccadic responses in LIPwere not modulated by expected reward but only reflected the biases exerted by the RC (Figure6). Thirdly, a reward-enhanced signal of action value is expected to be adaptive, facilitatingthe choice of action that harvests the higher reward. In contrast, the reward-enhanced responsesin our task were maladaptive, interfering with the required saccade. Finally, an account rootedin decision processes implies that reward prediction will improve with learning, becomingmore closely aligned with an optimal (reward-maximizing) strategy (Sugrue et al., 2004). Incontrast, learning in our task impaired performance, exacerbating the maladaptive effectsexerted by the RC− (Figure 2–Figure 6). It may be argued that monkeys did not detect thedetrimental effect of an RC− error, because the trial was anyway unrewarded and an errorreduced reward only over longer time scales (delayed the opportunity to progress to a rewardedtrial). However, this is highly unlikely. Had monkeys only used short-term reward evaluationthey would have immediately aborted each RC− trial; in addition, because an error trial wasimmediately repeated, its highest impact was on reward rate immediately following an error,precisely the time scale that monkeys seem to rely on when estimating expected reward (Sugrueet al., 2004). Thus, the RC effects in our task were distinguished from action value in that theyacted automatically, regardless of the operant significance of the reward predictors and theiroptimality for behavior.

Our findings do not exclude the possibility that in a restricted set of circumstances LIP mayprovide a de facto signal of action value as concluded in prior studies. Indeed, in the conditionsused in these studies all the stimuli presented to the monkey were part of the decision set (eachstimulus represented a decision alternative); in these circumstances the expected reward of astimulus is equivalent to the expected reward of the saccade, and LIP activity accurately reflectsaction value (Platt and Glimcher, 1999; Dorris and Glimcher, 2004; Sugrue et al., 2004).However, in natural behavior decisions are forged in the presence of multiple stimuli that mayhave reward valence but no relevance for immediate action. In these conditions, LIP is likelyto mediate attentional biases produced by reward predictors even when they come in conflictwith the objective gain of the action. Thus, LIP reflects reward evaluation at the level ofstimulus selection. In contrast, critical aspects of action evaluation (like critical aspects of asaccade decision itself (Gottlieb and Goldberg, 1999)) seem to be computed in separatestructures, possibly downstream from LIP.

Learning of attentional priorityThe tenacity and automaticity of the effects we describe suggests that these effects may berooted not so much in cognitive decision processes, but in more automatic forms of learningperhaps related to emotional learning. In human subjects, stimuli with intrinsic emotionalsignificance (e.g., fearful faces) automatically attract attention and modify visually evokedresponses and psychophysical performance even when they are irrelevant to a task(Vuilleumier, 2005; Phelps et al., 2006; Padmala and Pessoa, 2008). Our results extend thesefindings by demonstrating that attentional biases are also produced by predictors of appetitive,not aversive, outcomes, possibly representing a mechanism for automatic monitoring ofinformation regarding appetitive reward.

Prolonged training increased the attentional effects of the RC and produced changes in theearly visual response that were suggestive of visual plasticity. These findings extend priordemonstrations of learning in the visual and oculomotor system, by showing that profoundlearning effects can be obtained merely through training of stimulus-response associations inthe absence of operant associations (i.e., related to conditional visuo-motor learning (Chen andWise, 1996; Asaad et al., 1998), categorization (Sigala and Logothetis, 2002; Freedman and

Peck et al. Page 11

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Assad, 2006), decision making (Sugrue et al., 2004) or target selection (Bichot et al., 1996;Mruczek and Sheinberg, 2005, 2007)). In addition, our findings suggest that long-term trainingproduced plastic changes in the bottom-up visual response that were partly independent of top-down evaluation of expected reward. Three aspects of the probe trial results support thisconclusion. First, probe stimuli did not modify indicators of expected reward such asanticipatory licking and motivation, suggesting that monkeys actively evaluated reward basedon the first informative cue and not based on the probe. Second, valence-specific attentionalbiases (on saccade RT) were only produced by overtrained RC, whereas a top-down evaluationmechanism would be expected to occur for both newly-learned and over-learned RC. Third,LIP neurons had very short-latency valence selectivity on probe trials (Figure 8b) that isdifficult to reconcile with a cognitive evaluation mechanism. Thus, the most parsimoniousexplanation of the probe results is that long-term training modified the visual response evokedby reward predictors; this conferred upon the RC an intrinsic salience that was akin to thephysical salience of a conspicuous object but was determined by the training history of the RC.This form of learning may underlie the special salience of highly familiar stimuli such as theletters of the alphabet or a friend’s face, which automatically pop-out from a crowded visualscene. Thus our results support the idea that operant learning involves, at least in part, changesin the salience of task-relevant objects driven by learned associations between these objectsand reward (Ferrera and Grinband, 2006).

Supplementary MaterialRefer to Web version on PubMed Central for supplementary material.

AcknowledgmentsWe are indebted to members of the laboratory, most especially Puiu Balan, for insightful discussions and technicalhelp. We thank M. Osman and G. Asfaw for veterinary care, the Columbia University MRI Research Center for MRIscans, and Latoya Palmer for administrative assistance. Supported by The National Eye Institute (R01 EY014697-01and R24 EY015634), and a Swiss National Science Foundation Fellowship to MS.

REFERENCESAsaad WF, Rainer G, Miller EK. Neural activity in the primate prefrontal cortex during associative

learning. Neuron 1998;21:1399–1407. [PubMed: 9883732]Balan PF, Gottlieb J. Integration of exogenous input into a dynamic salience map revealed by perturbing

attention. J Neurosci 2006;26:9239–9249. [PubMed: 16957080]Balan PF, Gottlieb J. Functional significance of nonspatial information in monkey lateral intraparietal

area. J Neurosci 2009;29:8166–8176. [PubMed: 19553456]Balan PF, Oristaglio J, Schneider DM, Gottlieb J. Neuronal correlates of the set-size effect in monkey

lateral intraparietal area. PLoS Biol 2008;6:e158. [PubMed: 18656991]Bendiksby MS, Platt ML. Neural correlates of reward and attention in macaque area LIP.

Neuropsychologia 2006;44:2411–2420. [PubMed: 16757005]Bichot NP, Schall JD, Thompson KG. Visual feature selectivity in frontal eye fields induced by experience

in mature macaques. Nature 1996;381:697–699. [PubMed: 8649514]Bisley JW, Goldberg ME. Neuronal activity in the lateral intraparietal area and spatial attention. Science

2003;299:81–86. [PubMed: 12511644]Bogacz R. Optimal decision-making theories: linking neurobiology with behaviour. Trends Cogn Sci

2007;11:118–125. [PubMed: 17276130]Chen LL, Wise SP. Evolution of directional preferences in the supplementary eye field during acquisition

of conditional oculomotor associations. J Neurosci 1996;16:3067–3081. [PubMed: 8622136]Dorris MC, Glimcher PW. Activity in posterior parietal cortex is correlated with the relative subjective

desirability of action. Neuron 2004;44:365–378. [PubMed: 15473973]

Peck et al. Page 12

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Fanselow MS, Poulos AM. The neuroscience of mammalian associative learning. Annu Rev Psychol2005;56:207–234. [PubMed: 15709934]

Fecteau JH, Munoz DP. Correlates of capture of attention and inhibition of return across stages of visualprocessing. J Cogn Neurosci 2005;17:1714–1727. [PubMed: 16269108]

Fecteau JH, Bell AH, Munoz DP. Neural correlates of the automatic and goal-driven biases in orientingspatial attention. J Neurophysiol 2004;92:1728–1737. [PubMed: 15115792]

Ferrera VP, Grinband J. Walk the line: parietal neurons respect category boundaries. Nat Neurosci2006;9:1207–1208. [PubMed: 17001336]

Freedman DJ, Assad JA. Experience-dependent representation of visual categories in parietal cortex.Nature 2006;443:85–88. [PubMed: 16936716]

Gottlieb J, Goldberg ME. Activity of neurons in the lateral intraparietal area of the monkey during anantisaccade task. Nature Neurosci 1999;2:906–912. [PubMed: 10491612]

Green, DM.; Swets, JA. Signal Detection Theory and Psychophysics. New York: Wiley; 1968.Ipata AE, Gee AL, Gottlieb J, Bisley JW, Goldberg ME. LIP responses to a popout stimulus are reduced

if it is overtly ignored. Nat Neurosci 2006;9:1071–1076. [PubMed: 16819520]Klein RM. Inhibition of return. Trends Cogn Sci 2000;4:138–147. [PubMed: 10740278]Kobayashi S, Nomoto K, Watanabe M, Hikosaka O, Schultz W, Sakagami M. Influences of rewarding

and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 2006;51:861–870.[PubMed: 16982429]

Maunsell JH. Neuronal representations of cognitive state: reward or attention? Trends Cogn Sci2004;8:261–265. [PubMed: 15165551]

Mruczek RE, Sheinberg DL. Distractor familiarity leads to more efficient visual search for complexstimuli. Percept Psychophys 2005;67:1016–1031. [PubMed: 16396010]

Mruczek RE, Sheinberg DL. Context familiarity enhances target processing by inferior temporal cortexneurons. J Neurosci 2007;27:8533–8545. [PubMed: 17687031]

Oristaglio J, Schneider DM, Balan PF, Gottlieb J. Integration of visuospatial and effector informationduring symbolically cued limb movements in monkey lateral intraparietal area. J Neurosci2006;26:8310–8319. [PubMed: 16899726]

Padmala S, Pessoa L. Affective learning enhances visual detection and responses in primary visual cortex.J Neurosci 2008;28:6202–6210. [PubMed: 18550762]

Phelps EA, Ling S, Carrasco M. Emotion facilitates perception and potentiates the perceptual benefits ofattention. Psychol Sci 2006;17:292–299. [PubMed: 16623685]

Platt ML, Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature 1999;400:233–238. [PubMed: 10421364]

Roesch MR, Olson CR. Neuronal activity related to reward value and motivation in primate frontal cortex.Science 2004;304:307–310. [PubMed: 15073380]

Roesch MR, Olson CR. Neuronal activity related to anticipated reward in frontal cortex: does it representvalue or reflect motivation? Ann N Y Acad Sci 2007;1121:431–446. [PubMed: 17846160]

Schultz W. Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 2006;57:87–115.[PubMed: 16318590]

Sereno AB, Maunsell JH. Shape selectivity in primate lateral intraparietal cortex. Nature 1998;395:500–503. [PubMed: 9774105]

Sigala N, Logothetis NK. Visual categorization shapes feature selectivity in the primate temporal cortex.Nature 2002;415:318–320. [PubMed: 11797008]

Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietalcortex. Science 2004;304:1782–1787. [PubMed: 15205529]

Sugrue LP, Corrado GS, Newsome WT. Choosing the greater of two goods: neural currencies forvaluation and decision making. Nat Rev Neurosci 2005;6:363–375. [PubMed: 15832198]

Sutton, RS.; Barto, AG. Reinforcement learning. MIT Press; 1998.Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends Cogn Sci

2005;9:585–594. [PubMed: 16289871]

Peck et al. Page 13

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Watanabe M, Cromwell HC, Tremblay L, Hollerman JR, Hikosaka K, Schultz W. Behavioral reactionsreflecting differential reward expectations in monkeys. Exp Brain Res 2001;140:511–518. [PubMed:11685405]

Peck et al. Page 14

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 1.Behavioral task and licking behavior. (a) Task sequence. A stable display with twoplaceholders remains visible during the inter-trial interval. A trial begins when the monkeyachieves central fixation, bringing one of the placeholders into the RF (gray circle). After afixation period, a RC appears, followed by a delay period and illumination of one of theplaceholders. The monkey is required to make a saccade to the illuminated placeholder in orderto receive the outcome predicted by the RC. If applicable (i.e., on a correct RC+ trial), thereward is given 350 ms after the end of the saccade; otherwise, no reward is given. (b) Lickingbehavior during an example session. Trials are sorted offline by RC type and plotted inchronological order with the first trial on top. Blue horizontal lines indicate the times at which

Peck et al. Page 15

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

the monkey was licking during each trial. Rasters are aligned at the time of reward delivery onRC+ trials and at the corresponding time point (350 ms after saccade end) on RC− trials. Thedots mark trial events as indicated in the legend. (c) Frequency distribution of behavioral learnpoints for individual RC− during all 58 recording sessions.

Peck et al. Page 16

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 2.The RC exert spatial biases on saccade accuracy and RT. (a) Angular saccade accuracy duringall recording sessions (n=58, mean ± SEM) as a function of RC type and spatial congruencebetween RC and target (congruent: thick lines, incongruent: thin lines). (b) Saccade reactiontimes over all recording sessions in the same format as (a). (c) Endpoints of individual saccadesfor congruent trials in an example session. All saccades are included, regardless of whetherthey were scored as correct or errant. Saccade coordinates are normalized so that the target(indicated by the open square) was always mapped onto the point (1,0).

Peck et al. Page 17

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 3.Response of a representative neuron to newly-learned and over-learned RC in its RF. In theraster displays, each tick represents an action potential and each row a single correct trial. Trialsare aligned on RC onset and truncated at the end of the delay period, and they are shown inchronological order with the first presentation at the top. Trials with distinct RC within acategory are intermingled. The spike density traces (bottom panels) show the average firingrates for RC+ (blue) and RC− (red) trials, considering only trials after the learn point for eachRC. Shading shows SEM. The black horizontal bar denotes RC duration.

Peck et al. Page 18

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 4.Population analysis of reward modulation. (a) Average normalized firing rates for the sampleof neurons (n = 58) in response to RC− (red) and RC+ (blue) stimuli in the RF for newly-learned (top) and over-learned (bottom) stimuli. Responses are aligned on RC onset (time 0)and are truncated at the end of the delay period. Shading indicates standard error. Firing rateswere normalized for each neuron by subtracting the baseline rate (50 ms before RC onset) anddividing by the peak response across all RC types. Stars denote 100 ms firing rate bins(beginning at 40 ms and shifted by 50 ms) with statistically significant differences betweenRC+ and RC− (Wilcoxon signed-rank test, p < 0.05). An apparent minor response peak at ~400ms is due to a subset of neurons that showed an off response to the disappearance of the RC.

Peck et al. Page 19

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

(b) ROC analysis of reward modulation. Each row represents an individual neuron, and eachpixel an ROC index in a 10 ms time bin. The white dashed line shows RC onset (time 0). ROCvalues above 0.5 (blue) signify preference for RC+, values below 0.5 (red) signify preferencefor RC−, and values close to 0.5 (black) signify no preference. Magenta crosses mark thelatency at which each cell met the significance criterion for a reward effect. (c) Cumulativedistribution of the reward latencies for over-learned (green) and newly-learned (goldenrod)RC. (d) Population average of the ROC values in (b) for over-learned and newly-learned RC.Circles denote time bins in which the value is significantly different from 0.5 (p < 0.05) andblue crosses denote bins with significant differences between over-learned and newly-learnedstimuli. Shading shows SEM.

Peck et al. Page 20

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 5.Spatial effects of RC during the delay period. (a) Population firing rates on trials in which theRC appeared in the RF (black) or opposite the RF (gray) for each RC type. Shading showsSEM. The vertical scale is expanded and truncates the visual response in order to highlightdelay period activity. (b) Population ROC values for RC location in the last 300 ms of the delayperiod. All symbols show mean ± SEM. Circles indicate RC+ trials, and triangles, RC− trials.Filled symbols show values that are significantly different from 0.5 (t-test, p<0.05).

Peck et al. Page 21

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 6.Spatial effects of the RC on pre-saccadic activity. (a) Difference traces between activity onsaccade into the RF and saccade out of the RF trials. Trials are sorted by congruence (congruent:black, incongruent: gray) for each RC category and are aligned on the onset of the saccadetarget. Shading shows SEM. (b) ROC for saccade direction 100–200 ms after target onset. Allvalues are significantly greater than 0.5 and are thus shown in filled symbols.

Peck et al. Page 22

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 7.Probe task structure and licking behavior. (a) Structure of probe trials. A single placeholderwas present at the location opposite the RF during the inter-trial interval, making the locationof the informative RC and the saccade target. A trial began as before with presentation of aninformative RC followed by a 600 ms delay period. Simultaneous with target onset, abehaviorally-irrelevant probe was flashed inside the RF. The probe was flashed for 80 ms andextinguished before the onset of the saccade. (b) Licking behavior during the probe task.Percentage of time spent licking (mean ± SEM over all probe sessions; n = 34) immediatelybefore juice delivery (or the equivalent time on RC− trials) as a function of the expected rewardpredicted by the informative RC (x-axis) and the valence of the probe (blue: probe RC+, red:probe RC−).

Peck et al. Page 23

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Figure 8.Neural responses and saccade reaction times during the probe task. (a) Population neuralresponse (n=34) during the probe task (main panels) and standard task (same 34 neurons; inset).Trials were sorted by learning history (over-learned: top, newly-learned: bottom) and thevalence of the stimulus in the RF (blue: RC+; red: RC−). Responses to the probes are truncatedat 250 ms – the average time at which the saccade moved the neuron’s RF away from the probelocation. Responses during the standard task are truncated at 250 ms as well for the sake ofcomparison. Shading is ± SEM. (b) Average response of neurons with significant selectivityfor the valence of over-learned probes (n= 15). Inset shows the responses of the same neuronsfor newly-learned probes. (c) Saccade reaction times during the probe task. Mean reactiontimes (±SEM) are plotted as a function of expected reward (x-axis) and probe valence (blue:probe+, red: probe−), separately for over-learned and newly-learned probes.

Peck et al. Page 24

J Neurosci. Author manuscript; available in PMC 2010 March 9.

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

NIH

-PA Author Manuscript

Peck et al. Page 25Ta

ble

1

Firin

g ra

tes (

sp/s

, mea

n ±

SEM

) acc

ordi

ng to

RC

trai

ning

his

tory

and

loca

tion,

in th

e vi

sual

epo

ch (1

00–5

00 m

s rel

ativ

e to

RC

ons

et) a

nd a

t the

end

of t

hede

lay

perio

d (6

00–9

00 m

s afte

r RC

ons

et).

Cue

Loc

atio

nN

ewly

-lear

ned

Ove

r-le

arne

dV

isua

lD

elay

Vis

ual

Del

ay

RC+

In R

F45

.96

± 3.

72 ‡ 21

.96

± 2.

38 † 41

.06

± 3.

39 ‡ 21

.16

± 2.

26 †

Opp

osite

RF

14.7

5 ±

2.20

‡18

.16

± 2.

3615

.10

± 2.

24 †

17.4

3 ±

2.28

RC−

In R

F41

.34

± 3.

58 ‡

18.3

4 ±

2.15

33.0

2 ±

2.96

‡ 14.0

4 ±

1.47

†O

ppos

ite R

F17

.96

± 2.

3418

.54

± 2.

1721

.37

± 2.

4418

.53

± 1.

94†

p <

.05,

‡ p <

.001

rela

tive

to b

asel

ine

Bas

elin

e (5

0 m

s bef

ore

RC

ons

et) =

19.

12 ±

2.3

0 sp

/s

J Neurosci. Author manuscript; available in PMC 2010 March 9.