HAL Id: tel-03484040https://tel.archives-ouvertes.fr/tel-03484040

Submitted on 16 Dec 2021

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Implication of the locus coeruleus in effort processing :electrophysiological studies

Pauline Bornert

To cite this version:Pauline Bornert. Implication of the locus coeruleus in effort processing : electrophysiological studies.Neurons and Cognition [q-bio.NC]. Sorbonne Université, 2021. English. �NNT : 2021SORUS163�.�tel-03484040�

Sorbonne Université

École doctorale Cerveau Cognition Comportement

Institut du Cerveau et de la Moelle épinière / Équipe Motivation Cerveau et Comportement

Implication of the locus coeruleus in effort processing:

electrophysiological studies

Par Pauline Bornert

Thèse de doctorat de Neurosciences

Dirigée par Sébastien Bouret

Présentée et soutenue publiquement le 14 septembre 2021

Devant un jury composé de: Sébastien Bouret, directeur de thèse

Mark Walton, rapporteur

Etienne Coutureau, raporteur

Philippe Faure, examinateur

Fadila Hadj-Bouziane, examinatrice

Sabrina Ravel, examinatrice

1

Remerciements Tout d’abord, je souhaiterais remercier tous les membres de mon jury de thèse : Mark

Walton, Etinenne Coutureau, Philippe Faure, Fadila Hadj-Bouziane et Sabrina Ravel. Un grand merci à Philippe Faure et Fadila Hadj-Bouziane qui ont suivi mon travail au cours de mes deux CST, et m’ont prodigué des conseils précieux. Un grand merci aussi à Mark Walton et Etienne Coutureau, qui ont accepté le rôle laborieux de raporteurs.

Je souhaite aussi adresser mes remerciements à l’école doctorale Cerveau, Cognition et Comportement qui a accepté de financer mon travail.

Un grand merci à Sébastien Bouret, mon directeur de thèse. Merci de m’avoir accordé ta confiance pour mon stage de Master 2, et merci de ton soutien pendant cette année compliquée au cours de laquelle je ne pouvais pratiquement pas marcher. Merci de m’avoir renouvelé ta confiance pour mon doctorat. Merci pour ton accompagnement et ton encadrement tout au long de cette thèse, au cours de laquelle tu m’as toujours poussée à me remettre en question et à me dépasser, et pour ces quatre années de discussions scientifiques enrichissantes.

Je souhaiterais aussi remercier tous les membres de notre équipe Motivation, Brain and Behavior, présents et passés, que j’ai pu cotoyer, pour leur sympathie et la qualité des échanges scientifiques que nous avons pu avoir au cours des réunions de labo et dans l’open space. Merci en particulier à Jean et Mathias pour leurs critiques constructives lors de mes présentations.

Un grand merci à tout le personnel de la plateforme primates de l’ICM, sans qui les études présentées dans ce manuscrit n’auraient pas pu avoir lieu. Je tiens en particulier à remercier Estelle, pour sa bonne humeur et son efficacité redoutable. Un grand merci pour ton aide précieuse avec les singes, et pour les rigolades dans les vestiaires ou les couloirs. Un grand merci aussi à Sophie, Audrey, Marie et Coraline pour tout ce qu’elles ont fait ou font pour nos macaques au jour le jour.

Sur le plan personnel, je souhaite remercier Fab. Mon Fab, tu as été mon roc en particulier au cours des difficiles derniers mois de cette thèse. Merci d’avoir toujours été là pour moi dans ces moments compliqués, de m’avoir écouté râler, d’avoir géré toute la logistique pour que je n’aie pas à m’en préoccuper. Et merci pour ta précieuse relecture de mon manuscrit.

Merci aussi à mes parents, qui m’ont toujours soutenue dans mes études, vétérinaires d’abord, puis ce doctorat. Et merci à mon (petit) frère Jean-Eudes, qui entre dans le supérieur en même temps que j’en sors, pour son soutien à base de vannes plus ou moins subtiles.

Merci à mes amies, Barbara, Pauline (N°2), Constance, Marine, Samirah. Barbara, tu m’as soutenue même quand toi-même avais besoin de soutien, et on s’est épaulées au cours des années passées. Merci d’être toujours là ! Pauline (number 2 of course), ma co-galérienne préférée, merci pour ces trois années de délires et de potins du midi, et de décompression le weekend.

Merci à Jean-Noël et Carmen, qui m’ont eux aussi soutenue au cours de l’année passée. Jean-Noël, un immense merci pour tes conseils concernant mon avenir professionnel, et pour les délicieux déjeuners à Montesson qui me permettaient de souffler l’espace d’une petite heure pendant la rédaction.

Enfin, un grand merci à Fanny qui m’a aiguillée vers la recherche, et à son mari Sébastien qui m’a fait découvrir le travail de laboratoire il y a maintenant quelques années.

Abstract

Many decisions involve trading effort for reward. Manipulations of locus coeruleus (LC) activity

have demonstrated its causal implication in effort processing. Indirect measurements of LC

activity reflect physical effort and cognitive control exertion and single unit recordings have

provided evidence for physical effort encoding by LC neurons. Moreover, LC neurons respond

to commitment errors and pupillometry studies and non-specific enhancements of LC activity

showed its role in adjusting effort allocation based on action consequences. We hypothesized

that LC neurons encode physical and mental effort costs, facilitate future effort production,

and participate in adjustments of effort allocation strategies based on previous effort costs

and action consequences. Here, the activity of LC units recorded in 4 tasks was examined. LC

neurons activated with temporal dynamics coherent with responses to the termination of

decisions, and consistently encoded the effort costs of triggering and executing actions. We

confirmed that the timing of these modulations was only compatible with delayed effects on

effort production. Moreover, in a two-alternative choice task, increased LC activity early in

trials was associated to within-trial facilitation of effort production. Additionally, LC neurons

responded to execution errors and increased LC activity around action execution was

associated to increases in effort sensitivity in the next trial. We discuss the behavioral

implications of such effects, their relation with other demonstrated roles of LC activity such as

behavioral variability and set-shifting, and the potential inputs and targets responsible of

them.

1

Résumé

De nombreuses décisions impliquent un compromis entre effort et récompense. Les

manipulations de l’activité du locus coeruleus (LC) ont montré son rôle causal dans la gestion

de l’effort. Les mesures indirectes de l’activité du LC reflètent l’effort physique et le contrôle

cognitif mis en œuvre, et les neurones du LC encodent l’effort physique. De plus, les neurones

du LC répondent aux erreurs d’engagement et les études de pupillométrie ou les modifications

non spécifiques de l’activité du LC ont montré son rôle dans l’ajustement du comportement

en fonction des conséquences des actions. Ici, l’activité de neurones du LC a été examinée

dans 4 tâches. Les neurones s’activaient avec une dynamique compatible avec une réponse à

l’achèvement des décisions, et codaient les coûts de l’effort liés au déclenchement et à

l’exécution des actions. Nous avons confirmé que le moment de ces activations n’était

compatible qu’avec des effets retardés sur la production d’effort. De plus, dans une tâche de

choix force/récompense à deux options, les augmentations d’activité du LC étaient associées

à une facilitation de la production d’effort au sein d’un même essai. Par ailleurs, les neurones

du LC répondaient aux erreurs d’exécution et une activité augmentée au moment de

l’exécution des actions était associée à une augmentation de la sensibilité aux coûts de l’effort

dans l’essai suivant. Nous discutons les implications comportementales de ces effets, leur

relation avec d’autres rôles du LC tels que la variabilité comportementale et l’adaptation aux

changements de règles, et les potentielles afférences et efférences mises en jeu.

2

Table of contents

ABSTRACT 0

RESUME 1

I. INTRODUCTION 6

A. Physical and mental effort: characteristics and implications for decision-making 7 1. Physical effort 7

a) Aversiveness of physical effort: physical effort as a cost and decision variable 8 b) Mental representation of physical effort 9 c) Measuring physical effort 9

(1) Ratings 9 (2) Behavioral proxies 10 (3) Physiological proxies 10

2. Mental effort 12 a) Nature of mental effort: definitions 12 b) Categories of tasks requiring cognitive control 13 c) Sub-processes of cognitive control 15 d) Cognitive control and decision-making 16

(1) Cognitive control and optimization of decisions 16 (2) Cognitive control as a decision variable per se 20 (3) Decisions to allocate control and how much control to allocate: the expected value of control

(EVC) theory 24 e) What is the substrate of cognitive control? What is the resource? 31

(1) Limited intrinsic physiological resource 31 (2) Limited computational resource 36 (3) Motivational resource: exploration-exploitation trade-off and opportunity costs of control 37

f) Behavioral proxies for mental effort 38 (1) Reaction time 38 (2) Performance 42

g) Physiological proxies for mental effort: cardiovascular, metabolic and vegetative measures 43 (1) Cardio-vascular and metabolic proxies 43 (2) Pupil dilation 44 (3) EEG proxies for cognitive control 46

3. Mental and physical effort, evidence for common resources and partially common processing systems

48

B. Role of the locus coeruleus in effort processing: review of the literature 50 1. Anatomy, histology and physiology of the locus coeruleus 50

a) Anatomy and histology 50 b) Electrophysiological properties 52 c) Connectivity to other brain structures 53 d) Noradrenergic receptor distribution and actions in target areas 56

2. Assessing LC activity: proxies for LC activity 57 a) P3 ERP 57 b) Pupil dilation 59

3. LC activity and enhancement of processes requiring resource mobilization 61 a) LC activity and arousal 61

3

b) LC and stress-related responses 63 c) LC activity and attention 64

4. LC and execution of costly goal-directed actions 66 5. LC and decision-making 74

a) Action selection 75 b) Action triggering 81

6. LC activity and action monitoring 87

C. Mechanisms for the effects of LC activity on target areas to regulate cognitive processes, and

implications for effort processing 89 1. LC activity modulates brain metabolism: NA and response to high metabolic demands 89 2. From LC facilitation of sensory processing and plasticity to gain theory 90

a) Facilitation of sensory processing 90 b) Overarching theories of short-term effects of NA on neuronal activity: brain states and gain

modulation 95 c) Extending gain theory in time: LC activity and plasticity promotion 100 d) LC activity and brain connectivity and networks 101

D. Summary and problematic 105

II. EXPERIMENTAL RESULTS 108

A. Study 1: LC neurons encode difficulty to trigger actions in one-option forced choice tasks 108 1. Locus coeruleus neurons encode the subjective difficulty of triggering and executing actions: evidence

for a role in effort processing (Bornert and Bouret, in prep) 108 2. Additional results 153

a) Activity of LC neurons around cue onset and the outcome 153 (1) LC neurons responded to cue onset and the outcome 153 (2) Modulation of LC responses by task parameters at cue onset and the outcome 155 (3) Modulation of LC activity by behavior 157

(a) Cue-evoked activity and willingness to work 157 (b) Real delay and outcome activity 158

b) Dissection of the temporal dynamics of parameter encoding relative to activity changes 158 c) Intrinsic neuronal excitability shaped the activation in response to task events and the encoding of

task parameters and behavioral variables 162 (1) Higher baseline was associated to higher peak of activity 162 (2) Higher baseline was associated to higher signal-to-noise ratio 164 (3) Higher baseline was associated to stronger encoding of task parameters and behavioral variables

166 (4) Encoding of Reward discounted value 167

3. Discussion of additional results and of study limitations 167

B. Study 2: LC neurons facilitate delayed effort mobilization and monitoring in a two-alternative choice

task 171 1. Introduction 171 2. Materials and methods 174

a) Animals 174 b) Behavioral paradigm 174 c) MRI, surgery and recordings 176 d) Data analysis 178

(1) Eye-tracking and determination of action-selection and action-triggering time 178 (2) Value inference from choices 180

3. Results 181

4

a) Behavior 181 (1) Engagement in costly actions 183

(a) RT to the middle press represented the cost of triggering the middle action and starting a trial

183 (b) Decisions to engage based on cue information 185

(i) Choice engagement increased with increased sum of Rewards of options 185 (i) Action-triggering time scaled with difficulty to engage in the choice 187 (ii) Difficulty to engage in the choices scaled with difficulty to engage in the middle press 190

(2) Choosing between offers 193 (a) Influence of Forces and Rewards on binary choices between offers 193 (b) Action-selection time had a tendency to increase with choice difficulty 195 (c) Action-selection time did not influence choice accuracy 197 (d) Absence of relationship between engagement difficulty and choice accuracy 198 (e) Increased action-selection time is associated to decreased sensitivity of choices to Forces 199

(3) Execution of costly actions 202 (a) Middle press exerted force was negatively related to difficulty to engage in middle press 202 (b) Side press exerted force was negatively related to difficulty to engage in side press 204 (c) Performance was negatively related to action-triggering time 204

(4) Summary of behavioral results 205 b) Neurophysiology of the LC 206

(1) Middle press-related activity: encoding of middle press RT and enhancement of engagement-

related processes and of Force exertion 208 (a) LC neurons increased their rate before and during middle press 208 (b) LC neurons encoded middle press RT before middle press onset 209 (c) Middle press activity and engagement throughout the trial: higher pre-middle press activity

was associated to higher probability to engagement in the choice 212 (d) Middle press activity and force exertion: higher pre-middle press activity was associated to

increased middle press exerted force 216 (2) Activity around cues onset 217

(a) Responses to cues onset and choice onset 217 (b) LC neurons positively encoded immediate subjective difficulty to engage in choice around

cues onset 219 (c) LC neurons encoded Rewards but not Forces after cues onset 222 (d) Increased activity of LC neurons around cues onset was associated to increased probability to

choose high-Force option 224 (e) Activity around cues onset had a positive effect on side press exerted force in high-chosen

Force conditions 227 (3) Side-press related LC activity 230

(a) Evoked activity around action onset 230 (b) LC neurons encoded the difficulty of the action during its execution 231 (c) Higher press-related LC activity at trial N was associated to increased Force sensitivity of

choices at trial N+1 234 (4) Inter-trial activity of LC neurons 236

(a) LC activity increased during the ITI following failures to execute the side press 236

III. DISCUSSION 240

A. Behavior: signatures of effort in one- and two-alternative tasks 241 1. Cognitive control exertion for decisions 241 2. Cognitive control for triggering actions 243 3. Physical effort to execute presses 244

5

B. LC activation may signal the end of decision processes 245

C. LC neurons encode subjective difficulty, and possibly effort, to engage in actions and execute them 249

D. LC activity and value-based decision-making 252

E. LC and facilitation of effort exertion: mechanistic hypotheses 257

F. Relationship between noradrenergic facilitation of effort and promotion of behavioral variability 260

G. Modularity of the LC: evidence for differences in electrophysiological properties and consequences for

effort processing 261

H. Conclusions and perspectives 263

REFERENCES 264

LIST OF FIGURES 305

LIST OF TABLES 309

LIST OF ABBREVIATIONS 310

RÉSUMÉ EN FRANÇAIS 311

ANNEX: PHARMACOLOGICAL EVIDENCE FOR THE IMPLICATION OF NORADRENALINE IN EFFORT,

BORDERIES ET AL., 2020 327

6

I. INTRODUCTION

“Success is dependent on effort” – Sophocles

Will you choose to make the effort to read this manuscript, or will you choose to watch

a movie? Every day, several times a day, we face decisions about actions involving effort

exertion (in my example, deciding between reading my manuscript or watching a movie), and,

depending on the result of such decision, effort exertion (if you choose to read it, you will have

to make the effort to read it). In the Cambridge English dictionary, an effort is defined as a

‘physical or mental activity needed to achieve something’. ‘Being an effort’ is defined as being

‘difficult, tiring or boring to do’. In addition, the Oxford dictionary defines effort as a ‘strenuous

physical or mental exertion’. Put together, these definitions emphasize several characteristics

of effort. First, effort is goal-directed, as it aims at achieving an objective. Effort is indeed

usually exerted in order to reach a goal that would not have been reached without effort

exertion, e.g. reading a PhD thesis. Second, effort is aversive (difficult, tiring or boring), and

would be avoided if possible, e.g. you would definitely rather be watching a movie than

reading these words. This introduces that subjects use effort as a decision variable, i.e. effort

influences choices: if the same goal could be achieved through two courses of action, the least

effortful one will be chosen. For example, a student certain that he does not need to study for

a test to score an A grade will probably not study, as it would constitute an effort, while the

goal (scoring an A) could have been achieved without it. Third, making an effort is associated

to spending resources (strenuous) and extensive or intensive effort can lead to fatigue. This

provides a justification for the aversiveness of effort, although the nature of the resource is

not specified by the definitions. Fourth, both definitions draw a line between mental and

physical effort, although this distinction has long been debated. Dewey (1897), posited that

this distinction is merely a distinction based on the nature of the goal (moral or physical) rather

than a psychological one, and that any effort is intellectual.

Effort, in neuroscience, is generally opposed to rewards, i.e. the benefits of actions.

While the brain structures and neurotransmitters involved in reward processing are now well

known and have been extensively studied (see Ikemoto et al., 2015 for a review of the role of

dopamine in reward processing), behavioral and cognitive neurosciences have not yet entirely

solved the question of the neuronal basis of effort, nor of the nature of the resources

7

expensed. This work, however, intends to contribute to this vast subject, by proposing a

candidate neuronal substrate for processing and exerting effort: the locus coeruleus (LC)

noradrenergic brain system. In order to provide the reader with the context of this hypothesis,

the first part of this manuscript will consist in an extensive review, first, of the literature

regarding physical and mental effort, and their common points. It will be followed by a brief

summary of the anatomo-functional properties of the LC, before an extensive review of the

various studies that led to formulating the hypothesis of the implication of the LC in effort

processing. The corpus of the manuscript then includes the reports of two experimental

studies aiming at further exploring the relationship between LC activity and effort processing

and exertion in macaques. I finish with a general discussion summarizing the findings of the

experimental studies, addressing the broader impact of this work, its limitations, and

providing perspectives for future research.

A. Physical and mental effort: characteristics and

implications for decision-making

1. Physical effort

In everyday life as well as in neuroscience research, effort often refers to a physical

requirement. Understandably, most rodent and non-human primate studies on effort

processing have focused on tasks such as force exertion, barrier jumping or repeated lever

presses. This operationalization of effort has been extended to human research, with many

effort studies in humans involving repeated button presses (Porat et al., 2014; Treadway et

al., 2009) or pressing a hand-held dynamometer with variable strength (Bonnelle et al., 2016;

Chong, 2015; Chong et al., 2015; Cléry-Melin et al., 2011; Kurniawan et al., 2010; Prévost et

al., 2010). In this part, I review characteristics of physical effort relevant for decision-making

i.e. its perception as a cost and its use as a decision variable. I then briefly review studies

regarding the mental representation of physical effort. Finally, I provide an overview of the

proxies and physiological measurements than can be used as measurements of physical effort.

8

a) Aversiveness of physical effort: physical effort as a

cost and decision variable

Neuro-economics conceptualize decisions as relying on a cost-benefit trade-off

between the costs and benefits of offers, aiming at maximizing the utility of the chosen option

(Von Neumann and Morgenstern, 2007). Hull (1943) formulated the law of less work, that

states that effort intrinsically carries disutility:

“If two or more behavioral sequences, each involving a different amount of energy

consumption or work, have been equally well reinforced an equal number of times, the

organism will gradually learn to choose the less laborious behavior sequence leading to the

attainment of the reinforcing state of affairs.” Hull, 1944

This law has been widely applied to physical effort. Many experimental studies in

animals and humans have demonstrated the costly nature of effort, by repeatedly showing

that subjects required higher physical effort options to be associated with higher rewards to

choose them (e.g. Cousins and Salamone, 1994; Salamone et al., 2007; Stevens et al., 2005;

Walton et al., 2006). An example of such paradigms and the associated results is presented in

Figure 1.

(A) Walton et al., 2006, Figure 2A (B) Walton et al., 2006, Figure 2B

Figure 1 : Physical effort carries disutility and increased effort requirements need to be

associated with increased rewards to be chosen (adapted from Walton et al., 2006)

(A) Example of task design to demonstrate the disutility of physical effort. Rodents were

offered two alternative courses of action: a high effort-high reward (HR) and a low effort-low

reward option (LR). Both required to press a lever for a certain amount of times (FR, 4 for the

low reward lever, 4 to 20 for the high reward lever) to receive a certain reward (2 pellets for

the low reward lever, 4 for the high effort lever).

(B) Behavioral results: proportion of high reward (HR, y-axis) choices depending on the number

of presses (FR, x-axis) required to get the reward. Increasing press number led to a strong

9

decrease in the proportion of high effort-high-reward choices, indicating that the high reward

was depreciated by the increased physical effort.

b) Mental representation of physical effort

In the past, physical effort perception had been theorized to critically depend on

peripheral sensory afferences from the muscles and organs implicated in the execution of the

effort, but several studies led to the questioning of this idea. Specifically, physical effort

perception was not altered by removing afferent sensory feedback via epidural anesthesia of

afferent nerve severing (Braith et al., 1992; Smith et al., 2003; Zhao et al., 2003; reviewed by

Marcora, 2009). Moreover, it was increased by curare1 administration, which increased the

intensity of the central motor command necessary to maintain constant muscular contraction

(Gallagher et al., 2001). A concurrent hypothesis was proposed: the somatosensory cortex

could directly receive a copy of the motor command (de Morree et al., 2012; Marcora, 2009;

McCloskey, 1981; Proske, 2005). Neuronal disruption studies using transcranial magnetic

stimulation or continuous theta burst stimulation have suggested an implication of the

supplementary motor area (SMA) in effort perception, as disrupting SMA activity led to an

increase in exerted force (White et al., 2013) or a decrease in perceived physical effort (Zénon

et al., 2015).

c) Measuring physical effort

(1) Ratings

Due to the subjective nature of physical effort, many studies have explored potential

proxies for it. The most obvious measurements of subjective physical effort consists in asking

participants to rate their effort exertion (rate of perceived exertion, RPE), and RPEs constitute

a gold-standard measurement (Borg, 1982). Note that the fact that physical effort was shown

not to be perceived via sensory feedback suggests that rating questionnaires should be

formulated with caution so that participants do not rate the physical unpleasantness of the

action (e.g. muscle ache) but the subjective effort.

1 Curares are alkaloids, high affinity antagonists of the nicotinic receptor of acetylcholine. Their fixation on this receptor in the neuromuscular junction blocks the effects of acetylcholine and partially or completely (depending on the dosage) prevent muscular contraction (Manalis, 1977).

10

(2) Behavioral proxies

In the absence of ratings (because the task did not involve ratings, or in the context of

animal research), in tasks offering the possibility to refuse trials, the acceptance rate has been

used to evaluate the effort represented by options (e.g. Bowman et al., 1996; Jahn et al., 2020;

Salamone et al., 2007; Shidara et al., 2005; Varazzani et al., 2015; Walton et al., 2009; Zénon

et al., 2014, Figure 2B).

(A) Zénon et al., 2014, Figure 3D (B) Zénon et al., 2014, Figure 3E

Figure 2 : Acceptance rate decreased when physical effort increased in a force exertion task

consisting in pressing a grip at four different force levels (adapted from Zénon et al., 2014)

(A) RPEs for the 4 conditions across subjects: subjects reported perceiving a greater effort for

greater required force.

(B) Acceptance rates by condition: acceptance rates decreased with increased force.

A comparable strategy consists in evaluating the frequency of choosing an effortful

option when the alternative is a no-effort option (Salamone et al., 2007; Stevens et al., 2005;

Walton et al., 2006). In both cases, those measures reflect the anticipated effort (a priori effort

evaluation) and not the perceived effort (a posteriori effort evaluation), and do not offer trial-

by-trial resolution but rather session-wide assessments by trial category. Alternatively,

another proxy for the subjective effort of options, again available across trials, is the rate of

accepting to replicate a given effort i.e. how often subjects accept to replicate a given effort

for different amounts of reward (Zénon et al., 2014).

(3) Physiological proxies

Pupil diameter was proposed as a proxy for physical effort after accounts of increased

pupil diameter in response to increase mental effort (see part I.A.2.g)(2)). In a force exertion

task, Zénon and colleagues (2014) showed that pupil diameter during force exertion scaled

with the required force (after an initial dip that was identical across conditions, Figure 3). A

similar result was obtained in monkeys performing a force task (pressing a grip to get a reward)

(Varazzani et al., 2015). Moreover, by asking subjects to replicate their presses, the study

11

showed that the intensity of the pupil dilation during the first press negatively predicted the

probability to accept to replicate the press, even after factoring out the effects of the strength

applied during the first press. Coherently, pupil dilation predicted RPEs, even when including

the exerted force in the model. Altogether, the results of this study indicate that pupil

diameter could be a good proxy for the subjective effort of individual trials. Such a relationship

between physical effort and pupil diameter is of particular interest for this manuscript

considering the known relationship between LC activity and pupil diameter (see part I.B.2.b)).

Heart rate, oxygen uptake and blood lactate have also been proposed as a proxies for

subjective physical effort. Several studies have demonstrated a correlation between RPEs and

such peripheral measures reflecting metabolism (Andrade et al., 2020; Da Silva et al., 2020;

Tibana et al., 2019, Figure 4). However, due to slow variations and, for blood lactate

measurement, invasiveness of the measurement (blood sampling), such measures lack the

temporal resolution required to evaluate trial-by-trial physical effort in the context of brief

and relatively low-intensity actions involving few muscular groups (pressing a grip for

instance).

(A) Zénon et al., 2014, Figure 3A (B) Zénon et al., 2014, Figure 3B

(C) Zénon et al., 2014, Figure 3C (D) Zénon et al., 2014, Figure 4B

Figure 3 : During force exertion, after an initial dip, pupil diameter increased and was positively

correlated with the exerted force (figure adapted from Zénon et al., 2014).

(A) Pupil diameter, aligned onto action onset (grip press), by force category. After press onset,

pupil diameter decreased then increased, with a stronger increase for bigger requested forces.

(B) Mean pupil response (z-scored) by condition. The pupil response was bigger for bigger

requested forces (note that the tendency for an increase was absent between the two lowest

forces).

(C) Mean exerted force, aligned on action onset, by force category (by proportion of the

Maximum Voluntary Contraction – MVC).

(D) Relationship between grip force and pupil response, by subject.

12

(A) Andrade et al., 2020 Figure 2A (B) Andrade et al., 2020, Figure 2B

Figure 4: Positive relationship between RPEs and (A) oxygen uptake (percentage of maximum

uptake, VO2max) and (B) heart rate (percentage of maximum heart rate, HRmax) (adapted from

Andrade et al., 2020).

To summarize, physical effort, materialized by actions mobilizing muscular groups, is

perceived as aversive by subjects (animals and humans), and therefore tends to be avoided if

an alternative associated with less physical effort and sufficient reward is offered. Mental

representations of physical and therefore subjective effort perception appear, interestingly,

not to rely on somatosensory information but to critically depend on the SMA. Subjective

physical effort can be measured, or at least approximated, by asking subjects to rate their

exertion (RPEs). At the session level, the acceptance rate, the rate of choice of the high effort

option when a low effort option is available, or the willingness to replicate the effort can

provide some insight on the perceived physical effort of conditions. Regarding physiological

proxies, pupil dilation during the action is frequently used to assess the subject effort

perceived by subjects, although some have shown that metabolic indices such as heart rate,

oxygen consumption and blood lactate also constitute satisfying proxies.

2. Mental effort

a) Nature of mental effort: definitions

Gaillard defined mental effort as ‘energy mobilization in the service of cognitive goals’

(Gaillard, 2001, 1993) and Kool and Botvinick (2018) as the ‘subjective intensification of mental

activity mediating between how well an organism can potentially perform a task and how well

it actually performs on that task’. In other words, subjects perform mental effort to maximize

subjective utility.

Recent accounts of mental effort have grouped its subcategories under the concept of

cognitive control, defined as a set of cognitive processes facilitating goal-directed behavior by

promoting the active use of task-rules (Cohen et al., 2001; Dixon and Christoff, 2012; Posner

(B)

13

and Snyder, 2004 - reedited version of 1975 seminal paper). Shenhav and colleagues (2017)

proposed that ‘cognitive control can be viewed as the force through which cognitive effort is

exerted’.

b) Categories of tasks requiring cognitive control

Posner and Snyder first restrained the definition of cognitive control to overcoming

strong response biases. Such inhibition of response tendencies can indeed be tested using the

Posner task, a task consisting in detecting the position of a target presented after a distractor,

either invalid or valid (Figure 5A). Valid distractors predict the position of the target, while

invalid distractors attract attention to another location. Subjects have a bias towards

indicating a target position that follows the position of the distractor.

Recently, the notion of cognitive control has however been theorized to include a

larger range of cognitive processes crucial for complex executive tasks: information updating,

inhibition of default responses, and mental set-shifting (Miyake et al., 2000). Updating refers

to the ability to monitor upcoming information, and erase no outdated information. A classical

task mobilizing updating capacities is the N-back task (Kirchner, 1958). The task consists in

presenting a series of stimuli, usually letters, to the participants, who have to indicate whether

the current stimulus is identical to the one presented N trials back (Figure 5B). The N-back task

thus involves trial-by-trial updating of the information stored in working memory, which is a

form of mental effort.

Inhibition is the capacity to suppress response biases when required, i.e. default,

automatic or dominant responses. Such ability can be tested in the aforementioned Posner

task (Figure 5A), or using the Stroop task (Stroop, 1935), a cognitive interference task. Subjects

are required for example to give the color of a written word. The information provided by the

word itself can be congruent (e.g. the word green written in green), and facilitate the color-

naming, or incongruent (e.g. the word green written in blue), and render the color naming

more difficult (Figure 5C). In incongruent trials, both the Posner and Stroop tasks require

ignoring the bias towards responding according to the distractor, thus involving inhibition, a

subcategory of cognitive control.

(A) Posner task: Hayward and Ristic, 2013, Figure 1

(B) N-Back task: Lamichhane et al., 2020, Figure 1

14

(C) Stroop task: Nathan et al., 2012, Figure 2b

(D) Set-shifting task: Monsell et al., 2003, Figure 5

Figure 5: Tasks used to test the different components of cognitive control

(A) The Posner task (figure adapted from Hayward and Ristic, 2013). The objective of this task

is to indicate the position of a target (here, a cross) after its apparition. Importantly, just before

the target appears, a cue attracting attention towards one of the two positions is presented

(here, the bold square). The cue can either be valid (indicate the future position of the target)

or invalid (indicate the other position).

(B) The N-Back task (figure adapted from Lamichhane et al., 2020). The N-back consists stimuli

being presented in a row (e.g. letters) and subjects have to report whether the current one is

identical to the one presented N trials back. The task becomes more difficult for higher N.

(C) The Stroop task (figure adapted from Nathan et al., 2012). The color-word Stroop task

presented here consists in presenting a series of color words (blue, green etc) written in various

colors. The subjects have to report the color of the written word. The written word can be

congruent (green written in green, subjects have to respond ‘green’) or incongruent as

presented in the figure (green written in red, subjects have to respond ‘red’).

(D) Set-shifting task (figure adapted from Monsell et al., 2003). Subjects are presented with a

series of numbers from 1 to 10, displayed on a square. The square can be in two positions (left

side of the figure vs right side of the figure). If the square is placed like the square displayed on

the left of the figure, the subject has to tell whether the number presented is smaller than five.

If the square is in the position presented on the right side of the figure, the subject has to

indicate whether the number presented is even. RT: reaction time.

Finally, switching (set-shifting) corresponds to the capacity to disengage from the task

at hand or from the current task rules to commit to a new task set or use new rules (switch

task and/or rules, Monsell, 1996; Monsell et al., 2003). Set-shifting tasks typically involve

responding to stimuli according to two possible rules. The rule that has to be used for each

stimulus is indicated by a cue. For example, Monsell and colleagues (2003) used a task

consisting in presenting a series of numbers from one to ten to the subjects. Numbers could

be presented superimposed onto two different shapes, indicating the rule that had to be

followed to answer: subjects either had to tell if the number was even or if it was smaller than

15

five (Figure 5D). Switching between task rules requires engaging cognitive control. It is

important to note that task-switching is quite dependent on inhibitory control in a large

variety of tasks. Indeed, inhibitory processes are required to deactivate the no longer needed

task rules, a process known as backwards inhibition (Dajani and Uddin, 2015; Klimesch, 2011;

Mayr and Keele, 2000; Zink et al., 2019).

To summarize, tasks requiring cognitive control exertion can generally be put into

three possible sub-categories: updating (working memory) tasks, in which control is used to

renew the information stored in working memory, inhibition tasks, in which control is required

to block default responses, and switching tasks, in which task rules change at a fast pace and

control is required to adjust.

c) Sub-processes of cognitive control

Analyzing the characteristics of the tasks requiring cognitive control exertion allowed

Shenhav and colleagues (2013) to dissect it and identify three core component functions:

regulation, specification and monitoring. Regulation refers to the capacity of the control

system to “influence lower-level information-processing mechanisms” by adjusting the identity

of the attended information, the target parameter for the regulation, and the intensity with

which the processing system attends to that information, i.e. the degree of displacement of

the parameter. For example, in a Stroop task, regulation is involved in attending to the correct

stimulus (in case of a color task, the color of the word and not the word itself) and modulating

the strength with which it is inspected. Beyond this specific example, the regulation

component can modify various other parameters. For instance, it can modulate the thresholds

and biases implemented for decision-making (Bogacz et al., 2006; Wiecki and Frank, 2013),

the searching patterns implemented for searching through visual stimuli or memory

information (Olivers et al., 2011; Polyn et al., 2009), or regulate negative emotions (Gross,

2011; Johns et al., 2008).

Specification refers to decisions about engagement in control demanding tasks, about

goals that should be pursued and tasks that should be executed, and about control allocation.

Consequently, this process specifies identity and intensity for the regulation component.

Specification is distinct and upstream from regulation since specification determines the

identity and intensity of the control signal while regulation merely implements them. Details

16

regarding decisions to allocate control and their implementation will be provided in part

I.A.2.d)(3).

Monitoring happens downstream from regulation, as it serves to process information

about the result of the control exertion on task execution in order to optimize future

specification, using feedbacks such as reaction time (RT), reward, pain, delay to reward, errors,

or other negative feedbacks. Note that the monitoring unit does not provide the feedback

about the control exertion, but responds to that feedback. For instance, Botvinick and

colleagues (Botvinick, 2007; Botvinick et al., 2004, 2001) focused on cases in which a response

conflict is detected (possibly by monitoring RT), which can be considered as signaling

insufficient control exertion. In such situations, the monitoring system needs to detect the

conflict and adjust the parameters of the specification system, in a form of reinforcement

learning. In that frame, allocating control can be reframed as an optimization problem, with

the objective of maximizing short- and long-term utility (Bogacz et al., 2006; Dayan, 2012; Hazy

et al., 2007; O’Reilly and Frank, 2006; Todd et al., 2008; Yu et al., 2009).

To summarize, the cognitive control system has been hypothesized to be constituted

of three subunits: a specification unit deciding on the identity and intensity of the control, a

regulation unit implementing them, and a monitoring unit receiving feedback on the results

of the control exertion in order to adjust the decisions made by the specification unit.

d) Cognitive control and decision-making

(1) Cognitive control and optimization of decisions

Numerous behavioral studies in healthy patients (e.g. Hinson et al., 2003; Missier et

al., 2012, 2010), patients with frontal cortex lesions (e.g. Eslinger and Damasio, 1985; Manes

et al., 2002) or with attention-deficit hyperactivity disorder (ADHD, e.g. Mäntylä et al., 2012),

as well as brain imaging studies (e.g. Clark et al., 2004; Coricelli et al., 2005; De Martino et al.,

2006; De Neys et al., 2008) have pointed towards a link between various subcategories of

decision-making and cognitive control capacity. In addition, pupil dilation, a proxy for mental

effort (see part I.A.2.g)(2) for review), has been shown to increase during decisions (Beatty,

1982; de Gee et al., 2014; Gilzenrat et al., 2010; Lempert et al., 2015; Nassar et al., 2012; Figure

6A).

17

Removing any possible confound with the motor preparation of the response,

Einhäuser and colleagues (2010) asked subjects to make covert choices between cues (digits)

sequentially presented on screen, and to report the chosen digit only at the end of the trial.

Although the decision to choose a digit when it was presented did not involve immediately

producing a movement to report the choice, pupil diameter specifically increased during

presentation of the digit later reported as chosen. Pupil diameter has also been reported to

increase around the time of the end of decision processes, materialized by a motor action to

respond, in tasks consisting in continuously tracking perceptual stimuli and reporting

detecting changes (Einhäuser et al., 2008; Hupé et al., 2009). Interestingly, pupil diameter has

been shown to be larger during decisions in which subjects end up responding against their

bias (de Gee et al., 2020, 2017, 2014) or have a low confidence, i.e. difficult choices (Lempert

et al., 2015). Altogether, these results suggest that cognitive control exertion, approximated

by pupil dilation, can improve decision-making, particularly by facilitating responses away

from the default (responding according to existing biases).

(A) Lempert et al., 2015, Figure 3A (B) De Gee et al., 2017, Figure 2A middle pannel

Figure 6: Pupil diameter increases during decisions, especially when subjects will respond

against their bias.

(A) Time course of pupil dilation, aligned on stimulus onset, in an auditory decision task. Pupil

increased after stimulus onset, during the decision period. Dashed lines represent the standard

error of the mean (SEM), and dashed line represents mean across subjects (adapted from

Lempert et al., 2015).

(B) Quantification of response biases in trials with small task evoked pupillary responses (TPR,

blue) and big TPR (red) (median split), in a visual detection task. Increased task-evoked

pupillary responses (TPR) were associated with decreased response biases (adapted from De

Gee, 2017).

The predominant theory regarding the control mechanism applied to decision-making

is the ‘dual process theory’ (e.g. Epstein, 1994; Evans, 1984; Evans and Over, 2013; Evans et

al., 2007; Goel, 1995; Kahneman and Frederick, 2005; Sloman, 1996; Stanovich and West,

2000). This theoretical framework intends to explain failures to think rationally observed in

18

human behavior. De Neys and Glumicic (2008) provide an insightful example of such failure

by citing the case of a physical aggression in a European country being attributed to an ethnical

minority (African immigrants) during a period of high racial tensions. Although the small

relative proportion of African immigrants in the general population made the probability of

the aggression being perpetrated by one of them highly unlikely, the African community was

immediately blamed due to irrelevant priors regarding that community. The example is

particularly striking considering that the aggressors were later found to be Caucasian

Europeans.

One general control mechanism for decision-making has been proposed, with small

disagreements over details among authors. Dual process theory states that two systems using

different sources of information and operating at different paces compete to formulate a

choice. First, the heuristic system, relying on prior beliefs and knowledge, operates fast and

automatically, as its operations are not very computationally demanding. Conversely, the

analytic system uses demanding computations, is slower, and allows rational and logical

reasoning. In the previous example, the heuristic system pointed towards accusing African

immigrants, while the analytic system would have indicated that this reasoning was illogical.

In that specific example, a conflict arises between the two systems, and cognitive control is

believed to be required for the analytic system, using costly computations, to override the

belief-based response of the heuristic system.

There is still an open debate on whether the exertion of control allows merely detecting

the conflict i.e. being aware of one’s biases and prejudices and heuristic computations (Evans,

1984; Kahneman and Frederick, 2005), or overriding the default in case of a conflict, i.e.

favoring analytic computations (Epstein, 1994; Shenhav et al., 2013; Sloman, 1996). The latter

intuition relies on post-decisional reports from participants that they ‘knew’ their answer was

wrong but chose it because it ‘felt’ right (Denes-Raj and Epstein, 1994; Epstein, 1994; Epstein

and Pacini, 1999), suggesting that being aware of one’s priors is not costly, but overriding them

is. Some have intended to solve that debate by asking subjects to ‘think aloud’ while making

a decision (De Neys and Glumicic, 2008). Participants were asked to answer a question by

choosing between two alternative options, one answer being likely to be chosen if they

predominantly used the heuristic system and the other if they predominantly used the analytic

system. For example:

19

“In a study 1000 people were tested. Among the participants there were 5 men and 995

women. Jo is a randomly chosen participant of this study.

Jo is 23 years old and is finishing a degree in engineering. On Friday nights, Jo likes to go out

cruising with friends while listening to loud music and drinking beer.

What is most likely?

a. Jo is a man

b. Jo is a woman” (De Neys and Glumicic, 2008)

As expected from classical findings regarding sub-optimality of decision-making and

susceptibility of decision-makers to priors, participants, in most cases, picked the answer that

followed their prior regarding gender-related behavior, thus ignoring the low probability of

‘Jo’ being male, due to the large predominance of women in the described group. The thinking

aloud procedure however provided further insight on the interplay between the heuristic and

analytic systems, as authors report participants phrasing their reasoning as, e.g.: “ … because

Jo’s drinking beer and loud I guess Jo’ll be a guy, although there were more women…”.

Participants thus referred to the relevant information, and authors argue that they were

therefore aware of the result of the analytic computation but choose to ignore it, because too

much cognitive control would be needed to override the default answer from the heuristic

system. Consequently, their results are in favor of control serving to override heuristic

computations, not detecting a conflict between the two systems.

Additionally, comparison between the decision-making tasks highly dependent on

cognitive control (e.g. Dretsch and Tipples, 2008; Hinson et al., 2003; Missier et al., 2010; Shiv

and Fedorikhin, 1999) and tasks requiring low levels of cognitive control exertion (e.g.

Glöckner and Witteman, 2010; Hogarth, 2005; Stanovich and West, 2000) for good

performance has allowed identifying features of both categories (Botvinick and Cohen, 2014;

Cohen et al., 1990; Missier et al., 2012). Shiffrin and Schneider (1977) theorized that

information processing can be more or less automatized, and this continuum of automaticity

in task solving could suffice to categorize tasks along the high to low cognitive control demand

tasks axis. For instance, tasks requiring high levels of cognitive control fall under the category

of tasks requiring extensive processing of decision options, and consequences of choices.

Conversely, tasks requiring low amounts of cognitive control can be solved using low-level

20

computations, e.g. perceptual cues, and automatic processing, e.g. heavily practiced tasks,

learned associations, and recognition processes.

To summarize, decision-making is frequently associated with cognitive control

exertion, especially in the context of decisions requiring extensive information processing, as

shown by studies using pupil dilation as a proxy. Specifically, control is (probably) exerted to

override default responses emerging from low-level computations (heuristic system) and

decide based on more costly and complex computations (analytic system), at least when a

conflict arises between those two alternative responses.

(2) Cognitive control as a decision variable per se

“We like to solve problems easily. We can do so best if we can fit them rapidly into a

satisfactory category and use this category as a means of prejudging the solution.... So long as

we can get away with coarse overgeneralizations we tend to do so. Why? Well, it takes less

effort, and effort, except in the area of our most intense interests, is disagreeable.”

Allport, 1954

Decision-making involves weighing the benefits of the options offered against their

predicted costs (Kahneman and Tversky, 1979; Stephens and Krebs, 1986). Cognitive control

acts as a decision variable, as the “law of least effort” (Hull, 1943) applying to physical effort

applies to mental effort as well (Zipf, 1949). We saw in the previous part (I.A.2.d.1) that

applying cognitive control is beneficial for decision-making processes, especially complex

ones, requiring integration of important amounts of information to produce an accurate

response. However, in that context, humans tend to rely on simplifying strategies of

information gathering and processing (Gigerenzer, 2010; Gigerenzer and Goldstein, 1996;

Sidarus et al., 2019; Simon, 1955; Tversky and Kahneman, 1974). These heuristics (mental

shortcuts) are believed to serve the purpose of reducing the cost of control that would be

associated with the alternative optimal but computationally demanding decision strategy. In

that sense, decision-makers compute a trade-off between the cost of control exertion and the

optimality of their decisions. In that part, I will show how this finding can be generalized

beyond decision-making tasks, and review evidence that applying control is aversive and

serves as a decision variable.

21

Applying cognitive control is aversive (Kool et al., 2010; Kool and Botvinick, 2018,

2014), and courses of action associated with higher cognitive control demands are less likely

to be chosen. For example, Kool and colleagues (2010) designed a choice task to test for

cognitive demand avoidance. The task consisted in repeatedly choosing between pairs of cues,

without previous instructions on their characteristics. Choosing either cue revealed a number

between one and nine. The color of its display indicated if the subject had to tell whether the

number was greater than five (blue number, ‘yes’ or ‘no’) or whether the number was even

(purple number, ‘yes’ or ‘no’, Figure 7A). Importantly, one cue led to a low probability of task

switching, i.e. low-demand cue, and the other cue led to a high probability of switching

assignment, i.e. high-demand cue. Authors report that, throughout a session, subjects learned

to choose the low-demand cue more often (Figure 7B), indicating that subjects perceived the

high-demand cue as carrying an intrinsic negative utility. In this example, mental effort was

probably not required for the choice between cues itself, nor did it influence its quality, as the

cognitive effort was applied as a consequence of the decision, depending on the difficulty of

the given assignment. The rule of least effort seems to have applied here, and to be

extendable to the specific context of decisions requiring investing cognitive control.

Further proof of the negative utility of cognitive control exertion stems from the

observation that increased incentive (offered reward or intrinsic motivation) can compensate

the aversiveness of effort, as increased incentive in cognitive effort tasks leads to increased

performance, within subjects. Such an effect was demonstrated in various types of tasks

relying on the different components of cognitive control:

- updating (working memory tasks, Beck et al., 2010; Heitz et al., 2008; Jimura et al.,

2010; Kennerley and Wallis, 2009; Libby and Lipe, 1992; Taylor et al., 2004; Westbrook et al.,

2013),

- inhibition (Stroop task: Krebs et al., 2010, Posner task: Engelmann et al., 2009;

Modified Stroop task: Padmala and Pessoa, 2011, see Figure 8A, Dixon and Christoff, 2012,

Stop signal response task: Boehler et al., 2014; Leotti and Wager, 2010; Padmala and Pessoa,

2010),

- set-shifting (Aarts et al., 2010; Dixon and Christoff, 2012; Kleinsorge and Rinkenauer,

2012; Nieuwenhuis and Monsell, 2002; Umemoto and Holroyd, 2015). More generally, it was

demonstrated that intelligence test scores could be influenced by the motivational state of

the subjects (Duckworth, 2011).

22

(A) Kool and Botvinick, 2018, Figure 1A (B) Kool and Botvinick, Figure 1B

Figure 7: Behavioral demonstration of the aversiveness of mental effort (figure adapted from

Kool and Botvinick, 2018)

(A) Task design. Subjects were repetitively asked to choose between pairs of previously

unknown cues. When a cue was chosen, it revealed a number from 1 to 9 written either in blue

(rule: ‘state whether the number is greater than 5’) or purple (rule: ‘state whether the number

is even’). Each pair of cues (8 pairs throughout task) was associated to either a high (high-

demand cue) or low (low-demand cue) probability of task (and therefore number color)

switching.

(B) Behavioral results: throughout the task (x-axis, trial number), subjects learned to pick

preferentially the cue associated to a low cognitive demand (y-axis, probability of choosing the

low-demand cue).

Note that these enhancements of performance in response to increased incentive

were not observed in every situation, as a few conditions had to be fulfilled. First, adjusting

the intensity and direction of the control exerted had to be beneficial for task performance

(Camerer and Hogarth, 1999). For instance, task difficulty had to be compatible with the

individual skills and capacities of the subjects (Awasthi and Pratt, 1990; Bonner et al., 2000;

Rydval and Ortmann, 2004, 2004). Moreover, excessively big or small reward size could be

detrimental to performance, as small rewards could seem insulting (Gneezy and Rustichini,

2000) and huge rewards could make subjects ‘choke under pressure’ (Ariely et al., 2009;

Baumeister, 1984; Beilock, 2010; Bonner et al., 2000; Camerer and Hogarth, 1999; Mobbs et

al., 2009; Samuels and Whitecotton, 2011; Worthy et al., 2009). Nonetheless, beyond these

relatively anecdotal exceptions, the general rule that rewards incentivize control allocation

provides further evidence for the negative utility of control.

(A) Shenhav et al. 2017, Figure 2D top panel

(B) Shenhav et al. 2017, Figure 2D bottom left panel

(C) Shenhav et al. 2017, Figure 2D bottom right pannel

23

Figure 8 : Results of the modified Stroop task used by Padmala and Pessoa (2011). RTs were

shorter, accuracy was higher and difficulty had lesser effects on RT and accuracy when subjects

were rewarded.

(A) Selective attention task. Subjects were first informed by a cue of the potential reward of

the following trial. They were then presented with images either of houses or of buildings. A

task irrelevant word was written on top of the image, and could either be incongruent (house

written on a building image), congruent (house written on a house image) or neutral (random

word). Subjects had to tell whether the image represented a house or building. They were told

to respond as quickly and accurately as they could. NB: the numbers presented on the fifth

frame of the task are a feedback on the number of points earned in the trial (top) and since the

beginning of the task (bottom)

(B) RTs of subjects by difficulty (congruency of the irrelevant word) and reward category

(rewarded or not). Subjects were overall faster in rewarded trials and interference effects were

smaller.

(C) Accuracy of subjects by difficulty (congruency of the irrelevant word) and reward category

(rewarded or not). Subjects were overall more accurate in rewarded trials and interference

effects are smaller.

As a consequence, effort allocation can be analyzed in a neuro-economics frame

(Westbrook and Braver, 2015, 2013). For example, effort-discounting tasks, in which subjects

choose to execute different levels of difficulty of cognitive tasks to obtain different amounts

of offered reward, can allow determining individual indifference points, i.e. the subjective

value of each level of mental effort. Such studies were conducted both with healthy subjects

(Dunn et al., 2016; Kool et al., 2010; Schouppe et al., 2014; Westbrook et al., 2013) and with

psychiatric patients (schizophrenic patients: Culbreth et al., 2016). Note that behavioral

analyses in these tasks have allowed ruling out an interpretation of preferences for low effort

in terms of error (risk) avoidance. An illustration of an example task used to determine the

indifference point for an N-Back task (Westbrook et al., 2013) is provided in Figure 9.

Altogether, these results provide further proof that humans routinely choose not to

invest as much cognitive control as they potentially could, but rather compute a cognitive

control-reward trade-off to decide on the amount of control that is worth investing. In the

specific context of applying cognitive control to make accurate decisions, subjects make a

24

trade-off between cognitive-control related costs and decision accuracy-related benefits. In

that frame, decisions using simplifying heuristics, therefore risking inaccuracy, are not

necessarily suboptimal, as they putatively, in certain environments, correspond to the best

trade-off between expensed control resources and expected rewards (Gigerenzer, 2010; Todd

and Gigerenzer, 2012). This theoretical frame, contradicting the postulate that the best

strategy is the one allowing for maximum possible accuracy, is called ‘bounded rationality’

(Gigerenzer, 2008; Simon, 1997, 1956; Todd and Gigerenzer, 2012). As its name indicates, it

assumes that human rationality is inherently limited by the availability of the resources that

are required to make rational decisions. One major assumption for theory is therefore that

applying cognitive control to any process consume certain resources (see part I.A.2.e) for

discussion on the theories regarding the nature of such resources).

(A) Westbrook et al., 2013, Figure 1, top left panel

(B) Westbrook et al., 2013, Figure 1, bottom left pannel

Figure 9: Cognitive effort titration task (adapted from Westbrook et al., 2013).

(A) On-screen task display. Subjects had to choose between two possible offers involving

different N-back task difficulty levels for different amounts of money.

(B) Depending on the decision made at each trial, a new proposal was made to the subjects, in

order to progressively obtain a more precise evaluation of the subject’s indifference point

(titration). The used algorithm is shown here.

To summarize, cognitive control exertion is aversive, and the theory of bounded

rationality states that subjects compute a cost-benefit trade-off for allocating control.

Consequently, increased rewards can increase control allocation. This strategy does not allow

systematically maximizing rewards, but instead maximizes utility.

(3) Decisions to allocate control and how much control to

allocate: the expected value of control (EVC) theory

I established earlier that exerting control represents a cost, and that decision-makers

compute a trade-off between the expected rewards and costs from potential strategies and

25

intensities of control exertion. The following paragraph will thus focus on the core principles

of the most recent models of mental effort allocation.

Early work on decision-making strategies (Beach and Mitchell, 1978; Payne et al., 1988)

has demonstrated that people trade decision accuracy (probability to make the correct choice

i.e. prospective rewards) for mental effort, i.e. they compute a cost-benefit analysis. Smith

and Walker (1993) have summarized such accounts under the ‘labor theory of cognition’,

adapted from the classical economics ‘labor theory’2. More recently, several teams have used

hierarchical reinforcement-learning theories and artificial intelligence research to formalize

and generalize this theory. They also proposed neuronal bases for their implementation,

which I will not detail here outside of the context of the role of the LC (Dayan, 2012; Frank and

Badre, 2012; Holroyd and McClure, 2015; O’Reilly and Frank, 2006; Shenhav et al., 2017, 2013;

Verguts et al., 2015).

The notion of rational meta-reasoning or resource-rational analysis, i.e. methods to

decide to allocate computational resources, comes from research aiming at developing

intelligent systems interacting with their environment. Such systems have to make the best of

their finite resources (Hay et al., 2014) and have therefore been engineered to maximize the

value of their computations (VOC), i.e. the added utility obtained when more computational

resources are allocated minus the computational resources spent (e.g. Random Access

Memory). This concept was later extended to cognition and brain computations (Griffiths et

al., 2015; Lieder et al., 2014, 2012; Lieder and Griffiths, 2017). Lieder and colleagues have

proposed, and experimentally shown, that individuals learn to predict the VOC of individual

cognitive operations, and select computational strategies (sequences of operations) in order

to maximize the VOC (Figure 10A&B). Such accounts, however, consider operations as more

costly if they merely take longer (i.e. constitute an opportunity cost), while several studies

have shown that subjects can experience different levels of mental effort even when the

duration of trials does not differ (Dixon and Christoff, 2012; Kool et al., 2010; Westbrook et

al., 2013).

2 Labor theory states that the economic value of a good or a service is the consequence of the amount of labor that was required to produce it (theory by Smith, Ricardo and Marx, summarized in Cleveland and Ayres, 2004).

26

The EVC theory complements the VOC theory by introducing a cognitive control

intensity parameter. Practically, as mentioned before (see part I.A.2.c), Shenhav and

colleagues (2017, 2013) proposed that signals implemented by the regulation unit of control

have two dimensions: an identity (what control signal should be selected) and an intensity

(vigor of the control signal). The specification unit of the control system decides upon the

identity and intensity of the control signal based on the expected costs and outcomes, and the

regulation unit implements it. After control execution, the monitoring unit receives feedback

on the consequences of the process (e.g. expensed resources, payoffs) and adjusts the

parameters used by the specification unit to make future decisions about control allocation.

For example, when practicing a Stroop task (giving the name of a picture rather than reading

the word written on top of the picture, Figure 10C), the identity of the control corresponds to

what the subjects is attending to (the picture) while the intensity is the strength with which

the subject attends to the stimulus (Figure 10D). Note that one of the crucial assumptions of

that theory is that increasing the intensity of the control leads to increased costs, thus

justifying that the system cannot invest maximal control to get maximal reward, but has to

compromise. Under that assumption, subjects should select both the strategy and control

intensity that have the highest utility (reward minus control costs, including, but not limited

to, time costs).

(A) Lieder et al., 2014, Figure 2 (B) Shenhav et al., 2017, Figure 2a

(C) Shenhav et al., 2017, Figure 2d top panel (D) Shenhav et al., 2017, Figure 2b

Figure 10 : The VOC model and the EVC model

(A) Lieder and colleagues designed a task in which subjects had to choose between two

strategies (swapping numbers or creating sublists) to sort lists of numbers of different lengths

and different ‘sortedness’ (number of misplaced numbers in list), after training on both

strategies (adapted from Lieder et al., 2014)

(B) The VOC model, in the list sorting task, states that subjects select their sorting strategy by

evaluating, based on the length and ‘sortedness’ of the assigned list, the payoff minus cost

(spent time) of each strategy (adapted from Shenhav et al., 2017).

(C) Task used for the example: picture-naming Stroop task (adapted from Shenhav et al., 2017).

Subjects have to name the picture presented on screen (house or building) and ignore the word

27

printed on top (congruent, incongruent or neutral). NB: 2000 and 42000 are the points earned

by the subject in the current trial and throughout the session.

(D) When doing a picture-naming Stroop task, the identity of the control can be reading the

word, which leads frequently to a negative feedback (due to incongruent trials), or naming the

picture, which often leads to positive feedback. Complementarily, when attending to the name

of the picture, subjects can apply different amounts of control (intensity). Increased control will

lead to increased probability of positive feedback (adapted from Shenhav et al., 2017).

The EVC was mathematically formalized as follows:

EVC (signal, state) = [Σi Pr(outcomei|signal, state).Value(outcomei)] – Cost(signal)

The EVC is a function of two distinct variables, state (or current state, the current situation,

i.e. task difficulty, motivation, etc.) and signal (intensity and identity). Pr is the anticipated

(evaluated) probability of a particular outcome (or future state) following control exertion, in

the current state and when applying the specific control signal. The EVC is the expected value

of the outcome (sum of values of potential outcomes weighed by their probabilities) minus

the cost of the control applied to get these potential outcomes with such probabilities.

Note that the EVC does not only take into account the immediate rewards but also

more distant expected rewards. This can be formalized as follows, in a recursive fashion:

Value(outcome) = ImmediateReward(outcome) + γ.maxi[EVC(signali,outcome)]

The value of the outcome is the sum of the immediate rewards of the outcome and the

discounted (γ, ranging from 0 to 1) maximum expected value of the control that can be applied

in the future states over all feasible signals.

To specify the control signal, the control system must identify the signal identity and

intensity that maximize the EVC (Signal*):

Signal* ← maxi[EVC(signali, state)]

28

The regulation unit then implements this signal maximizing the value of the control. The

monitoring unit is responsible of assessing, through feedback from actual outcomes and

resource expenditure, that Signal* is indeed optimal, and, in the case of sub-optimality,

changes in payoffs or in difficulty, shall order adjustments from the specification system

(Figure 11).

(A) Shenhav et al., 2013, Figure 4A (B) Shenhav et al., 2013, Figure 4B

Figure 11: Representation of the determination of the optimal control intensity signal in a

certain state (payoffs and costs) and in case of increases in payoff or in difficulty (adapted from

Shenhav et al., 2013).

The mathematical equation of the EVC states that the EVC (blue) is the expected payoff of the

control (green) minus the expected cost of the control (red). For a given situation (given payoff

and costs, dashed lined), the specification system determines the intensity of the control that

maximizes the EVC (dashed arrow). Note the EVC can become negative for high intensity

control, i.e. exerting maximum control may carry negative value even though it maximizes the

payoff. In extension, in some situations (very difficult or very low payoff tasks), exerting any

control may carry less utility than exerting no control at all (for example refusing the task).

(A) In case of increase of the payoff (green arrow, full lines), the optimal control intensity

increases (left side shift, full black arrow).

(B) In case of an increase of the difficulty, the probability to get the reward in case of identical

control exertion decreases, i.e. the payoff function is lower, and the control intensity

maximizing the EVC increases (right side shift, full black arrow).

Many other teams have proposed their own version of hierarchical reinforcement

learning models for control allocation, but the core principles remain the same. However,

although the model from Shenhav and colleagues (2017, 2013) is quite exhaustive, it does not

formulate hypotheses regarding the shape of the cost function. This gap was breached by the

work of Kool and Botvinick (2014) that extended economic labor-leisure tradeoff theories

(Nicholson and Snyder, 2012) to mental effort exertion. Labor-leisure models were initially

designed to predict how workers would distribute their time between working hours and

29

leisure hours based on offered wages. Research on labor-leisure allocation before and after

manipulating wages (money earned per hour), while compensating for the difference with a

fixed income (fixed salary plus wage multiplied by number of hours) showed that utility curves

have a concave shape (see Figure 12 for details).

Kool and Botvinick (2014) created a paradigm for studying the mental labor-mental

leisure tradeoff in a comparable fashion. Throughout the testing session, subjects were

offered a choice between a high-demand N-back memory task, requiring exerting cognitive

control, and a low-demand reaction task (responding to specific images with a button press).

Participants performed the task during two separate sessions. The high-demand task was paid

in candy, with different payment systems (base salary and wage per trial, paid in candy)

between sessions. The low-demand task was not paid. The behavior of the subjects after

income-compensated increases or decreases in wage (reward per completed trial of the high-

demand task) confirmed the concave shape of the utility function, suggesting that the cost

function for cognitive control may be following such concave shape.

(A) Kool and Botvinick, 2014, Figure 1A

(B) Kool and Botvinick, 2014, Figure 1B

(C) Kool and Botvinick, 2014, Figure 1C

(D) Kool and Botvinick, 2014, Figure 2A

(E) Kool and Botvinick, 2014, Figure 2B

Figure 12: Labor-leisure theory, in economics and applied to cognitive control (figure adapted

from Kool and Botvinick, 2014).

(A,B,C) Labor-leisure theory in economics.

(A) Concave utility surface representing the utility of each individual labor-leisure combination.

Each black line represents iso-utility. Based on the available hours during which the worker can

work per period (T), and the hourly wage offered (w) a budget constraint is defined (plane in

top plot, projected as a line on bottom plot of figure A). The maximum income is therefore wT

(in case the worker works for T hours). To decide upon his working time, labor-leisure theory

states that the worker will identify the income-leisure combination that carries the highest

utility i.e. intersects the iso-utility line corresponding with the highest utility (red dot).

30

(B) Income-compensated wage decrease: the hourly wage is decreased, and a fixed income is

given so that for an equal number of working hours the worker will earn as much money.

Relatively counter-intuitively, workers tend to decide to work for less hours. The concave shape

of the utility functions explains this finding. Before the income-compensated wage decrease

(blue line), the income-leisure combination carrying the highest utility included more work than

after the decrease (red line, flatter than blue line) since the intersection with the highest iso-

utility line is shifted to the right.

(C) Income-compensated wage increase. The hourly wage is increased and the fixed income is

decreased (green line) compared to the initial payment system (blue line). The point that

maximizes utility after the increase is shifted to the left side, with the worker choosing to work

longer, compared to the baseline, to maximize utility.

(D, E) Labor-leisure theory applied to mental effort. Results of an experiment consisting in

offering the choice between a demanding cognitive task (3-back task with images) and a low

demand reaction task (responding to images of children, signaled by a tone). Subjects were

paid for the 3-back task only, with candy, and the income system was manipulated between

the two sessions of the task.

(D) In session 1 (blue line), the participants received a fixed wage per trial of the high-demand

task. In session 2 (red line), an income compensated wage reduction was imposed (free candy

in addition to small wage per trial). The income-leisure decisions of participants are

represented by the blue and red dots for the high wage and low wage with compensatory

income situations, respectively. In the latter, participants chose to perform less trials and earn

less candy than in the former, although performing the exact same number of trials would

have earned them the same exact amount of candy.

(E) In session 1, participants received a low wage and a high fixed income (blue line). In session

2, the wage was increased and the fixed income was decreased (green line). In the high wage

condition, participants chose to perform more trials of the demanding task compared to the

low wage condition, even though doing the same number of trials in both would have resulted

in equal candy amounts.

In summary, control is allocated rationally by subjects, who perform a cost-benefit

analysis to determine the optimal control strategy (identity) and strength (intensity), weighing

the cost of such allocation and the expected outcomes, should such control be exerted.

31

Shenhav and colleagues have mathematically formalized this tradeoff under the EVC model.

Additionally, generalization of economical labor-leisure theory to mental effort has allowed

demonstrating the concave shape of the cost function of control exertion.

e) What is the substrate of cognitive control? What is

the resource?

Overall, the extensive body of literature detailed above provides strong evidence for

the aversive nature of cognitive control exertion, and demonstrates that subjects make

decisions to allocate control as if sparing a depletable resource. One question however

remains, raised by the very formulation of the theory of bounded rationality. If human

cognitive control exertion is bounded by the availability of certain resources, what is the

nature of such limited resources? Two competing yet not incompatible theories have been

formulated, which I will both review. The first posits that cognitive control uses limited

intrinsic resources (metabolic or computational, synthetic representation of these accounts in

Figure 13). The second, based on the idea that exerting control takes time during which no

other tasks can be executed, formulates that control exertion carries opportunity costs and

that quitting or forgoing control-greedy task could be a manifestation of the exploration-

exploitation trade-off.

(1) Limited intrinsic physiological resource

The idea of control relying on a depletable limited resource was first formulated in the

specific case of tasks requiring sustained attention (Posner and Snyder, 2004, reedition of

1975 book; Shiffrin and Schneider, 1977). This concept was later extended to cognitive control

in general, after several teams found that subjects perform less well at tasks requiring

cognitive control if they previously performed another control-greedy task (e.g. Baumeister et

al., 1998; Garrison et al., 2019; Hagger et al., 2010; Muraven and Baumeister, 2000; Radel et

al., 2019; Schmeichel, 2007). For example, Garrison and colleagues (2019) first asked people

to write an essay about their last holidays. One group had no extra assignment, while the

second could not use the letters A or E. Following that writing assignment, subjects performed

a Stroop task (see Figure 5C for example of a Stroop task design). Garrison and colleagues

32

report that the participants that had previously performed the controlled-writing task had

overall lower performance at the Stroop task. This phenomenon of lower control capacity

after control exertion is called ‘ego-depletion’ (phenomenon reviewed by Hagger et al., 2010),

and has been described by some as a ‘refractory period’ occurring after cognitive control

exertion (Baumeister et al., 1998). This phrasing suggests a structural, physiological constraint

on cognitive control capacity.

Shenhav et al., 2017, Figure 1

Figure 13 : Schematic representation of the different constraints on control exertion that could

account for limited control capacities and control aversiveness (adapted from Shenhav et al.,

2017).

The example situation proposed is the case of a driver, attending multiple information (e.g.

visual, auditory) in an effort to drive safely and reach his destination. Metabolic accounts of

the limitations for control investment state that control consumes and therefore depletes a

metabolic resource critical for neuronal activity, glucose or glycogen. This account is shown in

green. Control-capacity hypotheses of control limitation state that the control that can be

applied is limited (due for example to working memory or information processing capacity

limitations) (blue) and interference-based accounts of cognitive control limitations state that

the system limits the number of ongoing processes to reduce the risk of interferences (orange).

As suggested by the figure, these theories are not mutually exclusive, but may constitute pieces

of a larger scheme.

The hypothesis that the limited resource for mental effort is metabolic was formulated

by Baumeister and colleagues in the late 1990s (Baumeister et al., 1998; Baumeister and

Heatherton, 1996; Muraven et al., 1998). Cerebral activity consumes glucose (Benton et al.,

1994; Fairclough and Houston, 2004; McNay et al., 2001, 2000). Although the brain is

continuously supplied with glucose by the blood flow, glucose can be consumed faster than it

is supplied, thus depleting active brain areas in metabolic substrate (Benton et al., 1996;

Fairclough and Houston, 2004; McNay et al., 2001). The energy management system of the

brain is organized around the neuro-glio-vascular ensemble, capable of adapting rapidly to

fast increases in metabolic demand. This metabolic system is organized around three major

33

metabolites: glucose, glycogen and lactate (Dienel, 2017; Gruetter, 2003; Hertz et al., 2007;

Hui et al., 2017; Müller, 2014; Pellerin and Magistretti, 1994; Walls et al., 2009). Glycogen is

stored quasi exclusively in astrocytes (Magistretti et al., 1981; Phelps, 1972; Watanabe and

Passonneau, 1973), representing, in glucose equivalents, more than all the cytosolic glucose

of neurons (Gruetter, 2003). Glycogen can be synthesized in periods of low brain activity (e.g.

sleep, Swanson, 1992) or high glycaemia (Nelson et al., 1968) to store excess glucose, while in

periods of high activity, it is depolymerized locally to form lactate (another metabolic

substrate for neurons), thus providing a substrate for the increased metabolism (Gailliot,

2008; Swanson et al., 1992). In that sense, astrocytic glycogen acts as a buffer, an energy

reservoir. In conditions of hypoglycemia, astrocytic glycogen can support brain activity for

more than an hour (Gruetter, 2003). Note that besides serving as glucose storage for

emergency situations, glycogen is in fact part of a dynamic system, with a variable proportion

of glucose systematically stored in the form of glycogen before becoming available to the

neurons via glycogenolysis. This mechanism is referred to as ‘glycogen shunt’ (Obel et al.,

2012; Shulman et al., 2001; Walls et al., 2009).

Blood glucose was first proposed as the depletable resource that cognitive control

aversion aims at sparing. Indeed, low blood glucose has well-described detrimental effects on

many mental processes (e.g. language, visual processing), for instance cognitive control.

Conversely, glucose intake has been shown to reduce ego-depletion effects following control

exertion, suggesting that it allowed compensating the putative depletion of resources. A large

part of the research focusing on the relationship between blood glucose and cognitive

performance was conducted in diabetic individuals, in whom dramatically low glycaemia can

occur as a result of excessive insulin administration, but also in healthy subjects after a short

period of fasting (Kennedy and Scholey, 2000; Benton et al., 1987; Donohoe and Benton, 1999;

Fairclough and Houston, 2004; reviewed by Feldman and Barshi, 2007, and Gailliot and

Baumeister, 2007).

For example, Gailliot and colleagues (2007) showed that, following a control-greedy

task, subjects’ glycaemia decreased, and subjects with low glycaemia before the task had

lower overall performance at control-greedy tasks. Moreover, the ego-depletion

phenomenon observed after subjects had been asked to perform control-greedy tasks for a

prolonged time was annihilated by glucose-drink consumption (Figure 14). Note that these

authors gave artificially sweetened drinks (zeros calories) to the group who did not receive

34

sugar, and controlled for the effect of drink liking on performance depletion, in an effort to

eliminate the potential confound of the sweet drink serving as an incentive for control

allocation.

Gailiot et al., 2007, Figure 2

Figure 14 : Ego-depletion after a controlled attention task, in a Stroop task with or without

glucose consumption (adapted from Gailliot et al., 2007).

Subjects were first instructed to watch a video, either without any specific assignment (“Watch

normally”, black bars), or with the instruction to focus only on the female character in the video

and not on words popping at the bottom of the screen (selective attention task, “Control

attention”, grey bars). After completing that task, subjects either received a glucose-

sweetened lemonade or an aspartame-sweetened lemonade and rated the likeability of their

drink. They then performed a Stroop task. Performance in the group receiving no glucose

followed an ego-depletion pattern: performance was lower after the high-demand task. This

ego-depletion effect was not present in the group receiving glucose.

The effects of blood glucose and glucose administration on cognitive performance,

however, have been widely discussed, especially due to repeated failures to replicate findings

from Gaillot and colleagues (2007) regarding decreases of blood glucose following control-

greedy tasks (Dvorak and Simons, 2009; Molden et al., 2012; reviewed by Kurzban, 2010).

Moreover, even if the findings from Gaillot and colleagues were to be replicated, the size of

the decrease of the glycaemia appears to be anecdotal and detractors of the theory argue that

it would be too small to affect cognitive processes (Christie and Schrater, 2015). Findings

regarding the enhancement of cognitive control following glucose administration have, on the

contrary, been replicated in humans (Denson et al., 2010; Owen et al., 2012) and animals

(Miller et al., 2010) but are merely in favor of a beneficial effect of glucose on control

allocation (possibly as an incentive) without validating glucose as the limited resource for

control.

An alternative theory regarding the resources that organisms intend to spare by

avoiding cognitive control exertion concerns astrocytic glycogen (Christie and Schrater, 2015).

35

Christie and Schrater theorized that the brain plans and manages its metabolic resource

consumption and trades-off cognitive effort, and therefore increased performance, for

reductions in metabolic substrate consumption by reductions of neuronal activity. These

authors, however, consider astrocytic glycogen as the metabolic substrate that the system

consumes and aims at maintaining at a sustainable level (Figure 15). In brief, Christie and

Schrater devised a model that accounts for both cognitive control avoidance and ego-

depletion phenomena, in an effort to spare glycogen for later demanding tasks. Noticeably,

LC activation promotes short-term glycogenolysis (depolymerization of glycogen to form

lactate) and longer term glycogen synthesis in target areas, and such effects and their

implications for the relationship between LC activity and effort processing will be discussed

later in the manuscript (see part I.C.1 and III). The metabolic relevance, i.e. the feasibility that

glycogenolysis can respond rapidly and sufficiently to adjust to the increased metabolic

demand in case of augmented brain activity, signaled by certain neurotransmitters (including

noradrenaline from the LC, see part I.C.1) was later tested by Coggan and colleagues (2018).

Their study showed, using a custom-made computational model, that Christie and Schrater’s

theory would be compatible with what is currently known of the enzymatic cascades involved

in astrocytic and neuronal metabolism.

(A) Christie and Schrater, 2015, Figure 1A (B) Christie and Schrater, 2015, Figure 1B

Figure 15 : Glycogen as a buffer for brain metabolic resources (adapted from Christie and

Schrater, 2015)

(A) Schematic representation of the reactions transforming blood glucose from capillaries into

glycogen, stored in astrocytes, later depolymerized to form lactate (another metabolic

substrate for neurons), and released to neurons in case of high metabolic demand (G-6-

P=glucose-6-phosphate). The mechanism of systematic capture and polymerization of glucose

into glycogen by astrocytes is referred to as the glycogen shunt.

(B) Energy flow in the model from Christie and Schrater (GLUT1 and 3: glucose transporters)

Note also that Holroyd (2016) has evoked the possibility that the aversiveness of

cognitive control could be associated to accumulation of waste (specifically amyloid-β protein

36

in the interstitial fluid) due to a saturation of the waste clearing system, rather than

consumption of resources. This possibility is complementary to the resource account of

mental effort aversiveness and will not be detailed here.

To summarize, several teams have proposed metabolic accounts for the aversiveness

of cognitive control exertion. Specifically, avoiding control would allow sparing glucose or

glycogen, or could be avoiding accumulation of metabolic waste. The hypothesis that

astrocytic glycogen could be the limited resource for cognitive control investment is of

particular interest for this manuscript considering the relationship between LC activity and

glycogen metabolism (see part I.C.1).

(2) Limited computational resource

Some authors have suggested that the limited resource for cognitive control could be

computational. Working memory capacity has been proposed as limiting (Anderson, 1983;

Cohen et al., 1990), as tasks requiring cognitive control very often involve working memory (if

only to recall task rules and information), and working memory capacity is known to be limited

(Cowan, 2005; Luck and Vogel, 1997; Miller, 1956). Only indirect proof can however be

provided to support that hypothesis. Moreover, limitations in working memory capacity can

hardly account for long-term depletion in control capacity after control exertion, and this

frame mostly accounts for the difficulty to conduct several tasks requiring working memory

simultaneously, leading this interpretation to be criticized (Shenhav et al., 2017).

Generalizing this theoretical frame, the multiple resource hypothesis states that

limitations in control exertion are the consequence of bottlenecks in processing (Allport, 1954;

Logan, 1985). For instance, one is not capable of resolving two mathematical equations at the

very same time, either due to intrinsic incapacity of the system to conduct two operations at

the same time, or due to an effort of the system to avoid any risk of crosstalk between

concurrent tasks. Just like working memory limitation, bottlenecks in processing mainly justify

that two control-demanding tasks usually cannot be conducted at the same time.

Nonetheless, this incapacity to efficiently conduct several demanding tasks at the same time

due to computational limitations would lead allocating control to one specific task at a given

time to carry opportunity costs (see below, part I.A.2.e)(3)), which could account for the

aversiveness of control.

37

To summarize, limitations in control exertion have been hypothesized to be caused by

computational limitations. Although this point of view can account for the difficulty or even

incapacity of multitasking, it can hardly account for ego-depletion phenomena. This does not

make this theory invalid, but merely insufficient, and the existence of bottlenecks in

processing could be part of the reasons for limited control exertion.

(3) Motivational resource: exploration-exploitation trade-

off and opportunity costs of control

In contradiction, or at least as a complement to the theories presenting control as a

process consuming limited depletable physiological or computational resources, several

authors have conversely proposed that depletion effects could be accounted for by

motivational mechanisms. Still, the motivational and metabolic accounts of cognitive control

aversion and depletion may not be incompatible but merely “components of a more general

framework” (Christie and Schrater, 2015; Kurzban et al., 2013).

The motivational hypothesis emerges from the findings that offering rewards (Boksem

et al., 2006; Inzlicht et al., 2014; Muraven and Slessareva, 2003) as well as other manipulations

(e.g. self-conviction of unlimited willpower, Job et al., 2010) can reduce and even eliminate

depletion effects. Moreover, theories of control allocation (expected value of control theory,

see part I.A.2.d)(3) state that subjects make a trade-off between cognitive control and

accuracy, defined as the maximization of the expected value (expected reward minus

expected effort). In that frame, decisions to explore, e.g. pick a new unknown option, would

rarely be categorized as accurate. Indeed, since the expected utility (expected reward and

required effort) of the new option (exploration) is usually highly uncertain, it is down-weighed

by its risk (Dow and Werlang, 1992), compared to the well-known option (exploitation). Since

permanent exploitation is maladaptive (Stephens and Krebs, 1986), some teams have

suggested that the necessity to explore could be why control exertion is costly and limited.

Specifically, Inzlicht and colleagues (Inzlicht et al., 2014; Inzlicht and Schmeichel, 2012)

theorized that the ‘refractory period’ following control-greedy tasks could be related to the

‘evolutionary pressure motivating organisms to balance their desires for exploitation versus

exploration’, causing a switch of motivational priorities to make the most of a given

38

environment. Pursuing control-greedy strategies, just like any exploration strategy, is

associated to opportunity costs. While focusing on a cognitive task, subjects cannot pursue

other courses of action that might pertain better rewards, i.e. they cannot explore. Such

opportunity costs are especially true for decision-making considering that decision strategies

involving high control allocation to ensure high accuracy tend to be associated with increased

time spent deciding, less decisions per unit of time, and therefore less potential rewards per

unit of time. Humans and animals have indeed long been known to behave according to a

speed-accuracy trade-off, i.e. they routinely do not spend as much time making a decision as

could be required to be perfectly accurate (Chittka et al., 2003; Franks et al., 2003; Heitz and

Schall, 2012; Wickelgren, 1977; reviewed by Heitz, 2014).

To summarize, motivational accounts of the limitations to control exertion state that

allocating high cognitive control is associated to opportunity costs, and these opportunity

costs are believed to constitute a decision variable for the determination of the amount of

control worth investing in a task (Kurzban et al., 2013). Dynamically balancing exploitation and

exploration, i.e. behavioral flexibility, implies regulating the extent to which one engages in

tasks, i.e. exerts cognitive control, as opposed to disengaging from the task to sample other

opportunities. Note that several recent studies on behavioral flexibility have pointed towards

an implication of the LC and of noradrenaline (NA). This literature will be reviewed further

(see part I.B.4).

f) Behavioral proxies for mental effort

In order to be able to access the amount of cognitive control exerted by subjects in

laboratory tasks, identifying reliable proxies for it is crucial. Indeed, a possibility would be

repeatedly asking subjects to rate their mental effort levels, but it would be time consuming,

and it is obviously not feasible in the case of animal research. In that part, I therefore examine

two behavioral parameters, RT and performance, that can be used to approximate how much

effort was invested by subjects.

(1) Reaction time

Everyday life cognitive operations, calculations for example, provide trivial examples

of the correlation between cognitive task difficulty, and duration of the task. For instance,

computing a one-digit multiplication in one’s head will take considerably less time than

39

computing a four-digit multiplication. Formally, this suggests that RT, i.e. the time elapsing

from the onset of a cognitively demanding operation until its end, increases with cognitive

demand. Under the assumption that control exertion is relatively constant over time, RT,

representing the duration of the cognitive effort, can serve as a satisfying proxy for the total

cognitive control allocated. This constitutes a simplification, as Shenhav and colleagues (2013)

did theorize that the intensity of cognitive control can vary across situations (tasks and trials).

In the 1970s, Wright, Bettman and Zins (Bettman and Zins, 1979; Wright, 1975)

provided some indirect account that more effortful strategies to solve problems tend to be

associated with longer RTs. They asked subjects to solve problems using different sets of rules,

differing in the level of mental effort required. The rule that subjects reported to be the least

effortful to apply corresponded to shorter RTs. However, no mechanistic account of this

relationship was formulated at the time.

Bettman, Payne and Johnson (1990) later proposed a theoretical framework to explain

such findings, summarizing and generalizing the theoretical work of several other teams

(Huber, 1980; Johnson and Payne, 1985; Newell and Simon, 1972; Payne et al., 1988; Shugan,

1980). They described decision-making as a succession of elementary information processing

events, and different decision-making strategies for the same decision could include more or

less of these elementary processes. For example, when asking subjects to choose between

two alternative courses of action, each action associated to a certain required effort and a

certain reward, a potential strategy would be choosing based on the rewards only, as opposed

to a strategy involving considering and weighing both dimensions. The former strategy will

trivially require less computational steps compared to the latter: to compare only rewards,

the subjects only have to compute a subtraction (one step), while to compare options based

in both attributes, they would for example have to compute the value of each option then

subtract these values (three steps). Intuitively, the former strategy, since it includes less steps,

will correspond to less cognitive effort compared to the latter. Experimentally, several studies

demonstrated a positive link between the putative number of elementary information

processing steps in a task and RT (Bettman et al., 1990; Carpenter and Just, 1975), as well as

a negative effect of the number of such steps on subjects’ choices of strategies to solve a task

(Bettman et al., 1990; Payne et al., 1988).

In the specific case of value-based decisions, RT is hypothesized to be decomposed into

at least three independent components: time necessary for stimulus processing, for sufficient

40

evidence accumulation to reach a decision, for triggering chosen action (Dmochowski and

Norcia, 2015). Over the past two decades, the drift-diffusion model (DDM), a sequential

sampling model, has become the preponderant model used to fit behavior in value-based (as

well as perceptual) decision-making (e.g. Bogacz et al., 2006; Ratcliff and McKoon, 2008). This

model assumes that decisions consist in the accumulation of evidence favoring one alternative

over the other. In the DDM, the three stages of the RT are sequential and therefore their

added durations form RT.

The DDM predicts that the greater the difference in value for the two options, the

shorter the RT, since the accumulation of evidence reaches a decision bound faster. Behavioral

studies have confirmed this finding (Hunt et al., 2012; Krajbich et al., 2010; Krajbich and

Rangel, 2011; Milosavljevic et al., 2010; Morris et al., 2019; Philiastides and Ratcliff, 2013).

Note that accuracy (proportion of trials in which subjects chose the best option, based on their

previous subjective ratings) increased with value difference, thus indicating that subjects

found choices with high value difference easier (Milosavljevic et al., 2010). This also confirms

that extensive information processing is perceived as mental effort (Anderson, 1990; Bettman

et al., 1990; Simon, 1956; Smith and Walker, 1993). Subjects therefore probably invested more

cognitive control to make the choice in case of low value difference, and this excess in control

exertion resulted in increased time spent to decide (increased RT).

Besides this effect of value difference on RT, a second interesting effect was observed

when examining RT variations when the overall value of the trial increased, i.e. when the sum

of the subjective values of the options presented increased. Hunt and colleagues (2012) report

that subjects were faster (negative effect on RT) when the overall value of the trial increased,

over and above the effect of value difference (Figure 16B). In coherence with that finding,

McGuire and Botvinick (2010) showed a positive relationship between mean RT throughout

blocks of trials and avoidance ratings of the whole block (Figure 16E). Cognitive control is

required to override default courses of action (Epstein, 1994; Sloman, 1996). In the case of

trials with low overall value or high avoidance ratings, the default response would have been

refusing the trial, which is not an option in that task. Consequently, subjects probably

mobilized more cognitive control to perform the trial in low overall value conditions.

Indirectly, the negative relationship between overall value and RT thus suggests that RT could

serve as a proxy for cognitive control in that case.

41

One limitation of the use of RT as a proxy for mental effort is the risk of circular

reasoning (Navon, 1984; Shenhav et al., 2017). One example for this circularity can be found

in the Stroop task, described above (Figure 5C). RTs are known to be in average longer in the

incongruent conditions, compared to the congruent conditions, and this difference is referred

to as the ‘Stroop effect’. In the theoretical frame that RTs correlate positively with the mental

effort invested in tasks, one would conclude that RTs generally increase in the incongruent

conditions because the subjects invest more effort in identifying the correct answer, as

distractor presence increases the difficulty. Another possibility could however be that, when

difficulty increases, subjects do not increase the invested effort sufficiently, and therefore take

longer to respond. In that case, RT would correlate with the subjective difficulty of the trial

rather than the effort invested.

(A) Hunt et al., 2012, Figure 2 (B) Hunt et al., 2012, Figure 3A (C) Hunt et al., 2012, Figure 3B

(D) Kool and Botvinick, 2018, Figure 1A (E) McGuire and Botvinick, 2010, Figure S1

Figure 16: Influence of value difference (VD) and trial overall value (OV) or avoidance ratings

and on RTs in a value-based decision-making task

(A) Task design. Subjects were instructed to choose between two options represented by

rectangles, with different reward sizes (rectangle size) and probabilities (percentage written

below). After choosing, they received a feedback on the rewards of the options (red for non-

rewarded, green for rewarded) (adapted from Hunt et al., 2012).

(B) RTs as a function of value difference (VD, black) and overall value (OV, red). RTs decreased

with both increasing VD and OV (adapted from Hunt et al., 2012).

(C) Size of the negative linear regression coefficients for value difference (VD) and overall value

(OV) on RTs. Both had significant negative effects on RT (adapted from Hunt et al., 2012).

(D) Task design from McGuire and Botvinick, 2010. Subjects were repetitively asked to choose

between pairs of previously unknown cues. One cue was associated to a low demand task, the

other to a high demand task (see Figure 7 for extra details, adapted from Kool and Botvinick,

2018).

(E) RTs as a function of trial number within block. Participants were told that trial blocks were

drawn from one of four ‘sources’. After each run, subjects rated their willingness to avoid

future blocks from the same ‘source’, thus providing post-hoc subjective mental effort ratings

42

for each cue. RTs are plotted by avoidance rating: level 1 in green, 2 in grey and 3 in red. Mean

RT is higher in high avoidance blocks (adapted from McGuire and Botvinick, 2010).

(2) Performance

Another potential proxy for the cognitive control involved in a task could be

performance, although it carries strong limitations. A first caveat lies in the definition of

performance. For instance, in a Stroop task, performance can be defined as the success rate

in giving the correct answer, but also as the inverse of the difference in RT between congruent

and incongruent conditions, as exerting cognitive control should reduce this difference.

Additionally, some authors argue that, under the condition that the subjects’ intrinsic

capacities are not exceeded and that they exert adequate cognitive control relative to the task

difficulty, increased effort should always allow maintaining performance when task difficulty

increases (Norman and Bobrow, 1975). In that frame, performance would not vary, thus

making it an inadequate proxy for the effort exerted (Backs and Seljos, 1994). Such a situation

is however merely theoretical, as studies using standard mental effort experimental

paradigms usually report differences in performance across difficulty levels, even when

participants do report exerting increased mental effort in the higher difficulty levels. This is in

line with Shenhav’s EVC theory (2017, 2013) that states that subjects do not systematically

engage the level of effort that would be required to absolutely maximize reward rate, but

compute a tradeoff between the cost of the effort engaged and the reward obtainable

through that effort, such that the EVC is maximized (see part I.A.2.d)(3)). In this frame,

performance is not a satisfying proxy for the cognitive effort. Again, using performance as a

proxy for mental effort is at risk of circular reasoning.

To summarize, behavioral proxies for cognitive control exertion have been proposed.

Performance appears quite imperfect, as it results from the interaction between subjective

difficulty and control exertion. Additionally, it can allow evaluating difficulty for groups of

trials, but not trial-by-trial. RT, conversely, can be measured trial-by-trial. It probably

correlates with the subjective difficulty of trials, but interpretations in terms of exerted

cognitive control must be taken with caution as they risk circularity. A way to solve the

circularity encountered with using RT and performance as proxies would be to introduce

mental effort exertion self-reports throughout the task. Another possibility would be assessing

43

mental effort through an independent measure, for instance a physiological measure. I review

the various measures proposed in the literature in the following part.

g) Physiological proxies for mental effort:

cardiovascular, metabolic and vegetative measures

Several teams over the past 60 years have investigated peripheral indicators of mental

effort: metabolic, cardiovascular and vegetative markers. I will review briefly these proxies.

(1) Cardio-vascular and metabolic proxies

Following early reports of the relation between heart rate and psychological tasks

(Lacey, 1967), several studies on mental load have reported increases in tonic heart rate.

These studies originally reported increases beyond what would have been expected from the

current physical (somatic) activity, and specifically beyond the concurrently measured

peripheral oxygen consumption (outside of the tasks, Carroll et al., 1986a; Turner and Carroll,

1985). These increases in heart rate were initially referred to as ‘metabolically exaggerated

heart rate’ or ‘additional heart rate’. In the same period, increases in energy expenditure

(oxygen consumption and carbon dioxide production) in response to mental effort (Carroll et

al., 1986a; Sims et al., 1988; Sims and Carroll, 1990; Turner et al., 1983; Turner and Carroll,

1985) were reported.

These findings were initially interpreted in terms of energy expenditure due to the

somatic demands associated to the mental effort task (e.g. moving a video-game joystick, Sims

et al., 1988; Turner et al., 1983; Turner and Carroll, 1985). Backs and Ryan (1992) however

ruled out such interpretation by having subjects perform a continuous working memory task

with two difficulty levels but equal motor demands. Participants were presented with a

succession of visual targets, and had to count the number of presentations of either one or

three targets. The study reports a small consistent increase in heart rate and oxygen

consumption, and faster and shallower breathing with increasing task difficulty. Backs and

Seljos (1994) completed this result by demonstrating that these metabolic and heart rate

increases were not dependent on differences in baseline metabolism and heart rate

(specifically across male and female subjects). Conversely, individuals with lower working

memory performance, and therefore perceiving the task as subjectively more difficult (as

44

shown by Ericsson and Simon, 1980), displayed stronger cardiovascular and metabolic

adaptations with task difficulty.

Furthermore, several have tried to refine the relationship between heart rate and

mental effort. Specifically, they decomposed the variations of heart rate by frequency (power

spectral analysis), and found a relationship between the 0.1Hz component of its variability and

several correlates of mental effort: working memory load (Aasman et al., 1987; Mulder and

Mulder, 1981), complex decision-making (Tattersall and Hockey, 1995), and variations in task

difficulty (Nickel and Nachreiner, 2003; Wilson, 1992; Jorna, 1992). Note that this frequency

component is supposed to be related to parasympathetic activity (Mulder, 1986, 1979).

Noticeably, differences in cardiovascular and metabolic activity across difficulty levels

were only reported in tasks requiring sustained mental effort (e.g. continuous working

memory task). They were not found in tasks requiring discrete computations separated by

relatively long time inter-trial-intervals (e.g. arithmetical task, Carroll et al., 1986a, 1986b),

although these tasks elicited overall increased heart rate and oxygen consumption. This

difference may merely reflect slow metabolic and cardiovascular adaptations or may be due

to an interaction between difficulty and time on task.

Overall, these early reports indicate that mental effort is associated to increased

metabolic and cardiovascular demand. These measurements are however imperfect, as

mental load, performance, physiological correlates of mental effort and effort ratings are only

correlated, and each proxy for mental load does not vary with the same components of mental

effort and with identical sensitivity (Sanders, 1979). Moreover, those findings do not allow

confirming the causality link between such cardiovascular and metabolic adaptations and

mental effort, since heart rate could increase in order to provide sufficient oxygen to the brain

regions involved in the current effortful task (direct pathway), or because effort requires

increased sympathetic tonus, which coincidentally increases heart rate and metabolic activity.

I will review the literature on the relationship between one component of the sympathetic

system, brain NA, and mental effort further (see part I.B.4).

(2) Pupil dilation

Another frequently used proxy for mental load is the task-evoked pupillary response,

i.e. the pupil diameter change in response to task stimuli or during task-related computations,

45

relative to a pre-stimulus baseline period (reviewed in Beatty, 1982; van der Wel and van

Steenbergen, 2018). This relationship is of particular interest for this manuscript considering

the well described positive correlation between LC activity and pupil diameter (Joshi et al.,

2016; Murphy et al., 2014; Varazzani et al., 2015), which will be reviewed later (see part

I.B.2.b).

The positive correlation between pupil diameter and mental activity is easily

observable in everyday life, as explained by Kahneman (1973):

“Face a mirror, look at your eyes and invent a mathematical problem, such as 81 times

17. Try to solve the problem and watch your pupil at the same time, a rather difficult exercise

in divided attention. After a few attempts, almost everyone is able to observe the pupillary

dilation that accompanies mental effort.”

First descriptions more than fifty years ago of pupil diameter correlating with mental

effort focused on tasks involving memory load (Kahneman and Beatty, 1966), arithmetical

problem solving (Hess and Polt, 1964) and decision-making (Simpson and Hale, 1969). In

recent years, pupillometry has become widely used in cognitive neuroscience studies, due to

its simplicity of use and to the low price of the equipment, and many teams have conducted

studies about the relation between event-related pupil dilation and mental effort. Most

studies, though, report a positive correlation between task difficulty and pupil dilation, but

cannot effectively differentiate between the difficulty of the task and the effort invested. For

example, several have used keep-track tasks, consisting in asking subjects to track a subset of

pre-assigned dots among a group of randomly moving dots (Figure 17A). This task is notably

more difficult when the number of dots that have to be tracked increases, and pupil dilation

during tracking has been reported to increase for increase numbers of dots to track, as shown

in Figure 17B (Alnæs et al., 2014; Wahn et al., 2016; Wright et al., 2013).

(A) Alnaes et al., 2014; Figure 1 (B) Alnaes et al., 2014, Figure 3

Figure 17 : The multiple-object tracking task (adapted from Alnæs et al., 2014).

(A) Task design: At the beginning of trials, subjects were presented with a certain number of

blue dots. Then a subset of the dots turned red (three here) for a short amount of time before

46

turning blue again. These dots would have to be tracked by the subjects throughout the trial.

During the rest of the trial, all the dots moved randomly (tracking period). At the end of the

tracking period, subjects had to report the position of the tracked dots by clicking on them.

Task difficulty increased with the number of dots tracked.

(B) Pupil time series in the multiple-object tracking task. Trials are separated by difficulty, i.e.

by number of objects to track. The x-axis represents the time from tracking onset (dots start

moving) and the y-axis represents pupil size, in mm. Pupil diameter was larger during tracking

period when the number of tracked objects was higher, i.e. task difficulty increased.

Note that several studies have additionally reported a positive relationship between

global task performance and baseline (outside of task) pupil diameter (Heitz et al., 2007; Meer

et al., 2010; Tsukahara et al., 2016). These authors argue that global task performance can be

a proxy for the intelligence of individuals in the domain tested (e.g. fluid intelligence, the

capacity to reason, independently of learned skills).

(3) EEG proxies for cognitive control

Electroencephalograms (EEG) are a non-invasive method of exploring brain activity

through electrodes placed in strategic positions on the skin of the head. Event-related

potentials (ERPs), stereotyped responses to stimuli recorded in EEG, can be decomposed in

several sub-components (Nidal and Malik, 2014), and the amplitude of two of them has been

shown to correlate with the cognitive control exerted: N2 and P3 (Figure 18). The N2

component has been shown to be related to conflict monitoring. Indeed, in Go/No-Go tasks,

the N2 is enhanced during No-Go trials compared to Go trials (Clayson and Larson, 2013;

Falkenstein et al., 1999; Folstein and Van Petten, 2008; Garavan et al., 1999; Tillman and

Wiens, 2011) and larger in subjects making fewer false alarm errors (Falkenstein et al., 1999).

P3 amplitude has also been linked to inhibitory control (Garavan et al., 1999). Additionally,

several studies established a link between N2 and P3 and working memory load (e.g. Pratt et

al., 2011; Qi et al., 2014; Wei and Zhou, 2020), and with task-switching costs (e.g. Gajewski

and Falkenstein, 2011; Jamadar et al., 2015; Kopp et al., 2020). Overall, these results indicate

that certain components of ERPs could serve as proxies for the perceived cost of the mental

effort or the allocated cognitive control.

47

To summarize, cognitive control exertion has been linked to increased heart rate,

oxygen consumption, pupil dilation (baseline and in response to task events) and amplitude

in two components of EEG ERPs, N2 and P3. Pupil diameter and EEG ERPs are of particular

interest because, first, they can be measured on a trial-by trial basis, and because pupil

diameter and P3 amplitude have been linked to LC activity (see, respectively, parts I.B.2.b) and

I.B.2.a) for details).

Figure from: Syntax in a blink: Early and automatic processing of syntactic rules as

revealed by event-related brain potentials. Dissertation, 172 S., Max Planck Institute for

Human Cognitive and Brain Sciences, Leipzig (2008).

Figure 18: ERP recorded in EEG. ERPs can be decomposed into several subcomponents: N1

(positive), P2 (negative), N2 (positive) and P3 (negative) 3

In brief, mental effort exertion, in the recent years, has been unified under the concept

of cognitive control. Cognitive control is required in tasks involving updating information in

working memory, switching between tasks and rules, and inhibiting automatic or default

responses. The brain system organizing cognitive control allocation has been theorized to

include three sub-units: a regulation unit implementing the control signal, defined by its

identity and intensity, a specification unit deciding on the control signal, and a monitoring unit

receiving feedback on the process of control exertion and consequently adapting the decision

parameters of the specification unit. Cognitive control is involved in decision-making,

especially (if not exclusively) if extensive processing of options is required. It also serves as a

decision variable, since exerting control is perceived as aversive and tends to be avoided.

Models of decisions regarding control exertion have been formalized. The aversiveness of

mental effort is believed to reflect a mechanism of resource sparing, although the exact nature

of such resources has been debated. They could be metabolic (blood glucose or astrocytic

3 Figure from: Syntax in a blink: Early and automatic processing of syntactic rules as revealed by event-related

brain potentials. Dissertation, 172 S., Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig

(2008).

48

glycogen), computational (bottlenecks in processing) or motivational (opportunity costs and

exploration-exploitation tradeoff). Measuring mental effort is difficult, but some indirect

proxies have been proposed: behavioral proxies, RT and performance, and physiological

proxies, heart rate, oxygen consumption, pupil dilation, and magnitude of EEG ERPs N2 and

P3.

3. Mental and physical effort, evidence for common resources

and partially common processing systems

“Mental effort is like physical effort—people dislike both.” Camerer and Hogarth, 1999

In the previous parts, I reviewed separately the characteristics of physical and mental

effort. Strikingly, both categories of effort seem to share common characteristics. Although

subjective physical effort perception could, at first glance, be considered to relate solely to

muscular activity, it was shown to be independent from sensory feedback and dependent on

a duplication of the motor command in the SMA (see part I.A.1.b). Additionally, physical effort

perception increases after curare administration (Gallagher et al., 2001), i.e. when extra

neuronal stimulation from the motor cortex to the muscles is required, suggesting that it is

the intensification of the neuronal activity required for the muscular contraction that is

perceived as effortful, not the muscular contraction itself.

Furthermore, mental fatigue has been repeatedly shown to impact physical effort

performance, thus generalizing the concept of ego-depletion (e.g. Boksem and Tops, 2008;

Martin et al., 2018; McMorris, 2020; Schiphof-Godart et al., 2018; Van Cutsem et al., 2017;

reviewed by Habay et al., 2021). For example, you may find it difficult to motivate yourself to

go for a run tonight after making the mental effort of reading this manuscript. At the

motivational level, a shift in the perception of physical effort following mental effort has been

suggested (McMorris, 2020). Along the same lines, excessive training, both in intensity and

frequency, can lead athletes to experience overtraining syndromes (Cadegiani and Kater,

2019; Meeusen et al., 2013; Smit, 1978). The major symptoms of overtraining syndrome

include (but are not restricted to) depression and low motivation (Meeusen et al., 2013),

indirectly suggesting that excessive physical effort exertion could lead to depleting a putative

critical resource for cognitive control.

49

At the cognitive level, one potential overarching cognitive explanation for such

relationship between cognitive control and physical effort exertion stems from the strong

possibility that physical effort requires exerting cognitive control, at least to decide to engage

in physical effort (or to decide to maintain it in the case of endurance training). Cognitive

control is required to override default responses (Epstein, 1994; Sloman, 1996). When facing

a decision whether to engage in a course of action requiring physical effort (or whether to

continue the ongoing physical effort in the case of endurance tasks), the default response

would be forgoing the effort, considering the aversive nature of physical effort (see part

I.A.1.a). For example, going for a run requires cognitive control to overcome the desire to stay

at home and rest. In that frame, mental and physical effort tasks may share (at least partly)

common resources, processing mechanism and neuronal substrates.

The physiological mechanism behind this impact of mental fatigue on physical effort

capacity and vice versa is, however, still debated. Some have proposed an accumulation of

adenosine (metabolic waste) in the brain, and in particular the anterior cingulate cortex (ACC)

(Pageaux et al., 2014). This idea is debated since increases in brain adenosine after physical

effort are short-lasted, while cognitive effects would require long-lasting increases in

adenosine (McMorris, 2020). Another possible mechanistic account, although not formally

demonstrated, comes from the combination of the hypothesis from Christie and Schrater

(2015, see part I.A.2.e)(1) that the depletable resource for cognitive control could be astrocytic

glycogen, combined with the result that physical effort depletes astrocytic glycogen (Matsui

et al., 2017). Together, these points would suggest that cognitive and physical effort could be

drawing on similar resources in the brain. This manuscript specifically aims at exploring the

specific role of the LC/NA system in effort processing, hypothesizing that it participates in the

processing of both (see part I.B.4).

50

B. Role of the locus coeruleus in effort processing:

review of the literature

The first part of this manuscript summarized the literature regarding physical and

mental effort at the conceptual level, their impact on decisions, and proposed proxies for

evaluating them. Additionally, clues for a potential common processing system for both

mental and physical effort were proposed. The central part of the work of this thesis concerns

the neurophysiology of the LC, and its involvement in effort processing in general. This part

will consist, first, of a brief review of the literature concerning the anatomy, histology and

physiology of the LC, and frequently used proxies for LC activity, followed by a part focusing

on accounts converging towards an implication of the LC in effort processing. Specifically, I will

review evidence regarding the implication in three processes involving resource mobilization,

arousal, stress and attention. Then, I will review general evidence for a role of the LC/NA

system in the execution of costly goal-directed actions, before focusing on the particular case

of cognitive control exertion for decision-making and its consequences. In a last subpart, I will

examine potential mechanisms for the effects of LC activity on cognitive processes, and

especially on cognitive control exertion.

1. Anatomy, histology and physiology of the locus coeruleus

a) Anatomy and histology

The existence of the LC has been known since the XVIIIth century, although it was

Dahlstroem and Fuxe (1964) who first identified that LC neurons produce monoamines, and

specifically NA. The LC is a small brainstem nucleus, constituted of quite large neurons

containing neuromelanin (Nieuwenhuys et al., 2008; Olszewski and Baxter, 1954), about

45000 to 50000 in humans, 10000 in rhesus macaques (Macaca mulatta, Sharma et al., 2010)

and 1500 to 2000 in rats and mice (Goldman and Coleman, 1981). It is located lateral to the

fourth ventricle, just under the anterior part of the cerebellum and the inferior colliculi (Aston-

Jones and Bloom, 1981; Bouret and Richmond, 2009; Grant et al., 1988; Kalwani et al., 2014,

Figure 19). Note that these landmarks are routinely used in magnetic resonance imaging (MRI)

51

and during electrophysiological recordings to locate the LC, since it cannot be visualized during

a routine MRI (MRI methods to image the LC are reviewed in Liu et al., 2017).

(A) Wikibrainstem, fibratlas.univ-tours.fr (B) Wikibrainstem, fibratlas.univ-tours.fr

(C) Grant et al., 1988, Figure 1

(D) Kalwani et al., 2014, Figure 5A (E) Kalwani et al., 2014, Figure 5B

(F) Kalwani et al., 2014, Figure 5F

Figure 19: Anatomy and histology of the LC in humans and monkeys

(A,B) Human MRI with T2 contrast indicating the position of the LC in red and the position of

nearby structures. (B) Nearby structure limits are highlighted with specific colors (anatomic T2

MRI, adapted)

(C) Histology of the LC in monkeys (Macaca arctoides). The position of the LC was marked after

recordings by iron deposits, made by passing anodal current when the electrode was located

immediately dorsal to the LC. These deposits (arrows), later marked with Prussian Blue coloring

are therefore just dorsal to the LC (adapted from Grant et al., 1988).

(D,E) Histology from the right hemisphere in a monkey (NB: the arrow shows a lesion from

electrode passage) (Adapted from Kalwani et al., 2014).

(F) Tyrosine hydroxylase immunohistochemistry allowing identifying NA producing neurons of

the LC in dark brown (Adapted from Kalwani et al., 2014).

Abbreviations:

4V and ven: Fourth ventricle; bc: brachium conjonctivium; 4x: trochlear decussation; Cb/Cb2,

cerebellum; CGPn: Central gray of the rhombencephalon; ctg: central tegmental tract; LC:

locus coeruleus; LV, lateral ventricle; mcp, middle cerebellar peduncle; Me5: mesencephalic

trigeminal nucleus; mlf, medial longitudinal fasciculus; MPB: medial parabrachial nucleus;

Mo5N: rostral tip of the motor trigeminal nucleus; Pn and PnO: pontine nuclei; scp : superior

cerebellar peduncle; subCD/subCV: dorsal/ventral parts of the subcoeruleus; vsc: ventral

spinocerebellar tract.

52

b) Electrophysiological properties

In vivo, as far as electrophysiology is concerned, LC neurons are characterized by low

rates of spontaneous activity, below 4Hz at rest and up to 5 to 6Hz during stress exposure,

broad waveforms (>0.6ms for the initial peak), and a stereotyped activation-pause responses

to brief unexpected stimuli (auditory or tactile stimuli, e.g. knock on door, see Figure 20). The

activation usually consists of a few spikes (sometimes only one), and neurons habituate fast

in case of repetition of the stimulus (Aston-Jones and Bloom, 1981; Bouret and Richmond,

2009; Bouret and Sara, 2004; Grant et al., 1988; Hervé-Minvielle and Sara, 1995).

Noradrenergic fibers emerging from the LC are not myelinated and therefore exhibit a slow

conduction velocity, ranging from 0.5 to 2.4m/s, with a mean of 0.9m/s (Aston-Jones et al.,

1985). As a consequence, the discharge of NA in target areas happens about 150ms from LC

neurons’ activation (30 to 135ms for the somatosensory cortex, 50 to 110ms for the frontal

cortex, 80 to 135ms for the occipital cortex, Aston-Jones et al., 1980; Berridge and

Waterhouse, 2003; Foote et al., 1983).

(A) Grant et al., 1988, Figure 7a (B) Grant et al., 1988, Figure 8a

(C) Kalwani et al., 2014, Figure 3H

Figure 20: LC neurons respond to surprising stimuli with a characteristic activation-inhibition

response

(A) Multi-unitary recording in the LC, peri-event time histogram. A surprising sound (black

arrow) produced a robust increase in rate (adapted from Grant et al., 1988).

(B) Single unit recording in the LC, around surprising sound (door knock). Top: event timing.

Bottom: spikes recorded from an LC unit, each vertical bar indicates a spike. The unit responded

with several spikes to the door knock (adapted from Grant et al., 1988).

(C) Multi-unitary recording in the LC, around unexpected auditory stimulus. Top: raster plot;

Bottom: peri-event time histogram. LC neurons responded with a burst consisting of a few

spikes around the onset of the beep, followed by an inhibition period (adapted from Kalwani

et al., 2014).

53

c) Connectivity to other brain structures

Recent studies on the anatomo-functional properties of the LC have benefited from

novel techniques allowing precise dissections of the internal anatomy and structure of the

connections of the LC with the rest of the brain. The most important recent finding regarding

the anatomy of the LC concerns its modular organization.

LC neurons receive diverse inputs from the rest of the brain (Aston-Jones et al., 1991;

Luppi et al., 1995; Schwarz et al., 2015). Viral tracing studies revealed that the LC receives

direct inputs from more than a hundred (111) regions of the brain, with individual LC cells

generally receiving inputs from nine to fifteen regions (Schwarz et al., 2015). In addition to

these various inputs, LC neurons may have diverse sensitivities to neurotransmitters as they

display a very variable pattern of expression of receptors. First, LC neurons are interconnected

and therefore sensitive to NA. However, adrenergic receptors are not evenly distributed

throughout the LC (Chamba et al., 1991). Overall, α2 receptors are predominant in quantity in

the whole LC, in addition to a minority of α1 receptors (Young and Kuhar, 1980), and these

two receptors are believed to have opposite effects on the firing rate of LC neurons (Szabadi,

2013). Importantly, α2 receptors were found to be more abundant in the posterior part of the

LC, while α1 receptors appeared to be mostly localized in the anterior part of the LC (Chamba

et al., 1991). Regarding sensitivity for other neurotransmitters, coming from axons projecting

from exterior brain areas, acetylcholine receptors, and in particular nicotinic receptors, are

not evenly distributed throughout the LC either, and such heterogeneity of distribution was

shown to be associated with heterogeneous excitatory response strength to acetylcholine

(Léna et al., 1999).

LC neurons are the sole source of NA, a catecholamine, for the whole forebrain and

the LC as a whole is virtually sending axons to all brain areas (Levitt and Moore, 1978; Morrison

et al., 1978). Note, though, that NA is not the only neurotransmitter produced in the LC, as

neuropeptide Y, galanin (Simpson et al., 1999; Takeuchi et al., 2016; Xu et al., 1998) and

several other neurotransmitters have been found. Co-release of these molecules with NA

could serve to modulate its effects in target areas (Tsuda et al., 1989). Not all LC neurons

produce all these neuropeptides, and small subpopulations of LC neurons producing certain

neurotransmitters can be identified (Sutin and Jacobowitz, 1991). Additionally, DA, the

precursor of NA, is synthesized by tyrosine hydroxylase not only in dopaminergic but also

noradrenergic neurons. Co-release with NA by LC terminals in the hippocampus and the

54

cerebral cortex and in particular the prefrontal cortex (PFC) has been demonstrated (Devoto

et al., 2005, 2001; Devoto and Flore, 2006; Kempadoo et al., 2016; McNamara and Dupret,

2017; Smith and Greene, 2012). LC release of dopamine (DA) accounts for around 73% of all

the dorsal hippocampal DA (Kempadoo et al., 2016).

Besides such neurotransmitter heterogeneity, LC projections onto the different brain

areas are not uniform. Agster and colleagues (2013) characterized the repartition of synaptic

boutons across the rat brain, finding greater density in frontal cortices than other cortices,

and in superficial than deep layers. Comparable results were obtained in primate brains (Lewis

and Morrison, 1989; Morrison and Foote, 1986). Complementarily, several studies have

brought to light the modular organization of the LC, using various techniques. First, messenger

ribonucleic acid (mRNA) sequencing consists in using viruses to inject specific mRNAs, called

barcode mRNAs, into precise subpopulations of neurons. The mRNAs are then transported to

the synaptic boutons of the axons. Identifying the final locations of these barcode mRNAs

allows identifying the projection patterns of the neurons in which the viruses were injected

(Kebschull et al., 2016). Second, viral-genetic tracing consists in injecting viruses into specific

brain regions. The viruses are modified to infect specific subpopulations of neurons (for

instance the LC) in genetically modified individuals (Cre mice) and cause the infected neurons

to express fluorescent proteins. In the case of anterograde viruses, the infection occurs at the

level of the synaptic bouton of neurons and the viral material transits towards the neuronal

body. The location of the infected cell bodies (e.g. of the LC) and therefore the inputs of the

area of interest can be revealed thanks to the fluorescent protein expression. The opposite is

true for retrograde viruses. This technique allows identifying the input and output circuitry of

areas of interest (Schwarz et al., 2015). Third, chemogenetics allow functional dissections of

neuronal subpopulations. For instance, to identify the role of projections of the LC onto certain

areas, retrograde viruses causing the expression of specific receptors (under a genetic

promoter present only in noradrenergic cells) are injected in the projection area of interest.

The retrograde virus thus causes the expression of the receptor specifically in the LC neurons

projecting onto the area of interest. The expressed receptor can bind with a certain drug, and

this binding causes either the activation or inhibition of the neurons expressing it. In behaving

animals, this method allows identifying the processes for which specific projections of the LC

are crucial (Hirschberg et al., 2017; Uematsu et al., 2017).

55

Overall, studies uding these novel methods showed that LC neurons are grouped in

modules, receiving specific inputs and providing target-specific outputs. First, neurons differ

in terms of connections (inputs and outputs), with, for example, neurons receiving inputs from

the amygdala virtually not projecting onto the medulla (Schwarz et al., 2015). Second, these

connectivity-defined modules tend to be morphologically different at the cell level, and

spatially grouped (especially along the antero-posterior axis, Loughlin et al., 1986). The dorsal

part of the LC is constituted mostly of fusiform cells while ventral LC neurons tend to be

multipolar (Loughlin et al., 1986; Swanson, 1976). Moreover, no medulla projecting neurons

can be found in the posterior part of the LC, and most forebrain-projecting LC neurons are

located dorsally. LC neurons projecting onto the cortex have, however, been found to be

scattered throughout the nucleus (Schwarz et al., 2015). Additionally, LC axons tend to

collateralize throughout the brain, i.e. individual LC neurons project onto several brain areas

(Nagai et al., 1981; Room et al., 1981; Steindler, 1981). Interestingly, Chandler and colleagues

(2014) demonstrated that such finding does not apply within cortical areas (orbitofrontal

cortex, anterior cingulate cortex, medial prefrontal cortex, and motor area 1). In other words,

LC neurons projecting onto a specific cortical area do not project onto another cortical area,

although they may still project both onto a cortical and a subcortical area (Uematsu et al.,

2017).

Third, different input and projection patterns are associated with differential

implication in cognitive processes, for example learning (Uematsu et al., 2017; Wagatsuma et

al., 2018), or nociception and aversion (Hirschberg et al., 2017). Despite focusing on restricted

subsets of cognitive functions and projection areas, these studies provided evidence of the

necessity to stop considering the LC as homogenous, anatomically and functionally. Early

studies reported that a small proportion of LC neurons exhibit different electrophysiological

properties ex vivo (brain slices, Williams et al., 1984) and in particular different shapes of

action potentials and different spontaneous firing rates. More recently, Totah and colleagues

(2018) recorded a very large number of LC single-units in vivo (anaesthetized rats) and

analyzed the relation between their spontaneous and evoked firing, their projection patterns,

and their the shape of their action potentials. They showed that LC neurons generally do not

fire synchronously (15 to 16% synchrony), although small ensembles of neurons, generally

spatially grouped, sharing gap junctions, and likely to project onto identical areas can be

synchronized. Additionally, the neurons of these ensembles tended to share waveform

56

characteristics, and waveforms differences were associated with differences across a wide

range of other electrophysiological properties. This study reinforced the idea that LC neurons

are not functionally homogenous, and that different electrophysiological properties tend to

be associated with different projection patterns, and therefore putatively different effects on

cognitive functions.

Overall, this body of literature provides evidence that LC neurons are not homogenous,

and are dividable into modules differing in their patterns of inputs and outputs, cell

morphology, electrophysiological properties, receptor expression pattern and

neurotransmitter expression.

d) Noradrenergic receptor distribution and actions in

target areas

Adding heterogeneity on top of the modular organization of the projections of LC

neurons, noradrenergic receptors are not evenly distributed throughout the brain. NA has

nine different receptors: α1A, α1B, α1D, α2A, α2B, α2C, β1, β2 and β3. α1 and β receptors are

located post-synaptically, while α2 receptors have both presynaptic and post synaptic

locations, with a majority located post-synaptically (U’Prichard et al., 1979). Adrenergic

receptors have different affinities for their ligand and distribution patterns throughout the

brain, with α2 adrenergic receptors displaying a much higher affinity for NA compared to the

others (Arnsten, 2000). Illustrating the uneven distribution of adrenergic receptors, higher

concentration of α2C receptors have been found in the deep cortical layers compared to the

superficial layers (Nicholas et al., 1993). Consequently, the effects of noradrenergic release at

the neuronal level is highly dependent on the brain area and on the current availability of NA

(reviewed by Arnsten, 2000). For instance, in the amygdala and hippocampus, fixation of NA

on β receptors (Cahill and McGaugh, 1996; Hopkins and Johnston, 1988; Roullet and Sara,

1998), in synergy with α1 receptors (Chaulk and Harley, 1998; Ferry et al., 1999; Puumala et

al., 1998) facilitates memory consolidation. Conversely, α2 receptor stimulation impairs it in

the amygdala (Genkova-Papazova et al., 1997) but not the hippocampus (Sirviö et al., 1991).

This pattern is reversed for the PFC. Indeed, β receptors do not have much effect on PFC-

associated functions such as working memory (Arnsten and Goldman-Rakic, 1985; Li and Mei,

1994), while α2 receptor stimulation has strong beneficial effects (Arnsten, 1997; Arnsten et

57

al., 1988; Arnsten and Goldman-Rakic, 1985; Franowicz and Arnsten, 1998; Mao et al., 1999;

Rämä et al., 1996; Tanila et al., 1996).

In summary, despite producing mainly one neurotransmitter, NA, LC neurons have

diverse inputs and targets, with a modular organization of their connections, diverse

electrophysiological properties, such as excitability, and diverse effects at the neuronal level

depending on the target area. In the following parts, I quickly review frequently used proxies

for LC activity, and then focus on functions of LC activity and NA release relevant to the

hypothesis of an implication in effort processing. I start with brief reviews of accounts of the

implication of the LC/NA system in resource mobilization for costly cognitive and physical

processes, starting with two global processes associated with overall increases in cognitive

and physical expensed resources: arousal and stress. I then review evidence for the

implication of the LC/NA system in attention. I finish with more detailed accounts of the

impact of LC activity on goal-directed functions requiring resource mobilization, i.e. effort

(cognitive control or physical effort), with a special focus on decision-making. Finally, I review

different mechanistic accounts for the effects of LC activity on target areas at the neuronal

and network level that could explain the behavioral effects.

2. Assessing LC activity: proxies for LC activity

a) P3 ERP

The P3 is a positive-going ERP (see Figure 18 for details). Although first found in humans

(and mostly used in humans), the P3 component can be found in a large variety of species:

monkeys, cats and rats (Arthur and Starr, 1984; Jodo et al., 1995; O’Connor and Starr, 1985).

P3 responses were shown to correlate with the activity of a large number of cortical and

subcortical areas, for example the PFC (Kirino et al., 2000; McCarthy et al., 1997), the parietal

cortex (Smith et al., 1990; McCarthy et al., 1997), the temporal lobe (McCarthy et al., 1989),

and the thalamus (Yingling and Hosobuchi, 1984). Overall, studies on the brain areas

responsible for the P3 component have pointed towards a relation with the temporal parietal

junction, the medial temporal complex and the lateral PFC (Soltani and Knight, 2000).

Considering the multiplicity and wide spatial distribution of brain areas contributing to

the P3 component, several teams have proposed that it might in fact be caused by the activity

of a neuromodulatory system sending projections throughout the brain, such as the LC

58

(Nieuwenhuis, 2011; Nieuwenhuis et al., 2011, 2005; Pineda et al., 1989). Indeed, the areas

whose activity has been linked to the P3 component happen to receive strong projections

from the LC. Moreover, P3 magnitude is affected by the temporal frequency of stimuli, with

greater magnitude for highly infrequent and therefore surprising stimuli (Polich, 2007;

Verleger et al., 1994). Such a pattern is reminiscent of the characteristic activation of LC

neurons by surprising and salient stimuli (Bouret and Richmond, 2009; Bouret and Sara, 2004;

Grant et al., 1988; Vankov et al., 1995), and the timing of the P3 responses would be

compatible with being caused by that LC response (Nieuwenhuis et al., 2005). Additionally, P3

responses are sensitive to attention, and strong proof for the implication of the LC/NA system

have been provided by a large body of literature, reviewed in part I.B.3.c).

Besides these indirect correlative accounts of the relationship between LC activity and

the P3 component, Swick and colleagues (1994) performed simultaneous recordings of LC

activity and P3-like potentials in monkeys, showing directly the correlation between LC bursts

of activity and the P3 component. Moreover, in order to establish the causality of the

relationship between LC activity and P3 responses, Pineda and colleagues (1989) recorded P3-

like potentials in monkeys after bilateral LC lesions. They report a decrease in the magnitude

of the P3-like component, with the decrease in magnitude correlating with the extent of the

lesions. De Rover and colleagues (2015) showed that blocking β adrenergic receptors with

propranolol modulated the amplitude of the P3 component in response to stimuli, in a

direction dependent of the participants’ initial level of arousal. More recently, Vazey and

colleagues (2018) used optogenetics to stimulate LC neurons, and showed that bursts of LC

spikes, but not low frequency discharge, elicited P3-like responses in the EEG (Figure 21).

Vazey et al., 2018, Figure 2B&D

Figure 21: ERPs evoked by LC optogenetic stimulation (adapted from Vazey et al., 2018).

Following laser simulation mimicking bursts of LC activity (pattern shown on the right side),

the ERP signal showed P3-like responses (330ms latency) (dark blue). It was not the case after

low frequency stimulation (cyan, pattern shown on the right side). The x-axis represents time

from optic stimulation, and the y-axis represents the recorded EEG signal.

59

Altogether, these studies indicate a correlation between LC activity and the P3. Note

however that this relationship lacks temporal precision and specificity, as the activity of many

other areas also correlates with these components.

b) Pupil dilation

Considering the difficulty of accessing the anatomical position of the LC and its relative

invisibility in regular MRIs, numerous teams routinely make the choice to approximate LC

activity with several physiological easily measured proxies, although these are less specific and

less precise (both regarding the brain areas correlating and regarding time dynamics). The

main proxy is pupil dilation, of particular interest for that manuscript considering the positive

relationship between pupil dilation and physical and mental effort (see part I.A.1.c)(3) and

part I.A.2.g)(2), respectively).

Several studies using electrophysiology (Joshi et al., 2016; Varazzani et al., 2015) or

functional magnetic resonance imaging (fMRI, Murphy et al., 2014) to measure LC activity

have reported positive correlations between LC activity and pupil dilation in different types of

tasks. The studies from Murphy and colleagues (2014) and Joshi and colleagues (2016)

however brought to light the lack of specificity of that measure since both reported

correlations between neuronal activity and pupil diameter in additional areas, including other

parts of the brainstem. A few pharmacological studies explored the causal relationship

between LC activity and NA availability in the brain and pupil diameter. Phillips and colleagues

(2000) used an α2 agonist (clonidine) to decrease the activity of LC neurons, or an α2

antagonist (yohimbine) to increase the activity of LC neurons, in healthy humans, and found

that pupil diameter decreased under clonidine and increased under yohimbine. Additionally,

patients suffering from a deficiency of the dopamine-β-hydroxylase (DβH), a critical enzyme

for NA synthesis, and therefore lacking brain NA, were shown to have abnormally small or

even absent pupil responses to task events (Jepma et al., 2011).

60

(A) Joshi et al., 2016, Figure 3C left panel (B)

(C) Murphy et al., 2014, Figure 2A (D) Murphy et al., 2014, Figure 2B

Figure 22: Positive correlation between pupil diameter and LC activity

(A,B) Positive correlation between pupil diameter (average over trial, z-scored, x-axis) and LC

rate (extracellular recordings, rate, y-axis), in example session. The lines represent the linear

fit of the data.

(A) Adapted from Joshi et al., 2016.

(B) Data pooled by bins for visual display (adapted from Varazzani et al., 2015).

(C,D) Positive correlation between LC blood-oxygen-level dependent (BOLD) signal and pupil

dilation.

(C) Whole-brain map showing brain regions whose unsmoothed BOLD signal was correlated

with pupil diameter at rest or during an oddball task, and overlap between the two.

(D) Areas of the pons with effect of pupil dilation on BOLD signal (colors indicate size of F), with

atlas-derived location of the LC highlighted. LC BOLD activity correlated with pupil dilation.

Note however that the areas correlating to pupil diameter in the brainstem were not restricted

to the LC (adapted from Murphy et al., 2014).

To summarize, two non-invasive proxies are quite frequently used as proxies for LC

activity, despite their lack of temporal precision and mediocre specificity. Note also that

studies looking for correlations between these two proxies have provided inconsistent results

(Kamp and Donchin, 2015; LoTemplio et al., 2021; Murphy et al., 2011).

61

3. LC activity and enhancement of processes requiring

resource mobilization

The hypothesis of the implication of the LC in effort stems from the various accounts

of its implication in processes involving global increases in cognitive and physical expensed

resources via global increases in cerebral and motor activity: arousal, stress and attention. I

will review the role of the LC/NA system in these three processes in the next parts.

a) LC activity and arousal

Arousal, the physiological and psychological state of alertness or awareness,

fluctuates, with two major states: sleep and wake. Within wake, arousal can vary from

drowsiness to high-arousal and stress. LC neurons are virtually silent during rapid eye

movement (REM) sleep, have a low rate during slow wave sleep, a moderate baseline rate

with bursts of activation for salient stimuli during stress-free waking, and a high baseline rate

during stressful events (Aston-Jones and Bloom, 1981). Arousal has been repeatedly shown to

be promoted by LC activity and NA discharge in the brain (Aston-Jones, 2005; Carter et al.,

2010; Foote et al., 1983; Jouvet, 1969; Saper et al., 2005; Sara, 2009; reviewed in Berridge and

Waterhouse, 2003). LC neurons are virtually silent during sleep (see Figure 47B in

experimental results for example, and changes in firing rate predict sleep-wake transitions.

Moreover, LC neurons respond to salient events, and these responses prolong waking periods

(Aston-Jones and Bloom, 1981; Foote et al., 1980; Hobson et al., 1975).

One widely used application of such impact of NA on arousal levels consists in

administrating α2-agonist drugs (clonidine, medetomidine, xylazine…) as sedatives for general

anesthesia (De Sarro et al., 1987; Gatti et al., 1988). Indeed, such drugs act on pre-synaptic

regulatory receptors of LC neurons, thus inhibiting NA neurotransmission and LC activity, as

sedatives for general anesthesia (De Sarro et al., 1987; Gatti et al., 1988). Coherently, blocking

the adrenergic receptors responsible for a large part of the post-synaptic excitatory effects,

i.e. α1 and β receptors, was shown to have sedative effects (although such pharmacological

methods are not routinely used for anesthesia, Berridge and España, 2000).

Considering the relatively low specificity and large diffusion of drugs targeting

adrenergic receptors, later accounts of the sedative effect of LC activity inhibition were made

62

using more precise techniques such as local infusions of drugs in the vicinity of the LC while

recording neuronal activity (Adams and Foote, 1988; Berridge and Foote, 1991). Conversely,

administration of NA itself directly into brain ventricles or the forebrain (Flicker and Geyer,

1982; Segal and Mandell, 1970) promotes wakefulness. Additionally, activation of the LC was

shown to promote forebrain activation (assessed via EEG, in anesthetized animals, Berridge

and Foote, 1991), and the NA-dependence of this effect was assessed via pretreatment with

noradrenergic inhibitors (α2-agonist and β-antagonist). Much recently, Carter and colleagues

(2010) used optogenetic techniques, that allow specifically targeting neuronal populations

such as NA producing neurons, to demonstrate the association between LC firing, cortical

activity, sleep-wake transitions, and arousal in general. Practically, they inhibited or

stimulated LC firing and observed that inhibition of LC activity promoted wake-sleep

transitions, without increasing the duration of sleeping periods, and that stimulating LC

activity caused sleep to wake transitions (Figure 23), thus providing precise and selective

evidence for a causal implication of LC activity in promoting waking states.

Overall, these accounts of the implication of LC activity in arousal provides a first

indication that the LC/NA system is involved in mobilizing resources for behavior, provided

that being awake requires more resources than being asleep (at least metabolic resources).

(A) Carter et al., 2010, Figure 3A (B) Carter et al., 2010, Figure 4A

Figure 23 : Optogenetic inhibition of LC activity promotes sleep while stimulation promotes

wake (Adapted from Carter et al., 2010).

(A) Optogenetic inhibition of LC activity (eNpHR-eYFP) reduces wake duration in proportion of

total time compared to control group (eYFP) and increases non REM sleep duration in

proportion of total time.

(B) Optogenetic stimulation of LC activity at 3Hz (ChR2-eYFP) increases wake duration in

proportion of total time and decreases sleep duration in proportion of total time, compared to

baseline (no stimulation) and control group (eYFP).

Abbreviations: eNpHR: halorhodopsin, yellow light sensitive chloride pump; ChR2:

channelrhodopsin-2, blue-light sensitive cation channel; eYFP: yellow fluorescent protein,

marker and control; REM: rapid eye movement sleep; NREM: non-rapid eye movement sleep.

63

*: p<0.05; **: p<0.01; ***: p<0.001.

b) LC and stress-related responses

Stress-related responses regroup a larger range of behavioral responses such as fear,

anxiety and avoidance. Stress promotes arousal, vigilance and attention, rendering the

cognitive resources necessary for an appropriate response to the stressing event available4. In

an effort to further explore the relationship between LC activity and resource mobilization for

costly processes, this part will review the litterature regarding the relationship between LC

activity and stress responses.

The LC is a central part of the ventral ‘fight-or-flight’ response system (Koob, 1999;

Sved et al., 2002), with strong reciprocal connections with the different regions of the

amygdala (Bouret et al., 2003; Buffalari and Grace, 2007; Van Bockstaele et al., 2001), a key

region for stress responses (LeDoux, 2000; McGaugh, 2004) and stress-related disorders

(Rauch et al., 2000; Shekhar et al., 1999). NA has been shown to be involved in the modulation

of responses to various types of stressors: social stress (Bingham et al., 2011), pain

(footshocks, Cassens et al., 1981), predator stress (Curtis et al., 2012) and homeostatic stress

(hemodynamic stress, Valentino et al., 1991). Several stress related disorders have been

associated with increased discharge or decreased sensitivity to inhibition of LC neurons (Koob,

1999; Olson et al., 2011; Raskind et al., 2013; Valentino and Van Bockstaele, 2008), and

therapeutic strategies for such conditions have focused on regulating noradrenergic effects.

For example, post traumatic stress disorders (PTSD), often encountered in people exposed to

prolonged stressful situations (e.g. soldiers during a war, abuse victims) have been successfully

treated with α1 adrenergic receptor antagonists such as prazosin (Raskind et al., 2013).

LC neurons respond strongly to stressors. Different studies have reported different

neuronal effects: increases of baseline rate of discharge (Curtis et al., 2012, 1997; Valentino

and Foote, 1988) or increases in rate evoked by salient stimuli and evoked response to

baseline (signal-to-noise) ratio (Bingham et al., 2011, see Figure 24A). Note that this

discrepancy could be due to the fact that, in rats, individuals in different age categories seem

to show different effects (Bingham et al., 2011).

4 Note that excessive stress tends to impair cognitive functioning, and at least some part of such impairment is

dependent on α1 adrenergic receptors (Birnbaum et al., 2004).

64

LC activity modifications in response to stressors were shown to be associated with

cortical EEG activation (Curtis et al., 1997). Moreover, it is necessary and sufficient for the

onset of anxiety behavior, although the opposite is not true, as inhibiting LC activity does not

produce anxiolytic effects (McCall et al., 2015). Additionally, stress-related LC activity

modifications were shown to be mediated by corticotropin-release hormone (CRH) secretion

in LC, most likely from axons emerging from the amygdala (McCall et al., 2015; Reyes et al.,

2008). Indeed, CRH microinfusions in the LC (Curtis et al., 1997; Page and Abercrombie, 1999),

as well as optogenetic stimulation of CRH-releasing fibers from the central amygdala (McCall

et al., 2015), produce robust increases in firing rate, and CRH antagonists prevent the increase

in rate of LC neurons in response to stress (Curtis et al., 2012; Valentino et al., 1991).

(A) Bingham et al., Figure 6A (B) Curtis et al., 1997, Figure 1A, bottom panel

Figure 24 : LC activity is modulated by stress, via the action of CRH.

(A) In rats, following social stress, LC neurons showed increased responses to salient stimuli

(grey) compared to control group not exposed to stress (black) (Adapted from Bingham et al.,

2011).

(B) In halothane anesthetized rats, after CRH microinfusion (30ng) in the LC, LC rate increased

immediately and remained higher than before infusion for 30 minutes (adapted from Curtis et

al., 1997).

In summary, an extensive body of literature points towards a preponderant role of the

LC in stress-related responses. For behavior to be adaptive, detecting stressful elements in the

environment has to trigger costly processes, and the fact that the LC activation is involved in

stress-related responses provides additional elements in favor of a role of the LC in mobilizing

energy for costly cognitive and physical processes.

c) LC activity and attention

First accounts of the role of the LC/NA system in attention were made in the 1980s

based on pharmacological and clinical studies (Clark et al., 1987; Robbins, 1984; Sara, 1985).

For example, clinically, selective NA reuptake inhibitors, such as atomoxetine, are efficient for

65

relieving the attention-related symptoms of patients suffering from ADHD (reviewed in

Clemow and Bushe, 2015).

Petersen and Posner (2012) described attention as composed of three sub-processes:

alerting, orienting and attentional executive control. In tasks requiring responding to certain

stimuli, if these stimuli are cued by a warning signal, RTs are shorter. This alerting effect

decreases in case of administration of α2 agonist drugs such as clonidine and guanfacine,

(Coull et al., 2001; Witte and Marrocco, 1997), suggesting that the LC/NA system may be

causally involved in attentional alerting. In coherence with that idea, individual LC neurons

produce a characteristic activation-inhibition pattern of response to novel salient stimuli, with

habituation if the stimulus is repeated a certain number of times (Bouret and Richmond, 2009;

Bouret and Sara, 2004; Grant et al., 1988; Vankov et al., 1995, Figure 20).

Attention orienting refers to the ability to focus attention at a particular point in time

and/or space. For instance, tasks used to test spatial orienting of attention consist in asking

subjects to report detecting a target after presenting either a valid (providing accurate

information on future target position) or invalid (inaccurate information) distractor. The

orienting effect consists in the observation of shorter RTs in the presence of valid distractors.

Several studies have reported an impact of pharmacological modifications of NA

neurotransmission on attentional orienting. Specifically, increases in brain NA with reuptake

inhibitor atomoxetine or optogenetic activation of LC neurons caused improvements of

orienting in the majority of subjects (Bari et al., 2020; Reynaud et al., 2019), while decreases

in LC activity in response to α2 agonist (clonidine) administration or optogenetic inhibition

caused deterioration of orienting (Bari et al., 2020; Clark et al., 1987; Coull et al., 2001). Note

however that this effect of NA on orienting is probably context dependent, as it was only

reported in highly predictive contexts (high proportion of valid targets) but was not found

otherwise (Witte and Marrocco, 1997).

Overall, several studies converge towards an implication of the LC/NA system in

attention, a process allocating resources to monitoring external stimuli. In later parts, I will

review a potential mechanism through which LC activity may promote attention: increased

signal-to-noise ratio in brain areas involved in sensory processing (see part I.C.2.a) for details).

This potential mechanism resonates with the finding from Reynaud and colleagues (2019) that

66

the effect of atomoxetine on spatial orienting is in part due to an enhancement of the sensory

information accumulation process.

4. LC and execution of costly goal-directed actions

In the previous parts, I reviewed evidence for the implication of the LC/NA system in

various processes involving resource mobilization: wakefulness, stress and attention. These

three however constitute cognitive states and are not goal-directed. In the following

paragraph, I review evidence for the implication of the LC/NA system in executing costly

actions, whether they be costly due to mental and/or physical effort requirements. Note that

I will focus specifically on the role of the LC in the exertion of cognitive control for decision-

making in a following part (I.B.5).

In a previous part of this manuscript, I reviewed extensively the positive relationship

between mental effort and pupil diameter, and between mental effort and the P3 component

of ERPs (see part I.A.1.c)(3). Since pupil diameter (Joshi et al., 2016; Murphy et al., 2014;

Varazzani et al., 2015) and the P3 (Swick et al., 1994; Vazey et al., 2018) are proxies for LC

activity (see part I.B.2), indirectly, LC activity could increase during mental effort exertion.

Evidence for a causal role of LC activity in improving cognitive functioning and in particular

mental effort was provided by studies using pharmacological modifications of LC activity

(reviewed by Robbins and Arnsten, 2009). Such studies were done in patients suffering from

neuropsychiatric diseases reducing cognitive control capacity, and in healthy humans and

animals, with numerous studies focusing on different components of mental effort separately:

working memory, inhibitory control and task-switching.

Atomoxetine, a reuptake inhibitor of NA, increases NA levels throughout the brain and

especially in frontal cortices. It is routinely used in the treatment of ADHD. This disorder is in

particular associated with impulsivity, and difficulties with tasks requiring inhibition, working

memory and task-switching (DSM-5, American Psychiatric Association and American

Psychiatric Association, 2013). This symptomatology is believed to be due to an impairment

of the capacity for cognitive control exertion (e.g. Arnsten and Rubia, 2012; Geurts et al., 2004;

Pennington and Ozonoff, 1996; Sergeant et al., 2002). The alleviation of ADHD symptoms by

atomoxetine indirectly suggests that NA brain levels may be critical for efficient cognitive

control exertion (Chamberlain et al., 2007).

67

Additionally, atomoxetine was shown to improve response inhibition, causing a

decrease in impulsive behavior in rats (e.g. Arnsten, 2006; Fernando et al., 2012; Navarra et

al., 2008; Robinson, 2012; Robinson et al., 2008), healthy humans (Chamberlain et al., 2009)

and patients suffering from a large range of neuropsychiatric conditions (reviewed by Pattij

and Vanderschuren, 2008). Atomoxetine as well as another reuptake inhibitor of NA,

desipramine, were shown to improve stop signal reaction times (SSRTs, an estimation of the

time taken to attend to and respond to a stop signal following a go signal) in humans

(respectively: Chamberlain et al., 2006a; Overtoom et al., 2003) and rats (atomoxetine:

Robinson et al., 2008). Atomoxetine administration in the context of ADHD was also shown to

improve Stroop task performance (Faraone et al., 2005). Although atomoxetine is not

perfectly specific of NA reuptake, and also tends to increase DA levels in the brain (Bymaster

et al., 2002), it was shown that the effects of atomoxetine on inhibitory control were not

reproduced by selective DA reuptake inhibitors (Bari et al., 2009), reinforcing the idea of a

critical role of NA. Additionally, in Go/No-Go tasks, α2 receptor blockade by yohimbine in

primates was shown to impair performance, suggesting that increases in NA availability may

act on response inhibition by stimulating these receptors (Ma et al., 2003).

Regarding the implication of the LC/NA system in inhibitory control, clear evidence for

a causal role of LC activity was provided by an optogenetics study by Bari and colleagues (2020,

Figure 25). They showed that optogenetic manipulations of LC activity in a task requiring

response inhibition (Figure 25A) modified performance: increasing LC activity increased

performance Figure 25C&D) while decreasing LC activity decreased performance (Figure

25E&F). Additionally, mice were trained on a modified Posner task (Figure 25B), in which

distractors either predicted the position of a target (valid distractor), provided no information

(neutral distractor) or indicated a position different from the future position of a target. In

that task, the influence of distractors on RT decreased in case of LC stimulation (Figure 25G&H)

and increased in case of LC inhibition (Figure 25I&J).

Finally, using Calcium imaging, the authors report finding, first, that responses with

lesser effects of distractors (less effect of distractor validity on RT and less errors) and better

inhibitory control (correct versus premature responses) were preceded by higher

spontaneous LC activity (Figure 25K-P) and stronger responses to distractor onset (Figure

68

25M) than responses with strong effects of distractors and insufficient inhibitory control

exertion. Second, responses to target onset were greater before fast responses (Figure 25Q-S).

Altogether, these results would indicate that the baseline rate of LC neurons as well as

the strength of their pre-action (post-target onset) responses are causally related to the

strength of the cognitive control exerted throughout a trial, at least in the context of inhibitory

control exertion.

(A) Bari et al., 2020, Figure 2A (B) Bari et al., 2020, Figure 3A

(C) Bari et al., 2020, Figure 2E (D) Bari et al., 2020, Figure 2F

(E) Bari et al., 2020, Figure 2I (F) Bari et al., 2020, Figure 2J

(G) Bari et al., 2020, Figure 3D (H) Bari et al., 2020, Figure 3H

(I) Bari et al., 2020, Figure 3F (J) Bari et al., 2020, Figure 3J

(K) Bari et al., 2020, Figure 4G (L) Bari et al., 2020, Figure 4H

(M) Bari et al., 2020, Figure 4I (N) Bari et al., 2020, Figure 4J

(O) Bari et al., 2020, Figure 4K (P) Bari et al., 2020, Figure 4L

(Q) Bari et al., 2020, Figure 4M (R) Bari et al., 2020, Figure 4N

(S) Bari et al., 2020, Figure 4O

Figure 25: Optogenetic manipulations and Calcium imaging of LC activity demonstrate a causal

role of the LC/NA system in inhibitory control exertion (adapted from Bari et al., 2020)

(A) Schematic representation of the two-choice task. Mice had to wait for the appearance of a

target in one of two locations during a delay and poke target location to receive a food reward.

Inhibitory control was required to avoid premature responses and sustained attention was

required to choose the correct target location. LC activity was modified during the delay.

(B) Modified Posner task. After a certain delay, a distractor could (cued condition) appear or

not appear (no-cue condition). Then the target appeared and the mice had to poke its location.

The distractor could either provide neutral, valid or invalid information about target location.

The task required response inhibition to refrain from premature responses and to avoid

incorrect responses due to distractors. LC activity was modified during the delay.

(C-F) Two-choice task: Effects of optogenetic (C,D) activation or (E,F) inhibition of LC (blue or

green bars, respectively) compared to control (grey) on (C,E) performance (percentage of

69

correct responses) and (D,F) premature responses. LC activation increased performance and

decreased premature responses, while LC inhibition produced the opposite pattern of results,

suggesting an enhancement of selective attention and inhibitory control.

(G-J) Modified Posner task: Effects of optogenetic (G,H) activation or (I,J) inhibition of LC

activity (filled blue or green dots respectively) compared to control (unfilled dots) on (G,I)

performance validity effect (percentage correct in valid - invalid conditions) and (H,J) RT

validity effect (RT valid - RT invalid). LC activation reduced validity effects and LC inhibition

enhanced validity effect, suggesting a causal role of LC activity in inhibitory control exertion.

(K-S) Calcium imaging of LC activity in the modified Posner task, during (K,L,N,O,P) pre-

distractor period (baseline), (M) distractor presentation, and (Q,R,S) target presentation.

Correct responses were preceded by higher baseline activity (K,L) compared to (K) incorrect

and (L) premature responses, and by (M) higher activity during distractor presentation

compared to premature responses. (N-P) Higher baseline activity and (Q-S) higher activity

during target presentation was associated to lesser validity effect on RT. (O,P) Increased

baseline and (R,S) post-target onset activity preceded (O,R) slow valid and (P,S) fast invalid

trials, compared to (O,R) fast valid and (P,S) slow invalid trials, respectively. There was no

difference in (N) baseline activity or (Q) target-evoked activity between fast and slow trials

with neutral distractors.

*: p<0.05; **: p<0.01; ***: p<0.001; n.s.: non significant; TW: time window, F/F: difference

between recorded signal (fiber photometry) and mean signal, divided by mean signal

(normalized), Corr: correct; incorr: incorrect ; prem: premature.

Regarding working memory, an early study using intra PFC injections of an α2

antagonist (yohimbine), and α1 antagonist (prazosin) or a β antagonist (propranolol)

demonstrated the critical role of α2 adrenergic receptors in working memory, since only α2

adrenergic receptor antagonism had effects on performance in a spatial working memory task

(Li and Mei, 1994). This finding was confirmed by various other pharmacological studies (e.g.

Gamo et al., 2010; Li et al., 1999; Mao et al., 1999; Tzavara et al., 2006; Wang et al., 2007) as

well as LC lesion studies (Coradazzi et al., 2016). Overall, β receptor blockade (propranolol) as

well as α2 receptor stimulation (clonidine) impaired working memory while NA reuptake

70

inhibitors enhanced working memory (reviewed by Chamberlain et al., 2006b; Ellis and

Nathan, 2001).

Based on their pattern of response to changes in rules, LC neurons could participate to

the learning of new task contingencies, and therefore to set-shifting performance. LC

responses to new task-relevant stimuli during learning appear early on, after only a few trials,

long before behavior adjusts to the new rules (Bouret and Sara, 2005). Early causal accounts

of the relationship between NA and set-shifting were provided by Devauges and Sara (1990),

through pharmacological enhancements of LC activity, and McGaughy and colleagues (2008),

through lesions of noradrenergic projections to the PFC. Additionally, Lapiz and Morilak (2006)

showed that α2 antagonist (atipamezole) administration improved set-shifting performance

by increasing NA availability for α1 receptors (effects of atipamezole were blocked by α1

antagonist administration). Similarly, increasing NA availability with acute or prolonged

administration of desipramine, a NA uptake inhibitor, in rats, increased set-shifting

performance (Lapiz et al., 2007), while selectively decreasing cortical NA availability through

lesions of the LC cells projecting there altered set-shifting performance (McGaughy et al.,

2008; Tait et al., 2007).

Besides facilitating adaptation to new task rules when the task imposes it, the LC/NA

system was shown to have a critical implication in switching from exploration to exploitation.

Indeed, in an exploration-exploitation tradeoff gambling task, Jepma and Nieuwenhuis (2011)

found that pupil dilation, which they used as a proxy for LC activity, increased more before

exploratory choices than exploitative choices, suggesting that brief activations of LC neurons

could lead to disengagement from exploration (Figure 26A). Moreover, individual pupil

baseline predicted individual tendencies for exploration, suggesting a relationship between LC

baseline activity and exploration (Figure 26B).

(A) Jepma ans Nieuwenhuis, 2011, Figure 2A (B) Jepma ans Nieuwenhuis, 2011, Figure 4A

Figure 26: Pupil diameter increased more before exploration than exploitation, and individual

pupil diameter baseline was positively correlated with individual tendency to explore (adapted

from Jepma and Nieuwenhuis, 2011).

71

(A) Time course of average pupil diameter aligned on trial-by-trial response (key press). Before

and after key press, pupil diameter was larger in exploratory trials.

(B) Positive correlation between average pupil diameter in mm (x-axis) and proportion of

exploratory choices (y-axis). Subjects exploring more had a larger average pupil diameter.

Gilzenrat and colleagues (2010) obtained similar results, with some complements. In

three distinct experiments, they demonstrated that exploration periods (task disengagement)

were associated with high pupil baseline while exploitation periods (task engagement)

corresponded to low pupil baseline with strong evoked pupil responses to task events.

Although pupil diameter is merely an imperfect proxy for LC activity, these results could

suggest that LC neurons spike at a high baseline rate during exploration and with a low

baseline but strong responses to task events during exploitation.

Providing proof of the causal link between LC activity and exploration/exploitation

tradeoff, increasing LC activity with DREADDs (designer receptor exclusively activated by

designer drugs) caused disengagement from exploration tasks and increased choice variability

(Kane et al., 2017, Figure 27A&B). Conversely, decreasing NA with clonidine (an α2 agonist

decreasing LC activity) led to decreased choice variability (Jahn et al., 2018, Figure 27C),

indicating a role of the LC in promoting behavioral variability and exploration.

(A) Kane et al., 2017, Figure 4, top left panel

(B) Kane et al., 2017, Figure 4, bottom left panel

(C)

P(c

ho

osi

ng

alte

rnat

ive

op

tio

n

Value of alternative option –

value of current option

72

Figure 27: Increasing LC activity caused increased exploration and decision variability, while

decreasing LC activity decreased choice variability.

(A) In a patch-foraging task, regardless of initial patch value i.e. initially available reward (x-

axis), increasing LC activity via DREADDs stimulation with clozapine-N-oxide (CNO) (blue and

green curves) decreased the time (number of trials) spent in the patch (y-axis) compared to

placebo (red) (figure adapted from Kane et al., 2017).

(B,C) Increasing LC activity increased choice variability.

(B) In a patch-foraging task, increasing LC activity caused the relationship between the

expected value of leaving the current patch (x-axis) and the probability of leaving the patch (y-

axis) to become shallower under CNO treatment (blue and green) compared to placebo (red).

(C) In a value-based choice task, decreasing LC activity by clonidine administration (orange)

caused increased choice variability compared to placebo (brown), materialized by a steeper

curve of the relationship between the value difference of the options (x-axis) and the

probability to choose the alternative option (y-axis) (figure adapted from Jahn et al., 2018).

More recently, Dubois and colleagues (2021) showed that, in an exploration-

exploitation trade-off task, administration of β receptor antagonist propranolol cause a

reduction of random exploration, i.e. of noise in choices. Note that propranolol could have

acted by increasing availability of NA (since it did not bind β receptors) and therefore increased

α2 receptor stimulation.

Considering the hypotheses regarding the aversive nature of cognitive control being

related to a regulation of the exploration-exploitation tradeoff, the involvement of the LC in

switching from exploitation to exploration could reflect an involvement of the LC in regulating

cognitive control allocation. This role could seem incompatible with its probable involvement

in promoting control engagement in working memory and response inhibition. However, set-

shifting and behavioral flexibility seem to depend on different receptors, with different

affinities for NA, than working memory and inhibitory control. In coherence with that,

atomoxetine doses that improve response inhibition (stop-signal performance in rats) appear

to be detrimental for set-shifting (Newman et al., 2008). Consequently, low doses of NA (low

firing rates of the LC) could be responsible for control exertion enhancement, while high doses

73

of NA (high firing rates of the LC) could be required for shifting from investing control in a

specific task (exploitation) to exploring other options. In other words, the relationship

between LC activity, NA levels, and cognitive control enhancement probably follows an

inverted U shaped curve, in which increasing NA facilitates control exertion but only up to a

certain point.

Regarding physical effort exertion, LC neurons have been shown to encode the exerted

force during force-discounting tasks, at the time when the force is exerted (Varazzani et al.,

2015). Additionally, pupil diameter has been shown to increase during physical effort exertion

in humans, and to correlate both with the objective and subjective perception of effort (Zénon

et al., 2014, see also Figure 3). These results would tend to indicate an implication of LC activity

in effortful physical actions, yet this increase in LC activity and pupil diameter could merely

reflect autonomic activation due to muscle contraction. However, pharmacological evidence

have complemented these results by providing evidence for a causal relationship between LC

activity and force production. Two studies, consisting in administrating an α2 agonist

decreasing LC activity, clonidine, to monkeys executing tasks requiring force exertion,

reported decreased force exertion (Borderies et al., 2020; Jahn et al., 2018). Interestingly, in

Borderies et al. (2020, in annex), we reported that this effect of clonidine was not the result

of a decreased capacity to exert force, at the muscular level, since the muscular contractility

index was not affected by the treatment. This indicates that decreased LC activity may cause

physical effort to be perceived as more aversive.

(A) (B) (C)

Figure 28: LC neurons encoded the exerted force during force exertion, and decreasing brain

NA decreased exerted force.

74

(A) Positive correlation between the firing of LC neurons around action onset (x-axis) and the

maximum force exerted by the monkeys (y-axis) during a one-option force-discounting task

(figure adapted from Varazzani et al., 2015).

(B) Exerted force (y-axis) in a reward-schedule task following placebo or clonidine

administration at different doses (x-axis). Clonidine administration decreased the exerted force

(figure adapted from Jahn et al., 2018).

(C) Maximum exerted force in percentage of the maximum force that the monkey can exert (y-

axis) in a two-options force-discounting task, after placebo (grey) or clonidine administration

(brown), as a function of the requested force (y-axis). Dashed line represents the Force target

i.e. percentage of the maximum force that had to be exerted to get the Reward, by condition.

Clonidine administration decreased the exerted force (figure adapted from Borderies et al.,

2020).

Altogether, numerous studies pointed towards a causal role of LC activity in all three

subcomponents of cognitive control: response inhibition, updating working memory, and task

switching. Generalizing effects on set-shifting, high levels of LC activity lead to disengagement

from task and exploration. Additionally, LC activity appears to be critical for physical effort.

Considering that cognitive control and physical effort both constitute decision variables (i.e.

subjects try to avoid having to make efforts, see parts I.A.1.a) and I.A.2.d)(2)) and that

cognitive control is beneficial for decision-making (see part I.A.2.d)(1)), I will now specifically

review evidence for the implication of the LC in decision-making.

5. LC and decision-making

In the following part, I will split the decision-making process into two sub-processes:

committing to a choice (action selection) and triggering the chosen action. Both steps are

susceptible of requiring cognitive control investment. For instance, at the stage of action

selection, cognitive control is required to choose with as little biases as possible. When a costly

action has been chosen, triggering it requires exerting control to override an automatic

(default) response consisting in not doing anything (see parts I.A.1.a), I.A.2.d)(2) and I.A.1.a)

and I.A.2.d)(2)). Here, I review the evidence regarding the role of the LC/NA system in these

two steps.

75

a) Action selection

Several pharmacological studies and lesion studies, in which LC activity or the effects

of NA on LC targets were manipulated over quite long periods of time, have indirectly pointed

towards a potential causal role of LC activity in the action selection process. Motivational

salience refers to the regulation of the intensity of the behaviors facilitating the attainment of

certain goal, and in that constitutes a prerequisite of action selection. Following the finding

that exposure to salient stimuli increases NA in the pre-frontal cortex (PFC) and nucleus

accumbens (Ventura et al., 2008, 2007), Ventura and colleagues (2007) decreased NA levels in

the medial PFC (mPFC) by producing selective destruction of LC neurons targeting the mPFC.

They found that mice with lower NA levels in the mPFC exhibited impaired motivational

salience.

Regarding action selection per se, pupil dilation studies provided indirect evidence for

the implication of the LC. Pupil dilation has been shown to increase with decisions (Beatty,

1982; de Gee et al., 2014; Einhäuser et al., 2008,2010; Gilzenrat et al., 2010; Lempert et al.,

2015; Nassar et al., 2012), especially when subjects respond against a bias, i.e. inhibit an

automatic response (de Gee et al., 2020, 2017, 2014). Considering the correlation between LC

activity and pupil diameter, and between cognitive control and pupil diameter, such findings

could indicate that the LC plays a role in allocating the control required to make decisions and

especially inhibit automatic responses. Additionally, in that very context, de Gee and

colleagues (2017) showed that the blood-oxygen level dependent (BOLD) fMRI signal in a

cluster of voxels including voxels compatible with the theoretical location of the LC was

negatively correlated with the strength of the decision biases (Figure 29A). This study further

advocates for the role of the LC in control exertion for decision-making and specifically action

selection, despite lacking spatial precision as well as not demonstrating a causality link.

76

(A) De Gee et al., 2017, Figure 2E left panel

(B) (C)

Figure 29: Effects of LC activity on action selection: increased LC activity, measured indirectly

through pupil dilation, was associated to decreased decision biases, while decreases in LC

activity by clonidine increased physical effort sensitivity.

(A) In a visual-detection task, the BOLD fMRI signal of a cluster of voxels corresponding to

brainstem neuro-modulatory nuclei including the LC (x-axis) was negatively correlated to the

strength of the bias in the evidence accumulation process of the decision (y-axis) (adapted from

de Gee et al., 2017).

(B) In a force-reward two-alternative choice task, decreasing brain NA with clonidine caused

an increase in the sensitivity of choices to the physical effort, materialized by a steeper slope

of the percentage of high force (HF) choices (y-axis) as a function of the absolute difference in

offered forces (|ΔForces|, x-axis) (figure adapted from Borderies et al., 2020).

(C) In the same task, clonidine did not affect the sensitivity to reward size, materialized by the

slope of the percentage of high reward (HR) choices (y-axis) as a function of the absolute

difference in offered rewards (|ΔRewards|, x-axis) (figure adapted from Borderies et al., 2020).

Complementing these findings, in Borderies et al., 2020 (in annex), we provided

evidence for a causal role of LC activity in physical effort-based action selection. We reported

that α2-agonist (clonidine) administration before a value-based decision-making task, with

options differing along the reward and physical effort dimensions, caused monkeys to be more

77

sensitive to the physical effort dimension while remaining equally sensitive to the reward

(Figure 29B&C). In other words, decreasing LC activity and brain NA levels influenced action

selection by making physical effort more aversive. Additionally, decreasing brain NA levels

caused monkeys to exert less force (Borderies et al., 2020; Jahn et al., 2018). We suggested

that decreasing NA increased the cost of effort, a computational variable representing the

amount of theoretical resources necessary for the execution of actions. If the cost of effort

increased, monkeys would theoretically avoid choosing high-effort offers more often, and this

would also have deleterious effects on action execution (force exertion). Thus, LC activity

appears to be critically but indirectly involved in the effects of physical effort on decisions.

Additionally, this finding could be interpreted in terms of effects on cognitive control

exertion for the decision process. Naturally, subjects have a bias towards avoiding high-effort

options. Consequently, choosing options associated with higher physical effort requires

exerting cognitive control, to override the default response. The decrease in high-effort

choices following decreases in LC activity could in fact indicate that LC activity is critical for

cognitive control exertion, at least for decisions. This idea also resonates with the negative

relationship between pupil dilation and decision biases and therefore putative decision

control exertion (de Gee et al., 2020, 2017, 2014).

Direct assessments of LC activity via in situ electrophysiological recordings, to our

knowledge, were never performed during value-based choice tasks with more than one offer

presented at a time (up to now, see experimental results of Study 2, part II.B). Consequently,

in one-option tasks, the choice that the subjects faced was between engaging in the trial, i.e.

trying to perform the requested action, and refusing the trial, i.e. not trying at all. Jahn and

colleagues (2020) report that, around cue onset in a one-option force-discounting task, LC

neurons differentiated between trials in which the monkeys would or would not engage, with

a higher spike count when the monkey would engage (Figure 30A). In the same vein, LC

neurons activated after a fixation break5 i.e. after disengagement from a trial (refusal to

engage), and the magnitude of this activation predicted engagement in the next trial (Figure

30B). Note however that this task did not allow differentiating value-based refusals from

putative automatic refusals of trials. Considering, however, that engagement was affected by

5 In this task, monkeys had to maintain gaze fixation onto the screen throughout trials, and therefore breaking fixation aborted trials.

78

task parameters in directions coherent with value-based refusals, I will interpret their results

under the hypothesis that monkeys performed a value-based analysis and cost-benefit

tradeoff to choose whether to engage in trials.

(A) (B) Engaged Not engaged Next trial: no engagement Engagement Cue Cue Fixation break Fixation break

Time from cue onset (s) Time from fixation break (s)

Figure 30: LC neurons encoded engagement in the current trial at the time of cue onset and in

the next trial around fixation breaks i.e. disengagement from trials (adapted from Jahn et al.,

2020).

(A) Raster plots of the activity of a representative LC unit aligned on cue onset, with trials split

between engaged trials (left, monkey will try squeezing the bar) and not engaged trials (right,

monkey will not try). The light brown line represents the spike density function. Around cue

onset in a force discounting-task, LC neurons positively encoded whether the monkey would

engage in the trial, i.e. not break fixation and try to perform the requested action (regardless

of the result of the action i.e. correct or incorrect).

(B) Raster plots of the activity of a representative LC neuron around fixation breaks. Trials are

split between fixation breaks followed by no engagement in the next trial (left) or followed by

engagement in the next trial (right). The light brown line represents the spike density function

and the grey dots represent cue onset. This neuron was activated around fixation breaks and

stronger activation predicted engagement in the next trial.

Choosing to engage in trials requiring force exertion can be considered as a form of

cognitive control exertion, since the default non-costly response would be refusing trials. In

79

refused trials, monkeys may have invested too little control and decided to forego the trial

even if this decision reduced reward rate, given that refused trials were repeated until

completion. The increased spike count of LC neurons around the presentation of cues when

monkeys would choose to engage (over and above the effects of task parameters) would be

compatible with a role of LC neurons in mobilizing cognitive control. Indeed, increased spike

count early after cue onset would have resulted in increased cognitive control exertion and

therefore higher probability to engage.

Selecting actions appropriately requires extensive information processes. Around the

time of presentation of information relevant for value-based decisions whether to engage in

trials, LC neurons exhibit an activation (Bouret et al., 2012; Bouret and Richmond, 2009; Jahn

et al., 2020; Varazzani et al., 2015). Several studies provided evidence of the encoding of the

amount of novel information carried by cues. Reward-schedule tasks consist in performing

series of presses to obtain a reward. The number of presses is variable, and along those series,

cues inform the monkeys of the number of remaining presses. Consequently, only the first cue

carries novel information regarding the length of the sequence, and the following ones only

duplicate that information. Notably, LC neurons have been shown to respond more strongly

(if not exclusively) to the cues signaling the first press of the sequence (Bouret et al., 2012;

Bouret and Richmond, 2009). In coherence with that, in a one-option force-discounting task,

Jahn and colleagues (2020) showed that, at the time of the presentation of the information

about the offer, LC neurons positively encoded the amount of information obtained. Indeed,

in case of refusal, trials were repeated, making a certain proportion of the cues uninformative

(providing redundant information), and LC neurons’ responses differentiated between

informative and uninformative cues (Figure 31).

80

(A) 1st cue Not 1st cue (B) New information No new information

Cue Cue Cue onset Cue onset

Time from cue onset (s)

Figure 31: LC neurons differentiated between cues carrying novel information and cues

carrying no novel information.

(A) Raster plots of the activity of an example LC unit in a reward-schedule task, around the

onset of the first cue of the sequence, carrying novel information about sequence length (left)

and other cues throughout the sequence (indicating the number of remaining presses, right),

carrying no additional information. Red and pink lines represent the spike density function. The

neuron exhibited no response to cue onset when the cue carried no additional information, and

activated in response to cue onset when the cue carried novel information (adapted from

Bouret et al., 2012).

(B) Raster plots of the activity of a representative LC neuron in a one-option force-discounting

task, contrasting trials with novel information associated to the cue (left) and trials with no

novel information carried by the cue (right) (adapted from Jahn et al., 2020). Raster plots show

a much weaker response of LC neurons to cue onset when the cue carries only redundant

information. The brown line represents the mean firing rate of the neuron across trials.

Moreover, LC neurons encode some of the information carried by the cues informing

subjects about the contingencies of trials, especially the offered reward (Bouret and

Richmond, 2015; Varazzani et al., 2015, Figure 32). In coherence with that, during economic

choices, pupil dilation has been shown to increase with expected value (Fiedler and Glöckner,

2012).

81

(A) (B)

Figure 32: LC neurons encoded some of the information provided by task cues on trial

contingencies, especially reward size.

(A) Raster plot of the activity of an example LC unit in a task consisting in pressing a grip for a

reward, around the onset of the cue indicating reward size. Trials are split by reward size,

darker blue indicates greater rewards. This unit encoded reward positively around cue onset

(adapted from Bouret and Richmond, 2015).

(B) Raster plots of the activity of an example LC neuron in a force-discounting task, around the

onset of the cue indicating the force-reward contingency. Trials are split along two dimensions:

offered reward (blue, horizontally) and requested force (red, vertically). Blue lines represent

the spike density function. This unit encoded reward size positively (adapted from Varazzani et

al., 2015).

b) Action triggering

LC neurons have been known for close to two decades to exhibit activation,

materialized by a few (sometimes even only one) spikes, slightly prior to action initiation

(Bouret et al., 2012; Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004; Clayton et al.,

2004; Varazzani et al., 2015), suggesting a potential implication of LC activity in the action

triggering process. Additional evidence comes from the recording of LC activity in Go/No-Go

tasks. Kalwani and colleagues (2014), in a Go/No-Go task consisting in triggering responses to

82

a stimulus (saccade) and sometimes receiving the order to abort that response, found that LC

neurons activated around responses consisting in a motor response, both correct (Go trials)

and incorrect (No-Go trials). They conversely did not activate around responses consisting in

the absence of production of a motor response, even if that response was the correct one

(No-Go trials). This type of task is however not value-based, and I will from now on focus on

the role of LC activity in triggering costly actions, which is believed to require exerting cognitive

control.

In the same vein, Bouret and Richmond (2015) used a paradigm in which some trials

were active, i.e. the monkey needed to perform an action (bar release), followed by a feedback

and a reward, and some trials were passive, i.e. the reward was delivered without any action,

after a “feedback”. Noticeably, LC neurons activated prior to feedback onset in active trials

(i.e. prior to action onset) but not in passive trials (Figure 33). These patterns of activity of LC

neurons eliminates the possibility that pre-action LC activation corresponds to reward

expectation and suggests that LC neurons may be involved in the action triggering process.

In tasks involving responding to a cue serving as a go-signal as soon as a decision has

been reached, LC neurons only responded once between cue onset and action onset, and that

activation was better aligned on the onset of the action than on the cue (Bouret and Sara,

2004; Clayton et al., 2004). In tasks separating in time the processing of the information

provided by the cue, during a waiting period, and the response period, following a go-signal,

LC neurons responded both to the cue and before the action onset (Jahn et al., 2020; Varazzani

et al., 2015). This pre-action activation could be compatible with an implication of the LC in

the cognitive effort required to trigger costly actions.

This idea is especially reinforced by evidence in these tasks for a dissociation in time of

the action-selection process and action triggering process. Varazzani and colleagues (2015)

and Jahn and colleagues (Jahn et al., 2020) used a force-discounting task in which monkeys

were instructed to fixate their gaze on the location of the apparition of the cue. Fixation breaks

were thus considered as errors and led to the trial restarting. Interestingly, monkeys virtually

never broke fixation after the onset of the go signal, suggesting that the decisions whether to

engage in the trial based on the offered reward and requested force were made during the

waiting period of cue presentation. In that sense, this task probably allowed dissociating the

action selection process, during the waiting and cue presentation period, from the action

83

triggering process, after the go-signal. The two activations of LC neurons could be intervening

in the control exertion for these two steps separately.

(A) Clayton et al., 2004, Figure 2C (B) Clayton et al., 2004, Figure 3A

(C) (D)

Figure 33: In forced-choice discrimination tasks, LC activity was better aligned on response

onset than on stimulus onset. LC neurons were only active if an action was executed, regardless

of reward expectation.

(A) Raster plot (top) and peri-event time histograms (bottom) of the activity of an example LC

unit in the forced-choice discrimination task. LC activity is aligned on response onset (adapted

from Clayton et al., 2004).

(B) Population stimulus-locked peri-event time histograms with trials split between correct

and incorrect lever presses and omissions (no press). Bottom curves represent the density

function of RTs, split between correct and incorrect presses. LC neurons responded prior to the

onset of correct and incorrect responses (lever presses) but not in the case of omissions. Activity

was stronger prior to correct compared to incorrect responses (adapted from Clayton et al.,

2004).

(C) Raster plots (top) and peri-event time histograms (bottom) aligning the activity of an

example LC unit in an odor-discrimination task on either stimulus onset (CS+, left panel) or

response onset (Go resp, right side panel). LC activity is better aligned on response onset than

on stimulus onset (adapted from Bouret and Sara, 2004).

(D) Raster plot of the activity of an example LC unit around feedback onset in the task used by

Bouret and Richmond, 2015. The task consisted of a cue indicating reward size, displayed

Feedback

Active

Passive

84

during a waiting period, followed by either an active trial, in which monkeys had to release a

grip after a go-signal to get the reward, or a passive trial, in which the reward was delivered

after a certain amount of time. The reward was always preceded by a feedback, either when

the action was detected (active trials) or when the pre-defined delay had elapsed (passive

trials). On the plot, trials are split between active (black) and passive (grey). The activity of LC

neurons increased prior to the feedback in active but not in passive trials i.e. increased prior to

action onset.

If tasks conditions differ by their level of reward and/or costs (physical effort), during

the activation period prior to action onset, LC neurons have been shown to encode task

parameters in different directions and magnitudes compared to the modulations observed

around cue onset. For instance, in a reward schedule tasks in which some trials were rewarded

and some were not, an information given by the cue, LC neurons differentiated between

rewarded and unrewarded trials before action onset, with more spikes in the case of

unrewarded trials (Bouret et al., 2012, Figure 34A. This study additionally included the

measurement of the frequency of occurrence of a Pavlovian appetitive behavior, lipping,

known to signal the value of ongoing events (Bouret and Richmond, 2015, 2009). Interestingly,

lipping around action onset was more frequent in rewarded than non-rewarded trials,

indicating that monkeys perceived the action as more valuable, i.e. less costly when it led to a

reward. Consequently, triggering unrewarded actions probably required exerting more

cognitive control, and this was associated to increased LC activity.

Note that this pre-action negative encoding of reward was not seen in several other

studies (Bouret and Richmond, 2015; Varazzani et al., 2015, Figure 34B). It is interesting to

note that one of those studies included lipping behavior recording (Bouret and Richmond,

2015), and lipping probability of occurrence around action was not affected by reward either.

This suggests that the negative encoding of reward by LC neurons may indeed reflect the

subjective cost of triggering the action and therefore possibly the cognitive control invested,

as it is only present when rewards have an influence on behavior. Additionally, in a one-option

force-discounting task, Varazzani and colleagues (2015) reported a positive encoding of the

force level of the engaged action just prior to the action itself (Figure 34C). In the same line as

previous interpretations, this modulation by the requested force could indicate that LC

85

neurons encoded the cost of triggering the action, and possibly the invested cognitive control.

Note that I reanalyzed this data looking for additional evidence of this hypothesis (see

experimental results of study 1, part II.A).

(A) Rewarded Unrewarded (B)

Action Action

(C) Action

Figure 34: LC neurons encode task parameters just prior to action onset: reward negatively and

force positively.

(A) Raster plot of the activity of LC neurons around action onset in reward schedule task, with

trials split between rewarded (left) and unrewarded trials (right side). Blue lines represent the

spike density function. Prior to action onset, LC neurons activate and this activation is stronger

in unrewarded trials (adapted from Bouret and Richmond, 2012).

Firi

ng

rate

(H

z)

Firi

ng

rate

(H

z)

Firi

ng

rate

(H

z)

Firi

ng

rate

(H

z)

Firi

ng

rate

(H

z)

86

(B) Population activity of LC neurons recorded in a force-discounting task. Trials are separated

by required force level (effort 1,2,3) and offered Reward level. Pre-action activity and peri-

action activity are stronger when the action triggered involves a higher force level.

(C) Population activity of LC neurons recorded in a force-discounting task. Trials are separated

by required force level (effort 1,2,3). Pre-action activity and peri-action activity are stronger

when the action triggered involves a higher force level (Varazzani et al., 2015)

It is important to note, however, that the timing of the activity of LC neurons around

action onset is quite close to the action onset itself (roughly 250ms prior to action onset). Such

a timing is probably incompatible with an involvement of the LC in the mobilization of control

for action triggering. Indeed, as mentioned earlier (see part I.B.1.b), LC axons are

unmyelinated with a slow conduction velocity, around 0.9m/s (Aston-Jones et al., 1985).

Consequently, LC inputs would reach areas involved in control exertion within a little below

150ms, i.e. at the very end of the RT. Still, LC activity before action onset could influence the

action itself, for example by improving the gain of the motor system in the case of physical

effort (see part I.C.2.a) for gain effects). Alpha motor fibers, which conduct the action

potentials responsible for skeletal muscle contraction, are myelinated fibers with high

conduction velocity, around 100m/s in primates (Chipman et al., 2012), meaning that the

transmission of the motor message from the motor cortex to the muscle takes less than 20ms.

Pre-action LC activity thus appears to have a timing potentially compatible with influencing

the execution of motor actions resulting from the action triggering process via an influence on

motor cortices. Such reasoning can putatively be applied to actions involving mental effort,

such as triggering the process of making a mental calculation. Additionally, LC neurons may

be reporting to other areas on how much control has been invested in the process of triggering

the action, and consequently be part of the monitoring unit of control, or at least provide

some information to it (see part I.A.2.c). The evidence for this hypothesis is clearly insufficient

at this stage, since the task parameters encoded by LC neurons prior to action onset are only

correlated with the subjective cost of triggering the action.

Altogether, a large body of literature provides some evidence pointing in the direction

of an involvement of the LC in the action triggering process, because LC neurons activate at its

end in a magnitude reflecting the subjective cost of the action being triggered. However, the

87

timing of this activation cannot account for a role of the LC in mobilizing cognitive control for

triggering the costly actions, as it appears to be too late, predicting the end of this process just

before it actually happens and the action begins. In the next part, I review the literature

regarding a potential involvement of LC activity in facilitating action monitoring, and argue

that this may be the function of such pre-action activation.

6. LC activity and action monitoring

Following the triggering of an action, subjects have to monitor its consequences both

in the short and long term (e.g. error or success, reward, punishment, fatigue). In that sense,

a crucial component of action monitoring is error monitoring. Post error slowing is an increase

of RTs following errors, reflecting behavioral adjustments following error detection (Rabbitt,

1979). Murphy and colleagues (2016), showed that the size of the error-evoked pupil response

could predict the magnitude of the post-error slowing. Considering the positive relationship

between LC activity and pupil dilation (see part I.B.2.b), this relationship between pupil

responses and post-error slowing could reflect a role of LC activity in error monitoring and

adjustments in response to the error, such as increased cognitive control exertion and

therefore response slowing (see part I.A.2.f)(1) for the relationship between RT and cognitive

control). Sellaro and colleagues (2015) showed that transcutaneous vagus nerve stimulation

(tVNS), a technique thought to activate the LC (Ariely et al., 2009; Dorr and Debonnel, 2006;

Frangos et al., 2015) enhanced post-error slowing.

Action monitoring and in particular error monitoring have been linked to the activity

of the ACC (e.g. Botvinick et al., 2004; Dehaene et al., 1994; Gehring and Knight, 2000; Luu and

Tucker, 2001; van Veen et al., 2001; van Veen and Carter, 2002). The ACC receives numerous

LC axons (e.g. Schwarz et al., 2015). Selective stimulation of LC to ACC projections increase

glutamatergic synaptic transmission and neuronal excitability in the ACC (Koga et al., 2020).

Moreover, the correlation of activity of single ACC neurons (spike count) may have increased

in case of high signal to noise ratio in LC activity (low baseline and strong responses to events),

and decreased in presence of low signal to noise ratio (Joshi and Gold, 2020, preprint). These

findings tend to indicate that task-related LC activity could be increasing the efficacy of

synaptic transmission in the ACC.

To summarize, LC activity increases, especially when this activity has a high signal-to-

noise ratio, have been associated to facilitated action execution and especially error

88

monitoring. In that frame, the burst of activity exhibited by LC neurons prior to action onset

could serve to enhance action monitoring and facilitate error detection. Additionally, the

pattern of activation of LC neurons in response to fixation breaks (errors by disengagement

from the task), featuring a burst of activity especially strong before re-engagement in the task,

reported by Jahn and colleagues (2020), could constitute a mechanism through which LC

neurons participate in action monitoring and subsequent adjustments.

In this part, I first detailed evidence of the involvement of LC activity in a large range

of non-goal directed costly cognitive processes: arousal, stress and attention. In the context

of goal-directed behavior, moderate LC activity has been linked to performance optimization

in tasks demanding cognitive control exertion, for working memory updating, response

inhibition and task-switching, although high LC activity leads to task disengagement and

exploration (for inverted U shape curve of the relation between LC activity and performance,

see part I.C.2.b). In the specific context of decision-making, LC activity influences action

selection, but that influence is very probably indirect, via alterations of the perception of effort

(mental or physical) when LC activity is artificially modified. Moreover, LC activity seems to

participate in the action-triggering process, and in particular report the difficulty of the action-

triggering process at its end. Additionally, moderate increases in LC activity lead to enhanced

action monitoring, and in particular error monitoring. Taken together, these roles of the LC in

decision-making could indicate a role of the LC in regulating effort allocation and production.

89

C. Mechanisms for the effects of LC activity on target

areas to regulate cognitive processes, and implications

for effort processing

“...adequate characterization of the functional properties of the noradrenergic system

requires that its impact be considered with regard to other neural systems projecting on the

same target neurons.”

Foote and Morrison, 1987

In the previous part, I reviewed evidence towards an involvement of the LC/NA system

in effort processing, and especially in monitoring the control exertion process and facilitating

action execution. In this part, I shall review potential mechanisms: molecular mechanisms via

metabolic adjustments, short-term and longer-term electrophysiological adjustments, i.e.

signal-to-noise ratio enhancement and synaptic plasticity promotion, respectively, and

network adjustments, i.e. connectivity changes across brain regions.

1. LC activity modulates brain metabolism: NA and response

to high metabolic demands

Cognitive control and physical effort are perceived as costly and aversive. The cost of

cognitive control has been hypothesized to be the consequence of the depletion of a specific

resource, which could be metabolic (see part I.A.2.e)(1)). Specifically, Christie and Schrater

(2015) have proposed the possibility that the avoidance of cognitive control exertion could be

an adaptive mechanism aiming at sparing astrocytic glycogen, a storage polymerized form of

glucose. Additionally, Matsui and colleagues (2017) have demonstrated that physical effort

exertion depletes astrocytic glycogen reserves, and that this depletion is critically linked to

fatigue following prolonged physical effort. Together, this literature resonates with the

extensive findings regarding the effect of noradrenergic release on astrocytic glycogenolysis,

i.e. release of lactate from glycogen, thus providing nearby neurons with metabolic substrate

(see Figure 15).

90

NA release in the brain signals high metabolic demand following increased brain

activity (Magistretti and Morrison, 1988). Via extrasynaptic transmission from LC axon

varicosities located close to glial cells throughout the cortex (Fuxe et al., 2012), NA release

modifies brain metabolism by actions on glycogen-containing astrocytes, from early stages of

development throughout adult life (Cohen et al., 1997; Gibbs, 2015; Gibbs et al., 2010; Hertz

et al., 2010; Müller, 2014; O’Dowd et al., 1995; Quach et al., 1978). First, regarding short-term

effects, NA induces lactate release from astrocytes by increasing glycogenolysis (Magistretti,

1988; Quach et al., 1978; Sorg and Magistretti, 1991). This effect is mainly dependent on β

adrenergic receptors although α adrenergic receptors may play some part (Sorg and

Magistretti, 1991). NA stimuli can mobilize more than half of the glycogen reserves, providing

fast and strong energy surges. On the longer time-scale i.e. several hours, NA participates in

the replenishment of glycogen reserves beyond what can be explained by the initial

glycogenolysis, by modulating the synthesis of proteins necessary for glycogen synthesis (Sorg

and Magistretti, 1992), specifically by promoting the transcription of genetic material related

to glycogen synthesis such as PTG (protein targeting glycogen, Allaman et al., 2000). Such

metabolic effects have strong consequences on cognitive process, such as memory

consolidation (Gibbs, 2015; Gibbs et al., 2010).

Overall, such accounts of the role of NA in astrocytic glycogen immediate

transformation into lactate and long-term replenishment supports the idea of an implication

of the LC/NA system in responding to higher brain metabolic demands. These accounts

regarding the role of NA in rendering metabolic substrates (lactate) available offer a potential

mechanism for the hypothesis that LC pre-action activity may participate in action execution

and action consequences monitoring.

2. From LC facilitation of sensory processing and plasticity to

gain theory

a) Facilitation of sensory processing

LC neurons have been shown to respond to unexpected sensory stimuli (Bouret and

Richmond, 2009; Bouret and Sara, 2004; Grant et al., 1988; Vankov et al., 1995, see Figure 20)

and task-relevant cues (Aston-Jones et al., 1994; Aston-Jones and Bloom, 1981; Bouret and

91

Richmond, 2015; Bouret and Sara, 2004, 2002; Jahn et al., 2020; Kalwani et al., 2014; Varazzani

et al., 2015, see Figure 35).

Cognitive control has been theorized to involve modifying the identity of the

information processed and intensity of processing (Shenhav et al., 2017). NA implication in

sensory processing has long been hypothesized based on indirect or in vitro proof (e.g.

Ciombor et al., 1999; Halliday et al., 1994; Heath et al., 2006; Turetsky and Fein, 2002), but

only in the recent years have causal manipulations confirmed this effect and allowed

identifying its mechanism. Such studies used electrical stimulation of the LC or modifications

of NA brain levels with direct application of NA in target areas, or pharmacological agents,

such as atomoxetine (increasing NA) or clonidine (decreasing NA), and simultaneously

recorded neuronal activity in areas of interest (electrophysiological recordings or fMRI).

(A) (B)

Figure 35: LC neurons respond to task-relevant cues.

(A) Raster plot of the activity of one single-unit recorded from the LC in a Pavlovian task,

aligned on the onset of task cue. The line of circles indicates when the red dot requiring ocular

fixation was displayed. Note that for each trial, response consisted in one to three spikes

followed by an inhibition period. (adapted from Bouret and Richmond, 2009)

(B) Activity of LC neurons around cue onset in a force-reward task. Top: raster plot of example

neuron responding to cue onset. Blue line is the spike density plot. Note that for each trial,

response consisted in one to three spikes followed by an inhibition period. Bottom:

92

representation of mean rate per neuron across time (sliding window procedure). Neurons

consistently responded to cue onset yet with variable latencies. One line corresponds to the

activity of one single unit (adapted from Varazzani et al., 2015).

Specifically, NA release in sensory cortices (auditory, visual, olfactory and

somatosensory), and other regions critical for treating sensory information (e.g. the dorsal or

lateral geniculate nucleus or the thalamus), enhances the signal to noise ratio (Ciombor et al.,

1999; Devilbiss et al., 2006; Foote et al., 1975; Hurley et al., 2004; Kossl and Vater, 1989; Segal

and Bloom, 1976; Waterhouse et al., 1998) and gates sensory inputs6 (Ciombor et al., 1999;

Jiang et al., 1996; Waterhouse et al., 2000). Such findings were reviewed by Hurley and

colleagues (2004). Figure 36A illustrates the effect of LC stimulation on the firing pattern of

thalamic neurons.

These effects have been attributed to the enhancement of excitatory responses via α1

receptors associated to an enhancement of inhibitory responses by β adrenergic receptors

(Waterhouse et al., 1988, 1980; Waterhouse and Woodward, 1980). Additionally, NA appears

to enhance the specificity of the coding of sound frequencies in auditory neurons, making the

auditory system more finely tuned (Manunta and Edeline, 2004). In the somatosensory cortex,

stimulating the LC shortly before touch stimuli, in addition to increasing the signal to noise

ratio, enhanced the temporal precision of the evoked response, i.e. the latencies to the

evoked responses were less noisy (Lecas, 2004), as can be seen in Figure 36B. This increase in

spike timing precision could facilitate stimulus encoding (Dan and Poo, 2006; Lestienne, 2001;

Tiesinga et al., 2008).

Overall, NA improves the sensitivity to incoming stimuli and their processing (Devilbiss

et al., 2006; Foote et al., 1975; Gelbard-Sagiv et al., 2018; McCormick, 1989; Navarra et al.,

2008; Rogawski and Aghajanian, 1980; Vazey et al., 2018). LC neurons selectively respond to

salient and task-relevant stimuli, but these responses could be too late to facilitate the

processing of these stimuli. For instance, Lecas and colleagues (2004) had to stimulate the LC

200ms prior to stimulus onset to observe significant effects on stimulus processing in the

somatosensory cortex (Figure 36B). Consequently, in normal conditions, NA would only

facilitate the encoding of stimuli presented after salient or task-relevant stimuli. Therefore,

6 Sensory gating is the process that filters irrelevant or redundant stimuli (Sofuoglu et al., 2014).

93

NA release in response to task cues and events (such as actions) could lead to improvements

in the processing of their consequences and therefore enhance task-associated information

processing. It is interesting to note that pre-action peaks of LC activity recorded in LC neurons

occur around 200ms prior to action onset (Bouret and Sara, 2004; Clayton et al., 2004;

Varazzani et al., 2015), which means that they have an ideal timing for enhancing the

processing of action-related sensory stimuli, critical for action monitoring and adjustments.

Overall, the facilitation of processing by LC activity could account, at least in part, for the

effects of NA on performance in a various range of tasks and especially on action monitoring.

(A) Devilbiss et al., 2006, Figure 3, lines 1&2 (B) Sara, 2009, Figure 4A

Figure 36 : Effects of LC stimulation on activity of simultaneously recorded thalamic neurons

and somatosensory neurons.

(A) The LC was electrically stimulated at different frequencies and activity in the thalamus

following sensory stimulation was recorded. Peri-stimulus time histograms of thalamic activity

(aligned on whiskerpad stimulation) show increased responses to the stimulus and decreased

baseline, i.e. increased signal-to-noise ratio. NB: The superimposed numbers indicate the

maximum response observed (rate at peak) (adapted from Devilbiss et al., 2006).

(B) The LC was stimulated 200ms prior to tactile paw stimulation. Activity in somatosensory

neurons was recorded, and is represented in these rasters aligned on paw stimulation. The top

raster represents the response of the somatosensory neurons in the absence of LC stimulation

and the bottom raster represents the activity of the somatosensory neurons after LC

stimulation. Evoked responses are more reliable and their latencies are less noisy after LC

stimulation (adapted from Sara, 2009).

Although such effects of facilitation of neuronal integration have (to our knowledge)

mainly been demonstrated in sensory cortices and other areas involved in sensory processing,

several studies have pointed towards a more generalized effect on several other brain areas

involved in other categories of functions. For instance, increased inputs from LC to cerebellum,

obtained by electrical stimulation of the LC, caused inhibition of baseline spontaneous activity

of Purkinje cells (Freedman et al., 1977; Hoffer et al., 1973) and promoted synaptic plasticity

94

(Carey and Regehr, 2009). In motor area M1, blocking NA reuptake with atomoxetine

increased neuronal excitability and improved motor performance (Plewnia et al., 2004).

As far as mental effort is concerned, several studies (Li et al., 1999; Wang et al., 2007)

provided some evidence for at least partial signal-to-noise ratio modulation of PFC neuronal

responses in the context of spatial working memory demands. These teams first identified PFC

neurons displaying spatial encoding of cue location7 during delay periods during which this

location had to be retained in working memory, and applied α2 agonist guanfacine or α2

antagonist yohimbine onto them. Guanfacine at moderate doses enhanced and yohimbine

suppressed responses of PFC neurons during the delay period of the task (Figure 37A-C),

suggesting that in this context NA enhances the signal in PFC neurons involved in spatial

working memory. These studies however provide no evidence for a decrease in the noise in

the signal. Note that Wang and colleagues (2007) additionally provide evidence that effects of

α2 receptor stimulation on working memory critically depend on a pathway involving

inhibition of effects of cyclic adenosine monophosphate (cAMP). cAMP opens HCN channels

(hyperpolarization-activated cyclic nucleotide gated cation channels), located on the dendritic

spines of neurons in the superficial layers of the PFC, and the opening of HCN channels shunts

nearby synaptic inputs, therefore weakening their effect. α2 receptors are located near HCN

channels and could probably modulate their opening, increasing the efficacy of the inputs

(Figure 37D).

(A) Wang et al., 2007, Figure 1A (B) Wang et al., 2007, Figure 2E left panel

(C) Wang et al., 2007, Figure 2F left panel (D) Arnsten and Pliszka, 2011, Figure 3

Figure 37: In a spatial memory task, α2 agonist guanfacine enhanced while α2 antagonist

yohimbine decreased spatial-related responses of PFC neurons during memory retain period.

(A) Schematic representation of the task used. Monkeys had to fixate a central dot on the

7 Note that PFC neurons displayed responses with a preferred direction, e.g. some neurons only responded when the cue was presented in a certain direction (angle from central fixation point). When mentioning suppression of enhancement of responses of LC neurons I therefore focus on trials in which the cue whose position had to be retained was presented in the preferred direction of the studied neurons.

95

screen. During fixation, a cue was displayed for a short amount of time. After a delay, monkeys

had to perform a saccade onto the position of the cue (adapted from Wang et al., 2007).

(B,C) Activity of PFC neurons (in the case of the cue being presented in their preferred direction)

in control condition (blue) and after (B) guanfacine or (C) yohimbine local (adapted from Wang

et al., 2007).

(D) Proposed mechanism for the effects of α2 stimulation in the PFC on synaptic efficacy. In

absence of NA, cAMP causes the opening of HCN channels (blue), which weakens the inputs to

the neurons by shunting them. In presence of NA, α2A adrenergic receptors are stimulated,

inhibiting the effects of cAMP and therefore the opening of the HCN channels. NA stimulation

therefore inhibits the shunting of the synaptic inputs and leads to increased synaptic efficacy

(adapted from Arnsten and Pliszka, 2011).

Altogether, these results indicate that the effect of improving synaptic transmission

efficacy, via increasing signal-to-noise ratio, referred to as neuronal gain modulation, could

therefore exist at a larger scale in the brain.

To summarize, LC activity and NA discharge onto many target areas rapidly causes

modifications in synaptic transmission: signal-to-noise ratio, gating, and even filtering of

information. Although these effects have mostly been demonstrated in areas responsible for

sensory processing, they are also present in various other regions, including cognitive control-

related regions such as the PFC. In the next part, I review the major theories providing global

theoretical frames for these effects.

b) Overarching theories of short-term effects of NA on

neuronal activity: brain states and gain modulation

NA brain concentrations and LC discharge have long been known to follow the Yerkes-

Dodson law, initially devised to explain the inverted U shape relationship between arousal or

stress and performance (Yerkes and Dodson, 1908). Due to different affinities of the different

adrenergic receptors present in the brain, Arnsten (2000) has hypothesized that this pattern

is in fact due to the engagement of qualitatively different receptors for quantitatively different

96

concentrations of NA, with moderate levels of NA engaging high-affinity α2 adrenergic

receptors and higher levels engaging α1 and β adrenergic receptors. These receptors have

opposite effects on executive functions with α2 receptors improving and α1 and β receptors

impairing cognitive functions. Note that this is believed to involve differential modulation of

PFC functioning (Robbins and Arnsten, 2009). Aston-Jones and Cohen (2005) attributed this

relationship to a balance between two modes of firing of the LC: tonic and phasic (Figure 38).

(A) Aston Jones and Cohen, 2005, Figure 2 (B) Usher et al., 1999, Figure 1A

Figure 38: Yerkes-Dodson relationship between LC discharge and performance.

(A) Schematic representation of the theory by Aston-Jones and Cohen. At low rates, subjects

tend to be inattentive to the task. At intermediate rates, task engagement and performance

are optimal, and at higher rates, subjects disengage from the task and become distractible

(adapted from Aston Jones and Cohen, 2005).

(B) Activity of an example single-unit of the LC (top plot, rate across time) in a vigilance task

and its relation to task performance (frequency of false alarms i.e. FA rate, bottom plot).

Increased baseline LC activity is associated with increased false alarm rate, i.e. lower

performance and increased distractibility (adapted from Usher et al., 1999).

The tonic mode is described as a high baseline period with low signal-to-noise ratio

(weak responses to salient or task-related events), associated to high distractibility and

disengagement from current tasks. The phasic mode, conversely, corresponds to periods of

low baseline, strong task- or salient stimuli-elicited responses and therefore high signal-to-

noise ratio, and this mode is thought to be associated with optimal performance. Studies

tracking pupil diameter during exploration-exploitation tradeoffs provided some arguments

in favor of this interpretation of the Yerkes-Dodson law (Gilzenrat et al., 2010; Jepma and

Nieuwenhuis, 2011, see part I.B.4). Note that although Aston-Jones and colleagues reported

observing such modes during in vivo recordings of LC single-units in awake behaving animals,

to our knowledge, no other team has succeeded in reproducing their observation, and this

account is now strongly discussed. Consequently, reference to these two modes will be

97

avoided, and replaced by reference to low versus high LC activity or low versus high

noradrenergic tone, associated, respectively, with high versus low signal-to-noise ratio.

Note that electrotonic coupling of LC neurons (Christi and Jelinek, 1993; Christie et al.,

1989; Ishimatsu and Williams, 1996), via electrical synapses (gap junctions), could account for

such differences in signal to noise ratio (Aston-Jones et al., 1999; Usher et al., 1999). Coupling

would average the uncorrelated neuronal noise across the population, leading to increased

signal-to-noise ratio in the LC (decreased baseline firing and increased task-related firing).

However, the existence of the necessary gap junctions between LC neurons in adult humans

and other animals is widely discussed. Indeed, electrotonic coupling is thought to be

dependent on the existence of gap junctions between astrocytes and LC neurons, but accounts

of such existence of gap junctions were made using brains from juvenile individuals (Alvarez-

Maubecin et al., 2000). In adult brains, studies of such coupling provided evidence for weak

coupling at best (Ishimatsu and Williams, 1996; Travagli et al., 1995). Alvarez and colleagues

(2002) have shown ex vivo that, in the adult rat brain, contrary to that of younger specimens,

firing synchrony and coupling between LC units is conditioned by the firing rate, with coupling

only for low rates of firing. This coupling is thought to be dependent on α2 adrenergic

receptors (Christie, 1997; Ennis and Aston-Jones, 1986), with a high NA affinity that could

account for the necessity for low LC rates for coupling in the adult brain. Considering that high

performance is generally associated to low baseline rates of LC neurons, with higher rates

causing disengagement from task and low performance, frequency-dependent coupling of LC

neurons could be one of the mechanisms involved in performance optimization.

Nonetheless, low LC discharge rates are associates to drowsiness, while intermediate

rates (about 2Hz) are usually associated to optimal performance, and high rates (about 3Hz)

with distractibility and poor performance at tasks (Rajkowski et al., 1994). Such observations

have led Berridge and Waterhouse (2003) to formulate the hypothesis according to which LC

discharge, depending on its intensity, modulates cognition by putting the brain into a certain

state, e.g. the waking state. Within this state, the LC is then hypothesized to modulate the

salient information processing, via a combination of modifications of synaptic excitability,

susceptibility to plasticity, etc.

Aston-Jones and Cohen (2005) extended this idea, formulating the adaptive gain

theory, which is, still today, one of the most influential overarching theories for the global

98

function of the LC-NA system. Servan-Schreiber, Printz and Cohen (Servan-Schreiber et al.,

1990) initially developed the theory of cognitive function improvement via neuronal gain

increase, although they did not offer a clear hypothesis for the mechanism underlying such

gain increases8. This team, using the example of the signal-detection system, modelled the

response of individual single units to sensory inputs as a logistic function, called activation

function, relating the strength of the input to the probability of neuronal firing in response to

the input (Figure 39).

Servan-Schreiber et al., 1990, Figure 1B

Figure 39: The activation function, relating the input of a neuronal system (x-axis, Net input)

to the probability of firing in response to the input (y-axis, activation).

The function belongs to the family of logistic functions (sigmoidal shape). If gain (G) increases,

the temperature of the sigmoid curve increases. (adapted from Servan-Schreiber et al., 1990).

At every time point, theoretical single units receive inputs from other neurons,

excitatory (positive) in presence of the target and inhibitory (negative) or null in the absence

of the target. The unit’s net input is the current input minus the mean noise, and the unit fires

to signal the presence of the target only if a certain threshold is reached. Consequently,

increased noise or weaker inputs lead to lesser probabilities of the unit correctly categorizing

the target as present or absent. At the level of a group of single units, the result on behavior

would be target misses (wrong absence categorization) or false alarms (wrong presence

categorization). The temperature of the sigmoid of the activation function of individual units

can be modified by adjusting gain (Figure 39), which, at the level of a chain of units,

consequentially increases detection performance of the target.

Based on the findings regarding synaptic effects of LC activity on target areas, Aston

Jones and Cohen (2005) applied the model from Servan-Schreiber, Printz and Cohen (Servan-

Schreiber et al., 1990) to LC activity and NA effects on the brain. They summarized all effects

8 They only showed that their model fitted satisfyingly methylphenidate (global central nervous system stimulant) induced performance increases in the continuous performance task (series of letters flashed sequentially, subjects have to report when two consecutive letters were identical).

99

of LC discharge under the notion that NA causes increases in gain in target structures. Below

the optimal gain point (maximum of the Yerkes-Dodson inverted U shaped curve), increases

in gain are beneficial for task performance, i.e. increases in LC firing beneficially increase gain

in target areas, resulting in better task performance. In that sense, moderate LC activity

promotes exploitation. Beyond that point (in the downward part of the inverted U curve),

increases in LC activity are no longer beneficial for task performance, as they increase gain so

much that they cause distractibility because the system becomes sensitive to task irrelevant

stimuli. In that sense, high LC activity promotes exploration. Note that such inverted U shape

of LC activity on the signal-to-noise ratio, i.e. gain, of the activity of target areas can be

visualized in the pattern of response of the top thalamic neuron of Figure 36 to increased LC

tone.

Reconciling this account with our hypothesis of an implication of the LC/NA system in

effort processing requires considering it in the context of tasks requiring cognitive control

exertion, which, in fact, only constitutes a particular case of the larger proposed rule. For

instance, in the case of decision-making, optimal neuronal gain could allow better processing

of relevant information while efficiently ignoring irrelevant information. In addition, this

theory offers a theoretical frame and potential mechanism for regrouping accounts of the

implication of the LC/NA system in effort processing and costly cognitive processes in general

with findings regarding an implication in behavioral flexibility.

In the particular case of decision-making, and specifically of two-alternative forced

choices, Shea-Brown and colleagues (2008), generalizing models theorized for target

identification (Gilzenrat et al., 2002; Usher et al., 1999; Usher and Davelaar, 2002), devised a

multilayered neuronal model of decision-making. In brief, they created a two layers model in

which the first layer, a sensory layer, encoded task information, and transferred that

information to a second layer of response accumulators. When sufficient evidence was

accumulated in favor of one option in the accumulator layer, the response was triggered.

Additionally, when the accumulated evidence reached a certain threshold, which was below

the response threshold, gain was adjusted in both layers, i.e. gain was adjusted when nearly

enough evidence had been accumulated, leading to an acceleration of the evidence

accumulation towards the response. They then compared the maximum reward rate

obtainable in the absence of gain modulation compared to reward rate in a model with gain

modulation. If model parameters, including gain, were optimized, reward rate increased by

100

14 to 24% compared to a two-layers model with no gain adjustments. Note that, by

construction, in this model, gain was increased at the very end of the RT, and speeded up the

response, and such a pattern could be compatible with the timing of pre-action LC activations

(see part I.B.5.b).

c) Extending gain theory in time: LC activity and

plasticity promotion

At larger temporal scales than adjustments of signal-to noise ratio of individual

neurons, LC activity has been shown to influence synaptic plasticity. Plasticity is the ‘activity-

dependent modification of strength or efficacy of synaptic transmission of preexisting

synapses’ (Citri and Malenka, 2008), via long-term potentiation and/or depression (LTP and

LTD). Plasticity allows the brain to adjusts its activity and connectivity, and therefore

subsequent behavior, and is particularly critical for memory formation.

In addition to signal-to-noise ratio increasing effects, plasticity-promoting effects of NA

were observed. As indicated by their names, LTD and LTP are long-lasted, and NA effects on

neuronal gain could be prolonged in time by these phenomena. Studies reporting NA-

dependent plasticity were initially performed in sensory networks, visual (Pettigrew, 1982;

Pettigrew and Kasamatsu, 1978) and olfactory (Sullivan et al., 1989) (for exhaustive recent

review, see Perez, 2020). NA was shown to promote plasticity by inducing LTP, in the CA3

region of the hippocampus (Hopkins and Johnston, 1984; Stanton and Sarvey, 1985), as well

as the CA1 and the dentate gyrus (Harley and Sara, 1992; Heginbotham and Dunwiddie, 1991;

Neuman and Harley, 1983; Segal and Bloom, 1974; Stanton and Sarvey, 1985). More recently,

these findings were complemented by studies showing that NA can induce LTD in the

hippocampus via fixation on β adrenergic receptors (Hagena et al., 2016; Lemon et al., 2009;

Straube et al., 2003), and in the visual cortex via fixation on α1 and β adrenergic receptors

(Salgado et al., 2012). The balance between NA favoring LTD or LTP appears to be

concentration-dependent and spike-timing dependent (Lemon et al., 2009; Salgado et al.,

2012). Note that the co-release of DA with NA (see part I.B.1.c) is thought to have a critical

role in plasticity promotion, at least in the hippocampus (Sonneborn and Greene, 2021).

101

In addition, various studies have demonstrated that increasing brain NA levels

naturally (stress exposure), pharmacologically (α2 antagonist or β agonist drugs) or directly

(infusions of NA in target areas) produce robust and fast increases in transcription of genes

and protein synthesis critical for long term plasticity (e.g. Bing et al., 1992, 1991; Gubits et al.,

1989; Maity et al., 2020; Stone et al., 1997, 1991; Walling and Harley, 2004).

Regarding the mechanisms regulating gain and plasticity effects of LC activity, the

GANE model (glutamate amplification of NA) posits that NA concentrations in LC target areas

are modulated locally by glutamate, with high glutamate in targets causing enhancement of

NA release by stimulation of LC synaptic bouton NMDA (N-methyl-D-aspartate) receptors,

thus creating ‘NA hotspots’ (Mather et al., 2016). In the hotspots, NA would facilitate the task-

relevant processes and representations. Around these hotspots, lateral inhibition would be

enhanced, such that irrelevant processes and representations would be inhibited.

Additionally, in the absence of NA hotspots, NA levels would be sufficient only to activate

inhibitory receptors with high affinity. In that sense, the GANE theory can be understood as a

larger scale generalization of the gain theory. While the adaptive gain theory focuses on the

signal-to-noise ratio of individual units, the GANE theory proposes that NA neurotransmission

is also associated to a change in the “signal-to-noise ratio” of larger neuronal networks and

processes, enhancing relevant and inhibiting irrelevant computations and neuronal

connections.

Altogether, these accounts suggest that not only is the effect of NA on synaptic

transmission efficacy probably generalized in space, i.e. throughout the brain, but also in time,

as NA induces long-term circuitry adjustments by promoting synaptic plasticity. In the next

part, I focus on a larger spatial scale, and review evidence for connectivity adjustments and

network rearrangements following NA release.

d) LC activity and brain connectivity and networks

Bouret and Sara (Bouret and Sara, 2005; Sara and Bouret, 2012) proposed an

overarching theory for the role of the LC/NA system on behavior: network reset for behavioral

shifts (Figure 40), later extended by Corbetta and colleagues (2008). In that context,

responding to a cue or making a choice after a deliberation period constitutes a behavioral

102

shift, as well as switching from exploitation to exploration. Thus, the finding that the LC/NA

system is critically involved in set-shifting, and that LC neurons respond to stimuli requiring

behavioral adjustments (cues) and prior to behavioral changes (responses) advocates for such

a role in behavioral shifts. Recent brain connectivity studies have provided evidence for such

brain network reset in response to LC activation (reviewed Corbetta et al., 2008).

Figure 40: Network reset theory (adapted from Bouret and Sara, 2005).

In a given state of a task (e.g. set of task rules), the network serving to solve the task is in a

certain configuration, with certain units (circles) activated (green) and certain silent (grey).

Following a shifting stimulus (e.g. cue or change in rules), LC neurons activate and NA is

released. NA acts on the network and modifies the pattern of response of its units. Afterwards,

a different subpopulation of the network is active.

Complementarily, Yu and Dayan (2005) proposed that NA could serve to signal

unexpected uncertainty, i.e. brutal changes in the environment, for instance task

contingencies, and therefore trigger adjustments to such changes. They opposed this role to

a putative role of acetylcholine in signaling expected uncertainty (e.g. known reward

probabilities in a probabilistic task or standard deviation in the distribution of outcomes). Note

however that following experimental studies only managed to demonstrate a role of NA in

uncertainty signaling in general (Jepma et al., 2018; Jepma and Nieuwenhuis, 2011; Nassar et

al., 2012; Payzan-LeNestour et al., 2013). Nassar and colleagues (2012) specifically linked

expected uncertainty (uncertainty in the expected mean of the distribution of the outcome)

and unexpected uncertainty (probability of a sudden change in the distribution of outcomes)

103

to pupil diameter around outcome onset, and specifically with pupil diameter average and

pupil diameter change, respectively, during this epoch.

Coull and colleagues (1999) provided early accounts of the involvement of LC activity

in network reconfiguration throughout the brain. They showed that, during an attentional

task, α2 agonist (clonidine) administration decreased the functional strength of the

connections between areas critical for the task (from and to the visual cortex and from cortex

to thalamus) while connectivity with the parietal cortex increased. Additionally, stress, which

is associated to LC activation, has long been known to cause extensive network adaptations

throughout the brain, with increased activation in the salience network (Hermans et al., 2011;

van Marle et al., 2010), promoting vigilance and altering cognitive control capacity (Corbetta

et al., 2008; Hermans et al., 2014). Note that this effect of stress on functional connectivity is

blocked by β receptor antagonist (propranolol) administration, and therefore probably

dependent on β receptors.

Recent accounts of the relationship between the LC/NA system and network

reorganization have involved assessing functional connectivity following modifications of the

activity of the LC or the availability of NA. Functional connectivity refers to the temporal

correlation between the BOLD MRI signal voxels or brain areas (Fox and Raichle, 2007). Guedj

and colleagues (2017b) report that atomoxetine administration caused resting state

functional connectivity reorganization, with strong changes relating to the connectivity of the

fronto-parietal attention network to other brain areas. These findings indicate that LC

activation effects on functional connectivity could be critical for attention. This study however

focused on resting state connectivity, i.e. outside of task demands, a gap filled by several other

teams.

Atomoxetine administration was found to cause decreased functional connectivity in

the default-mode network, a functional connectivity pattern observed outside of tasks, with

connectivity negatively correlated to task demands (Brink et al., 2016), suggesting that NA

could critically modify brain connectivity to adjust to task demands. In Parkinson’s disease,

atomoxetine administration was shown to increase resting-state functional connectivity

between the dorso-lateral PFC and the ACC, and the strength of this increase correlated with

the degree of cognitive improvement observed in the patients (Borchert et al., 2016). In the

context of cognitive control exertion, Hernaus and colleagues (2017) demonstrated that,

104

during a working-memory task, atomoxetine administration increased the functional

connectivity between various brain areas. The increase of connectivity between the insula and

the dorsolateral prefrontal cortex was of particular interest as, first, it only appeared in case

of high, but not moderate working memory load, and, second, the magnitude of the increase

was predicted by individual working memory capacity.

Pharmacological modification of NA availability and LC activity are not entirely specific.

Zerbi and colleagues (2019), conversely, very specifically modified LC activity using DREADDs.

They confirmed the previous findings regarding changes in functional connectivity in several

brain connectomes following stress exposure or enhanced NA neurotransmission. These

effects were driven by α1 and β1 and, to a lesser extent, α2 adrenergic receptors. Note that

such a pattern of receptor involvement may have been due to the strong activation of LC

neurons caused by the chemogenetic manipulation, mimicking stress exposure, considering

the much higher affinity of α2 adrenergic receptors compared to α1 and β1 (Arnsten, 2000).

Similar receptors may be involved in putative network reconfiguration to switch from

exploitation to exploration. Conversely, different receptors (possibly mostly α2 adrenergic

receptors) may be involved in the putative connectivity modifications happening in the case

of inhibitory control exertion and in those observed in the context of enhancements of NA

neurotransmission. Indeed, inhibitory control and working memory improvements are

observed for small pharmacological increases in NA brain concentrations, while set-shifting

requires stronger NA increases, which are detrimental for working memory and inhibitory

control (Newman et al., 2008).

These effects of LC activity on large-scale neuronal networks could be the brain-wide

consequence of gain modulation, possibly through coherence of the gamma rhythm9 (Guedj

et al., 2017a) combined to LC modular organization. Depending on the context, different

modules of the LC are activated, and may assemble into functional arrays relaying relevant

information. The output of these neurons would adapt the neuronal gain in different target

structures, to promote adaptive behavior, resulting in coherent gamma rhythm across them,

constituting a synchronous network capable of better information transmission (Fries, 2005;

Nir et al., 2008; Womelsdorf et al., 2007). In that frame, effort, mental or physical, is merely

an example situation in which LC activity could modulate the neuronal and network properties

9 Gamma rhythm is a pattern of neuronal oscillation with a frequency comprised between 30 and 80Hz (Whittington et al., 2011).

105

in order to facilitate task completion. Cognitive control exertion depends on a specific

network, distinct from the salience network (Seeley et al., 2007). One can hypothesize that

specific environmental situations and behavioral states associated to cognitive control

exertion would lead to specific LC ensembles releasing NA in target areas involved in this

network, probably in amounts sensibly lower than those that cause salience network

enhancement. The same is probably true for physical effort.

D. Summary and problematic

First of all, effort, both mental (cognitive control) and physical, is aversive, i.e.

constitutes a decision variable that drives decision-makers away from high-cost options.

Additionally, mobilizing effort is beneficial for task performance, i.e. for goal attainment. For

instance, in the context of decision-making, cognitive control facilitates information

processing and inhibition of response biases or automatic responses. Effort allocation follows

a tradeoff between effort costs and benefits, and decision-makers exert effort in a way that

maximizes the expected value of that effort exertion. Physical and mental effort may rely, at

least partly, on similar depletable resources, as cognitive control exertion decreases the

capacity for physical effort exertion and vice versa. However, the nature of the resources

involved is debated, as the effort allocation system could aim at sparing metabolic substrate

(glucose, glycogen), and/or at minimizing interactions between computations conducted in

parallel, and/or merely at avoiding opportunity costs.

The LC is a heterogeneous noradrenergic brainstem nucleus receiving inputs from and

sending axons to virtually the entire brain. NA increases signal to noise ratio in LC target areas,

increases lactate availability for neuronal metabolism and adjusts brain connectivity to task

demands. The LC/NA system is causally involved in several non-goal directed processes that

lead to increased mental and physical expenses: arousal, stress and attention. In the context

of goal-directed processes, and in particular physical effort, pharmacological decreases of LC

activity have demonstrated a causal relation between LC activity and subjective difficulty

(effort costs), a theoretical computational variable affecting choices and action execution in a

coherent fashion. LC neurons activate and the pupil dilates around the onset and execution of

costly actions. In force tasks, LC activity and pupil diameter correlate positively with the

106

required and exerted force around action execution, although the exact dynamics of encoding

of these parameters and the relative contributions of objective difficulty and exerted force to

LC activity had not been examined in details (up to now). In case of cognitive control exertion,

indirect measures of LC activity (EEG P3 component and pupil diameter) have well-described

positive correlations with mental effort exertion, while pharmacological or optogenetic

modifications of LC activity influence mental effort performance (inhibitory control, task-

switching and working memory). LC neurons respond to the presentation of task-relevant

information, and pupil diameter increases during decisions, negatively predicting the strength

of biases, i.e. the cognitive control allocated to the decision. Finally, LC neurons respond to

errors (at least commitment errors), predicting the probability to reengage, and could

therefore signal inadequate effort exertion and facilitate the required behavioral adjustments.

In coherence with that, non-specific enhancements of LC activity (tVNS) have beneficial effects

on behavioral adjustments to errors (post-error slowing).

Altogether, these results suggest a critical role of the LC/NA system in physical effort

and control processing. Specifically, we hypothesized that the LC may encode effort around

the time of its production, and signal the consequences of the effort, for instance errors, which

are consequences of insufficient effort exertion. Second, based on results of long-term

manipulations of LC activity, we hypothesized that the LC could participate in adjustments of

effort allocation strategies based the costs and consequences of past effort production. In

other words, we hypothesized, first, that the LC has a facilitative role for effort production,

and second, that it is involved in monitoring effort costs and consequences in order to adjust

later effort allocation.

First, we would therefore expect LC neurons to encode effort regardless of the nature

of the challenge at hand, physical or cognitive, i.e. to exhibit a domain-general encoding of

effort. Second, considering the slow conduction velocity of LC axons, we would expect this

online encoding of effort to have delayed consequences on later effort production rather than

on the ongoing processes. Importantly, we would not expect these delayed effort facilitation

effects to be domain-specific, i.e. we would expect LC activity to facilitate later effort

production regardless of the type of effort. Third, we would expect this signaling of effort

production to participate in the adjustment of future effort allocation, i.e. to serve as a

feedback on the cost of pursued actions, for the effort-allocation system to adjust its

decisions.

107

The activity of LC neurons of five rhesus macaques was examined in three tasks

designed to mobilize physical effort and/or cognitive control (and a control task): two one-

option tasks in Study 1, and a two-alternative choice task in Study 2. Classically, physical effort

was operationalized by differential force exertion on a grip. We expected LC neurons to be

more active at the time of action execution when a higher force was executed by the monkeys.

On the other hand, cognitive control was operationalized in two ways. First, cognitive control

is critical for overriding default courses of action (Epstein, 1994; Shenhav et al., 2013; Shiffrin

and Schneider, 1977; Sloman, 1996). We posited that monkeys would have an automatic

tendency to refuse low value trials, and that they would therefore have to exert more

cognitive control to engage in trials consisting in low value offers, and trigger the associated

actions. Under the hypothesis that LC neurons were involved in encoding the cognitive control

exerted and possibly in facilitating its production, we expected their activity at the time of

triggering actions to scale negatively with the tendency of the monkeys to refuse the offer(s).

Second, in the context of multiple alternatives, cognitive control is required to decide

accurately (Epstein, 1994; Shenhav et al., 2013; Sloman, 1996). Under the postulate that LC

activity encodes control exertion, in the two-alternative choice task, we expected LC neurons

to be more active around accurate choices.

Moreover, the delayed effects of LC activity on effort production were also explored in

a two-alternative choice task, as it had several steps (two presses separated by a binary

choice). We expected LC activity at early steps of the task to facilitate effort production (both

cognitive control and physical effort) at later stages of the trials. Finally, in that task, we also

expected LC activity to serve as a feedback signal on effort for structures deciding on future

effort allocation. Practically, we expected LC activity at the time of the action of one trial to

scale with the ongoing effort production, and to predict adjustments in effort sensitivity of

choices in the next trial. In other words, we expected that increased LC activity around the

action of one trial would lead to increased effort sensitivity of choices in the next trial.

108

II. Experimental results

A. Study 1: LC neurons encode difficulty to trigger

actions in one-option forced choice tasks

1. Locus coeruleus neurons encode the subjective difficulty of

triggering and executing actions: evidence for a role in effort

processing (Bornert and Bouret, in prep)

Abstract

The brainstem noradrenergic nucleus locus coeruleus (LC) is thought to be involved in

various costly cognitive processes: arousal, stress and attention. Recent evidence has pointed

towards an implication in effort processing and especially effort production. To assess the

dynamic relation between LC activity, effort and difficulty, we recorded the activity of 193 LC

single-units in five monkeys performing two effort-discounting tasks: a delay-discounting task,

a force-discounting task, and, as a control for the relation of LC activity with low-level sensory-

motor processes, a target-detection task. First, in the effort-discounting tasks, LC neurons

displayed a phasic activation before action onset, modulated by task parameters in directions

opposite to those observed for willingness to work, suggesting an encoding of the subjective

difficulty of triggering the actions, i.e. of the required effort production, scaling negatively with

the automatic tendency to disengage. Additionally, LC neurons encoded reaction times (RT)

over and above task parameters. Importantly, this effect was absent in the target-detection

task in which RT variations across conditions solely reflected sensory-motor differences. RTs,

and therefore pre-action LC activity, could probably be interpreted in terms of subjective

difficulty of triggering the actions and/or in terms of cognitive control produced for that

process in the effort-discounting tasks. Second, in the force-discounting and target-detection

tasks in which the action consisted in producing force on a grip, LC neurons activated during

the execution of the press. The force exerted on the grip was only encoded in the force-

discounting task, in which it did represent an effort, as it affected the probability to engage in

trials. Our data points towards an domain-general online encoding of effort requirements

(subjective difficulty) and possibly effort production by LC neurons. We argue that the timing

109

of these modulations is more likely to influence the consequence of the effortful processes

(action execution for pre-action activity, and monitoring for peri-action activity) than the

processes themselves. Altogether, our work demonstrates the involvement of LC neurons in

effort processing and suggests a mechanism for the effects of NA on effort production and

sensitivity.

INTRODUCTION

The locus coeruleus noradrenergic (LC/NA) system has been shown to be involved in

several cognitive functions: arousal (Aston-Jones, 2005; Carter et al., 2010; Foote et al., 1983),

stress (Buffalari and Grace, 2007; Cassens et al., 1981; Curtis et al., 2012; Koob, 1999; Sved et

al., 2002; Valentino et al., 1991; Van Bockstaele et al., 2001), attention (Bari et al., 2020; Clark

et al., 1987; Coull et al., 2001; Reynaud et al., 2019; Witte and Marrocco, 1997), and decision-

making (Doya, 2002; Nieuwenhuis et al., 2005; Shea-Brown et al., 2008). We recently

proposed a role in effort processing (Borderies et al., 2020; Jahn et al., 2018; Varazzani et al.,

2015).

Effort encompasses multiple notions, but we defined it as a computational variable

representing the theoretical quantity of energy required to face a challenge (Pessiglione et al.,

2018; Shenhav et al., 2017). In that sense, effort constitutes a cost, discounting the value of

rewards (Cousins and Salamone, 1994; Hull, 1943; Phillips et al., 2007; Stevens et al., 2005;

Tsunematsu, 2001; Walton et al., 2006, 2003), but increased effort production is beneficial for

performance. In the particular case of cognitive effort, individuals are thought to discount the

benefits with the costs of the intensification mental activity needed to meet task goals and

inhibit automatic responses (Kool et al., 2010; Kool and Botvinick, 2018, 2014; Shenhav et al.,

2017, 2013; Zipf, 1949). In other words, subjects allocate cognitive control rationally in order

to maximize short- and long-term utility.

In that frame, it remains unclear whether the LC/NA system contributes to effort

processing by participating in decisions about effort allocation based on the estimation of

expected effort costs, referred to here as subjective difficulty, and/or facilitates the execution

of effortful actions. The recent result that pharmacological decreases of LC activity cause

increases in effort-discounting and decreases in physical effort production (despite conserved

motor capacities, Borderies et al., 2020) is compatible with a role of NA in facilitating effort

110

production (mobilization of theoretical resources for action execution), rather than a role in

signaling effort as a decision variable. Indeed, impairing the integration of the effort dimension

of offers would probably induce a decrease rather than an increase in effort sensitivity, as was

observed for reward-based decisions in case of dopamine depletion (Le Bouc et al., 2016;

Martínez-Horta et al., 2014). In coherence with that idea, LC neurons were never found to

encode the cost dimension of offers at the time of decisions, whether the cost is a physical

effort requirement or a delay to the reward represented by a number of presses to execute

(Bouret et al., 2012; Varazzani et al., 2015). Conversely, long term impairments of effort

production would result in increased effort avoidance, comparably to what is observed when

motor capacities are altered after curare administration (Gallagher et al., 2001). Additionally,

we provided evidence that LC neurons encode the exerted force in a large epoch around

action execution in force-discounting tasks (Varazzani et al., 2015), supporting a role in online

effort mobilization for ongoing costly processes.

In the present study, we examined the relation between LC activity and physical effort

and cognitive control production, using single-unit recordings in rhesus macaques. We

hypothesized that the activation of LC neurons would signal the effort produced around the

time of its production, both in the case of cognitive (e.g. inhibiting automatic behavior) or

physical effort (e.g. executing a difficult action). By contrast, the activation of LC neurons

would not be modulated by low-level sensory-motor processes as long as these processes

were not associated with effort. In other words, we expected LC activity to dynamically predict

changes in cognitive and physical effort production (energy expenditure), computed in a sub-

second scale, thus possibly participating in effort production.

We compared LC activity across two one-option tasks manipulating cognitive control

constraints for action-triggering (force-discounting and delay-discounting tasks), and/or

physical effort constraints for action execution (force-discounting task). First, cognitive control

is critical for overriding default responses (Epstein, 1994; Shenhav et al., 2013; Shiffrin and

Schneider, 1977; Sloman, 1996). Our force- and delay-discounting tasks manipulated trial

value such that monkeys exhibited differential engagement rates across conditions. Thus, the

cognitive control allocated when triggering the actions scaled negatively with the tendency to

disengage from the conditions offered. We expected LC activity to reflect these differences in

cognitive control production around the time of action triggering. Second, in the force-

discounting task, differential engagement rates also indicated that physical effort was

111

required to execute the actions. We expected LC neurons to represent the physical effort

produced around the time of action execution. As a control for the relation of LC activity with

mere sensory-motor requirements, we used a target-detection task. Engagement rates did

not vary across conditions, indicating that the cognitive control to trigger the actions and the

physical effort to execute them did not vary across conditions. Therefore, we did not expect

LC neurons to modulate their activity across conditions at the time of action triggering and

execution. Overall, our results were compatible with our predictions.

MATERIALS AND METHODS

Animals

A total of 5 male rhesus macaques (Macaca mulatta) were included in the study; 2 in

the Delay-discounting study (Monkey L, 9kg and Monkey T, 9.5kg), 1 in the Target-detection

task study (Monkey J, 12kg), and 2 in the Force-discounting study (Monkey D, 11kg and

Monkey A, 10kg),. During testing days, they received water as reward, and on non-testing

days, they received amounts of water matching their physiological needs. The experimental

procedures for the Force-discounting task and Target-detection task studies were designed in

association with the veterinarians of the ICM (Brain and Spine institute), approved by the

Regional Ethical Committee for Animal Experiment (CREEA IDF n°3) and performed in

compliance with the European Community Council Directives (86/609/EEC). The experimental

procedures of the Delay-discounting study followed the ILAR Guide for the Care and Use of

Laboratory Animals and were approved by the NIMH Animal Care and Use Committee.

Behavior

During sessions, the monkeys squatted in a primate chair, in front of a computer screen

on which the visual stimuli of the task were displayed. For the Force-discounting task and the

Target-detection task, force grips (M2E Unimecanique, Paris, France, pneumatic for force-

discounting task, electronic for Target-detection task) were mounted on the chair at the level

of the monkey’s hands, 1 for the Force discounting task and 3 for the Target-detection task

(left, right and middle grips). For the Delay-discounting task, a touch sensitive bar was installed

on the chair at the level of the monkey’s hands. The monkeys received water reward from a

tube placed between their lips but away from their teeth.

112

Tasks

Behavioral paradigms were controlled using the REX system (NIH, MD, USA) and

Presentation software (Neurobehavioral systems, Inc, CA, USA) for the Force-discounting and

Delay-discounting tasks, and using the EventIDE software (OkazoLab Ltd, London, UK) for the

Target-detection task.

Delay-discounting task

To examine the link between LC activity and cognitive effort, we designed a task in

which sensory and motor requirements would be virtually equivalent across all options, to

facilitate the comparison of cognitive processes across conditions. Therefore, the task

involved a delay to the reward as a cost. Practically, the task involved releasing a touch

sensitive bar in a given interval of time in order to get a reward of a certain size, after a certain

delay (Fig. 1A). The reward size and delay factors were orthogonalized. There were nine

experimental conditions, corresponding to the combinations of three levels of reward size (1,2

or 4 drops of water) and three levels of delay to reward delivery (400-600ms, 3000-4200ms,

6000-8400ms, Fig. 1B). Within sessions, these nine conditions were randomly distributed with

equal probability of appearance. Each combination of reward size/temporal delay was

associated to its own visual cue, displayed at the beginning of the trial to indicate the Reward-

Delay contingency of the trial. Each trial began when the monkey touched the bar, and the

cue was then displayed for 400ms (Fig. 1A). Then, a red dot appeared, for a random duration

between 1000 and 2000ms, after which it turned green, indicating that the monkey had to

release the bar. If the monkey responded between 200 and 1000ms, the green point turned

blue, as a feedback to indicate correct performance. On correct trials, monkeys had to wait

for the delay indicated by the cue before getting the announced liquid reward. A new trial

then began after a 1000ms inter-trial interval. If the monkey failed to release the bar during

the appearance of the green point (omission), or released the bar too early (anticipation), the

trial was interrupted, all visual stimuli disappeared from the screen, and the same trial started

again.

Force-discounting task

The Force-discounting task was described in detail in Varazzani et al. (2015). In

summary, the task consisted in squeezing a pneumatic grip above a force threshold in order

to get a reward, delivered after each successful force exertion (Fig. 1B). At the beginning of

each trial, a red dot was presented on screen before the apparition of a cue. Subjects were

113

instructed to fixate their gaze on the dot throughout the trial. The cue indicated the amount

of reward at stake in the trial (3 reward levels, 1,2 or 4 drops of water) and the minimum

amount of force to exert in order to get that reward (position of the force threshold, 3 force

levels, Fig. 1C). The two dimensions were orthogonalized, therefore the task had 9

equiprobable conditions, presented in pseudo-random order. After a variable amount of time

(1500±500ms from cue display), the dot turned green (Go-signal) and the monkeys had

1000ms to initiate the action i.e. to start squeezing the pneumatic grip. If the force exerted

reached the minimum threshold instructed by the cue, the dot turned blue, and remained

blue if the force exerted remained above the threshold for 500±100ms. If the force had been

maintained above threshold for the whole period, the reward announced by the cue was

delivered. An error was registered if the monkey did not engage at all (no squeezing), engaged

(started squeezing) before the go-signal, did not press with enough force, did not maintain

the force exerted above threshold for long enough (during the whole duration of the blue

dot), or broke gaze fixation on the dot. Erroneous trials were repeated until completion.

Figure 1: Summary of the Delay-discounting, Force-discounting and Target-detection tasks

114

(A,B) Delay-discounting task.

(A) At the beginning of each trial, a cue appeared, indicating the combination of reward size

and delay. After 500ms, a red point appeared, superimposed on the cue, indicating that the

monkey had to wait. After a random interval of time (1000-1500ms), the red point turned

green, lasting 1500ms. If the monkey released the bar between 200 and 1500ms after the

onset of the green point, the point turned blue, indicating correct performance. In correct trials,

the reward announced by the cue was delivered after a random delay comprised in the interval

of time announced by the cue. If the monkey did not perform any bar release, or performed it

too early or too late, the same trial started again.

(B) The task had nine conditions, a combination of three levels of Delay (400-600; 3000-4200;

6000-8400ms) and three reward sizes. Each condition was associated to a specific cue.

(C,D) Force-discounting task.

(C) At the beginning of each trial, a red dot appeared, and the monkey had to fixate his gaze

onto it. After a random period of time, a cue appeared, indicating the combination of Reward

and Force that was offered. The monkey had maintain fixation on the red dot for the whole

duration of cue presentation (1000-2000ms). The red dot then turned green (Go-signal), and

the monkey had to trigger the response within 1000ms, i.e. press the grip to reach the

instructed force threshold. When the threshold was reached, the dot turned blue and the force

still had to remain above threshold for 400-600ms. If the force was correctly maintained, the

reward predicted by the cue was then delivered.

(D) The task had nine conditions, a combination of three levels of Reward (1,2 or 4 drops of

water) and of three levels of Force. Each combination was associated to a specific cue.

(E,F) Target-detection task

(E) Trials began with the onset of a red dot in one of nine positions on the screen. After a

random delay (800-3000ms), the red dot turned green and the monkey had to press one of the

three available grips within 1000ms. The force threshold was set just above baseline, such that

any force would be sufficient. If he pressed the grip, the dot turned blue and a Reward (5 drops

of water) was delivered. In case of omission (lack of press) or anticipated press (during red

dot), the trial was restarted.

(F) The dots could appear in nine different locations on screen, defined by three potential

horizontal and three potential vertical coordinates. Horizontal coordinates indicated which

grip had to be used: if the dot was on the left side of the screen, the monkey had to press the

115

left side grip, if the dot was on the right side of the screen, the monkey had to press the right

side grip, and if the dot was in the middle column of the screen, the monkey had to press the

middle grip.

Target-detection task

The Target-detection task was designed to control for the modulation of LC activity

across conditions that differed more in terms of sensory-motor features than in terms of value

and/or effort. The tasks consisted in pressing a grip with very little force constraint to get a

fixed reward (Fig. 1E). The force threshold was set just above baseline, such that any attempt

to press was successful. Critically, the force threshold and the size of the reward were

equivalent across all conditions, which only differed by the location of the target on the screen

and the lever to be pressed. A trial began with the presentation of a red dot. The dot could

appear in nine positions (Fig. 1F). The position of the dot was maintained during blocks of trials

of random duration (5 to 50 consecutive trials) before changing location. After a very variable

amount of time (1900±1100ms), the red dot turned green (go-signal) and the monkey had to

squeeze one of the three grips within the next 1000ms. If the dot was displayed in one of the

three positions on the left side of the screen, the monkey had to press the left grip, if the dot

was displayed in one of the three positions on the right side of the screen, the monkey had to

press the right grip, and if the dot appeared in one of the 3 central positions, the monkey had

to press the middle grip (Fig. 1F). If the monkey squeezed the correct grip, the dot turned blue

for 50ms (feedback) and the reward was delivered. An error was registered if the monkey

squeezed any grip before the go-signal, or squeezed the wrong grip during go-signal

presentation, which virtually never happened during recording sessions.

Surgical procedures

Surgical procedures were the same as previously described (Bouret & Richmond, 2009;

Varazzani et al, 2015). Briefly, the approximate location of the LC was identified using 1.5T

MRI. Under general anesthesia, a sterile surgery was performed to place the head fixation post

and the recording chamber, centered stereotaxically over the body of the LC with a ~15° angle

for all monkeys.

116

Electrophysiology

Electrophysiological recordings were performed using Tungsten micro-electrodes

(UEWLEHSM3PNM, FHC, Bowdoin, ME). The electrode was placed using a stereotaxic plastic

grid (Crist Instruments) with holes 1mm apart, inserted through a tungsten guide tube (Crist

Instruments) and lowered using a hydraulic micromanipulator (Narishige). LC neurons were

identified using classical electrophysiological criteria (Grant et al., 1988): a low rate of

spontaneous activity (below 4Hz), broad waveforms (>0.6ms for the initial peak), a modulation

of firing rate across states of vigilance and a characteristic activation-pause response to brief

auditory or tactile stimuli (e.g. hand clap). Finally, we also performed clonidine tests on a series

of representative LC units at locations where LC neurons could reliably be obtained during the

course of the experiments. For this, we injected the α2-receptor agonist clonidine (20μg/kg,

IM), and all identified LC units displayed a reversible decrease in firing rate (often close to

zero, Bouret & Richmond, 2009; Grant et al., 1988). None of the units identified as non-LC cells

(e.g. neurons from the neighboring Mes5 nucleus, which respond to jaw movement) changed

their activity after clonidine injection. 92 neurons were recorded in the Force-discounting task

(n=63 in Monkey D and n=29 in monkey A), 75 in the Delay-discounting task (n=52 in Monkey

T and n=23 in Monkey L), and 26 in the Target-detection task (monkey J).

Data analysis

Analyses of the data from the Force-discounting and Target-detection tasks were

performed using the Matlab software (Matlab 2019a, Mathworks) and analyses of the data

from the Delay-discounting task were performed on the R software (R 3.5.0, R Foundation for

Statistical Computing, 2016).

We only analyzed trials in which the monkeys engaged after the Go-signal, i.e. tried to

perform the action, in order to be able to compute response time (RT). In the Delay-

discounting and the Target-detection tasks, engaging after the Go-signal ensured succeeding

due to the simplicity of the action to be performed. In the Force-discounting task and the

Delay-discounting task, preliminary inspection of the neuronal activity did not reveal

differences across monkeys, so the data were pooled by task.

All the following analyses were performed after z-scoring each parameter (Reward,

Delay, Force category, RT, maximum exerted force) and spike count by neuron.

117

Target-detection task and Force-discounting task studies: response time

determination

The force signal was digitized and sampled at 1kHz. During data acquisition, RTs were

evaluated online and defined as the time when the force signal reached the target threshold.

But to avoid the potential influence of the threshold position, RTs were re-evaluated offline

and defined as the time when the force signal started to diverge from baseline, i.e. the

beginning of the squeeze on the grip. In brief, the mean slope of the force signal was computed

in successive 10ms windows (difference in signal value between the end and the beginning of

the time window). We considered that the animal had started pressing the grip when the slope

was above the maximum slope that could be detected in the baseline period outside of

periods of force exertion, in three consecutive time windows.

Target-detection task and Force-discounting task studies: maximum exerted force

determination

To assess the trial-by-trial maximum exerted force in the two force tasks, we first

subtracted the baseline (mean of the signal in a [-500;0ms] epoch before action onset) from

the force signal. We then looked for the maximum in the force signal in a [0;900ms] epoch

after action onset (based on the RT previously determined), since signal inspection revealed

that the presses lasted between 600 and 900ms.

Behavioral analyses pooling sessions by monkey

To analyze the behavior (RT and engagement) per monkey per task, we pooled the

data from all the sessions for each monkey. In order to conserve across-session variations in

RT, RTs were not z-scored by session before pooling the data of all the sessions together.

However, results were similar when z-scoring RTs per session before pooling the sessions. We

then fit a generalized linear model (GLM) per monkey on the pooled data for each combination

of regressors.

Delay-discounting and Force-discounting tasks: Sliding window procedures for

effects of task parameters and behavioral indicators

Spikes were counted in a 200ms window of time, which were moved forward in 25ms

steps, in a [-800;800ms] epoch around action onset. In order to quantify the effects of cost

118

(Force or Delay), Reward and RT around action onset on spike count, and qualify their

dynamics, we then performed, for each 200ms epoch of the sliding window counting of spikes,

a neuron-by-neuron GLM for spike count with trial-by-trial Reward, cost (Force or Delay) and

RT as parameters.

Delay-discounting and Force-discounting tasks: model comparison for pre-action

spike count

To justify including RT in the GLM explaining the pre-action spike count of LC neurons,

we performed a model comparison procedure. We pooled all the trials from all the neurons,

per task (n=75 for Delay-discounting task, n=92 for Force-discounting task). To take into

account differences in baseline firing rate, we did not z-score the spike count per neuron

before pooling the trials of the different neurons (but results were similar when z-scoring rate

per neuron before pooling the neurons). We also did not z-score RTs per sessions to conserve

across-sessions variations in RT, but similar results were obtained when z-scoring RTs per

session before pooling the sessions. After pooling the sessions, we fit three different GLMs to

the spike count of the neurons (using the fitglm function in MATLAB and the bic.glm function

in R): one with only RT as regressor, one with only task parameters as regressors, and one with

all task parameters and RT. We used Bayesian Information Criteria (BICs) to compare the fit of

the models, but similar results were obtained using Akaike’s information criteria (AIC).

RESULTS

In this study, we tested the hypothesis that the activity of LC neurons around the

triggering of the action was related to effort production, i.e. to the dynamic mobilization of

resources necessary to face a challenge. In that frame, the firing of LC neurons should be

related to both cognitive and physical effort associated with producing an action, rather than

the sensory-motor properties of the action. To test this hypothesis, we compared LC activity

around actions involving cognitive effort, physical effort or both, and examined its dynamic

relation with these two types of effort around action onset. We will first present behavioral

data that provide information about effort production across the three tasks, and then

describe the dynamic modulation of LC activity across these 3 tasks.

119

Behavior

We compared behavior in the three tasks, all involving a manual response to a visual

stimulus (Go-signal, a red point turning green). In each of these tasks, monkeys completed

multiple trials across which actions varied in terms of sensory-motor requirements and/or in

terms of reward contingencies. At the beginning of each trial, monkeys received information

about the current task condition using a specific visual cue and had to adjust their behavior

accordingly. We measured two behavioral responses: engagement (whether they attempted

to perform the trial, or not) and Response Time (RT, the interval between the Go-signal and

action onset). Indeed, even if trials were repeated until correct completion, monkeys could

decide to perform the trial or not. Thus, in each condition, the proportion of trials in which

monkeys engaged (engagement rate) reflected their average willingness to work (WTW) given

the expected sensory-motor constraints and reward contingencies. Critically, since trials were

repeated until correct completion, there was no instrumental interest in refusing to perform

any trial. In such task, the optimal behavior would be to perform all trials regardless of the

associated costs and benefits, because refusing one only increased the delay until reward

delivery. But still, monkeys failed to perform the task more often in trials associated with

higher cost and/or lower rewards, which indicates that they could not repress a natural

tendency to disengage in such conditions, even if in that situation if was counterproductive.

Thus, we assumed that in task conditions associated with lower willingness to work, engaging

in the task and performing the action required a higher level of cognitive control to overcome

the stronger tendency to disengage, compared to conditions in which average willingness to

work was higher. Thus, in each task, we used the contrast in WTW across conditions to

evaluate the influence of these conditions on cognitive difficulty, and potentially on the

amount of cognitive effort mobilized to overcome that difficulty. Since all these tasks involved

triggering a response to a visual target, we could also measure RT, i.e. how quickly the animal

responded to the stimulus. In line with previous studies, we assumed that RTs could be

affected both by sensory effects (how difficult it was to identify the target stimulus), motor

effects (how difficult it was to execute the action), and cognitive effects, namely the amount

of cognitive control required to trigger the action in the current condition (here, as a function

of the natural tendency to disengage from the task in the current condition) (Dmochowski and

Norcia, 2015). Critically, the increase in RT in conditions associated with greater cognitive

control could reflect both the difficulty itself and/or the mobilization of resources (i.e. time)

120

in order to overcome that difficulty (Lee and Daunizeau, 2021; Navon, 1984; Shenhav et al.,

2017). To capture the potential influence of sensory-motor and cognitive effects on RT, we

compared behavior across three tasks manipulating both sensory-motor and reward

parameters across conditions (Fig. 1). The different features of the three tasks are summarized

in Table 1, together with the predicted influence of sensory-motor and cognitive effects on

RT.

Table 1: summary of the characteristics of the three tasks used in this study

Delay-

Discounting

Force-

Discounting

Target-

Detection

Target Detection ✓ ✓ ✓

Var. Position (target & response) 0 0 ✓

Force Production 0 ✓ ✓

Force Constraint 0 ✓ 0

Reward Delay ✓ 0 0

Reward Size ✓ ✓ 0

Sensory motor constraints: 0 ✓ ✓

Cognitive constraints ✓ ✓ 0

In short, all three tasks involved detecting a simple visual target (a red dot turning

green). In the Delay-discounting and in the Force-discounting task, the target stimulus was

always presented in the middle of the screen and only one responding device was available.

In the Target Detection task, the target stimulus could be presented in one of nine possible

positions on the screen and there were three response devices (left, middle and right). In the

Delay-discounting task, the action was a simple bar release whereas in Force-discounting and

Target-detection task, monkeys had to squeeze a grip, and exert a given level of force. In the

Force-discounting task, the level of force necessary to complete the trial was varied

121

systematically across trials according to three difficulty levels, whereas in the Target-detection

task, the level of force required was set to a minimum and equivalent across all task

conditions. Finally, we also manipulated Reward parameters: In both the Delay-discounting

and Force-discounting tasks, we systematically varied the size of the reward (volume of juice)

according to three levels. In the Delay-discounting task, we also systematically varied the delay

between correct responses and reward delivery, according to three levels.

Based on these features, we predicted that, across task conditions, WTW should differ

only in Delay-discounting and Force-discounting tasks, but not in the Target-detection task.

Accordingly, RTs should be affected only by cognitive effects in the Delay-discounting task. In

the Target-detection task, however, RTs should only be affected by sensory-motor effects.

Finally, in the Force-Discounting task RTs should display both sensory-motor and cognitive

effects. To validate these predictions, we compared engagement rates and RT modulations

across conditions in the three tasks.

In the Delay-discounting task, there were clear differences in WTW across conditions:

engagement rates showed a significant positive modulation by Reward and a significant

negative modulation by Delay in both monkeys. For each monkey, we fit a logistic regression

for WTW with Reward and Delay as parameters. The Reward effect was significantly positive

for both monkeys (Monkey T: beta=0.40; p<10-11; Monkey L: beta=0.23; p<10-4) and the

Delay effect was significantly negative for both monkeys (Monkey T: beta=-0.42; p<10-14;

Monkey L: beta=-0.31; p<10-8, Fig. 2A&B). Since in this task sensory-motor constraints were

equivalent across conditions, we expected task conditions to affect RTs through cognitive

effects, and thus according to their influence on WTW. Indeed, in both monkeys, task

parameters had significant opposite effects on RT, as measured using linear modeling (GLM,

Fig. 2C&D). Reward had a negative effect on RT (Monkey T: p<10-34; Monkey L: p<10-33) and

Delay had a positive effect (Monkey T: p<10-95; Monkey L: p<10-52). Thus, task parameters

clearly affected WTW and consequently the cognitive difficulty associated with engaging in

the task and triggering the action. The corresponding differences in RTs across conditions

confirmed this interpretation in terms of cognitive difficulty to trigger the actions, but also in

terms of resources (time, at least) invested in order to overcome the difficulty and perform

the action.

122

Figure 2: Behavior in the Delay-discounting task, the Target-detection task and the Force-

discounting task.

123

(A-D) Behavior in the Delay-discounting task.

(A,B) Coefficients of the logistic regression for engagement in trials (0 if the monkey made no

action, 1 if he performed an action) with task parameters (Reward and Delay) as regressors,

by monkey. For both monkeys, Reward had a positive effect and Delay had a negative effect

on the probability to engage in trials.

(C,D) Coefficients for the GLM for RT with task parameters (Reward and Delay) as regressors,

by monkey. For both monkeys, Reward had a negative effect and Delay had a positive effect.

(E-H) Behavior in the Target-detection task.

(E) Mean engagement rate across sessions by dot position on screen. There was no difference

in engagement rate across the nine conditions (ANOVA).

(F) Mean RT across sessions by dot position on screen. A two way-ANOVA with vertical and

horizontal dot coordinates showed that RTs were significantly shorter for middle grip presses.

(G) Time course of the exerted force on each grip across sessions after action onset. Thick lines

represent the mean exerted force and the thinner lines represent one the standard error of the

mean (SEM) above and below the mean. The maximum exerted force was higher for middle

grip presses (blue) than for right grip (green) or left grip (red) presses.

(H) Coefficients for the GLM for RT with only the maximum exerted force as parameter. RT was

longer if the maximum exerted force would be stronger.

(I-N) Behavior in the Force-discounting task.

(I,J) Coefficients of the logistic regression for engagement in trials (0 if the monkey made no

action, 1 if he performed an action) with task parameters (Reward and Force) as regressors,

by monkey. For both monkeys, Reward had a positive effect and Force had a negative effect

on the probability to engage in trials.

(K,L) Coefficients for the GLM for RT with task parameters (Reward and Force) as regressors,

by monkey. For both monkeys, Reward had a positive effect (only a tendency for monkey D)

and Force had a negative effect.

(M,N) Coefficients for the GLM for RT with only the maximum exerted force as parameter. For

both monkeys, there was a strong negative relationship between RT and the maximum exerted

force.

**: p<0.01; ***: p<0.0001; n.s.: non-significant, error bars represent standard error of the

mean (SEM); RT: response time

124

In the Target Detection task, sensory-motor constraints did vary across task conditions

(positions of targets and of response levers) but they did not affect WTW (one-way ANOVA on

session-by-session rates of engagement in each condition, dot position as parameter, p>>0.05,

Fig. 2E). By contrast, these sensory-motor constraints did affect RTs: we fit a two-way ANOVA

for RT with the horizontal position of the dot (i.e. position of the grip to be used: left middle,

right), and vertical position of the dot as parameters (low, intermediate, high). The effect of

the grip used was significant (F(2)=27.06; p<10-4, Fig. 2F), with lower RTs when using the

middle grip (post-hoc t test with correction for multiple comparison, p<0.05), but no effect of

the vertical position of the dot (p>>0.05). These constraints were also associated with

differences in terms of force production: we fitted an ANOVA for the exerted force with the

side of the grip as a parameter. The effect of side was significant (F(2)=858.6 ; p<10-20) with

the force applied on the middle grip being the highest (multiple comparison of means, p<0.01,

Fig. 2G). We finally confirmed the link between RT and exerted force by looking at the

relationship between RT and the peak of exerted force by fitting RT and exerted force with a

linear model. The effect of the exerted force on RT was significant and negative (p<10-14, Fig.

2H). In summary, in the Target Detection task, the absence of WTW contrast across conditions

indicated that the amount of cognitive control required to trigger the action (press) was

probably equivalent across conditions. Additionally, RTs were clearly modulated by sensory-

motor constraints but there is no reason to interpret differences in RT in terms of cognitive

difficulty and/or associated cognitive effort.

In the Force-discounting task, as described in the original paper (Varazzani et al., 2015),

WTW was strongly modulated across conditions. Using a logistic regression, we found that

engagement was modulated positively by Reward and negatively by Force category (p<0.01

for both parameters for both monkeys, Fig.2I&J). Thus, as in the Delay-discounting task, the

amount of cognitive control required to perform the task should be modulated across task

conditions, and RTs should be influenced by cognitive effects. We then examined the

modulations of RTs across task conditions by fitting a GLM for RTs, with the offered Reward

and the category of Force requested as parameters. In both monkeys, RTs were positively

modulated by Reward (Monkey A: beta=0.58; p<10-4; Monkey D: beta=0.00076; p=0.47) and

negatively modulated by Force category (Monkey A: beta=-0.13; p<10-20; Monkey D: beta=-

0.087, p<10-15), which is the opposite of the pattern expected for pure cognitive effects, given

the influence of task parameters on WTW (Fig.2K&L). But these effects of task parameters

125

could also be accounted for in terms of simple motor constraints, rather than cognitive

constraints. To explore that possibility, we examined the relationship between trial-by-trial

RTs and exerted force (maximum exerted force on the grip during press). As was the case in

the Target-detection task, there was a significant negative relation between RT and the

amount of exerted force (GLM, p<0.01 for both monkeys, Fig. 2M&N). Thus, in this task, RTs

were clearly affected by sensory-motor constraints. However, even if the pattern of WTW

suggests that they might also be affected by cognitive constraints, we could not find evidence

for it.

In short, even if all three tasks involved triggering an action in response to a visual

target, the three tasks differed in the relative weight of sensory-motor versus cognitive

constraints on RT. In the Delay-Discounting task, responses were mostly influenced by

cognitive constraints, with virtually no difference in sensory-motor processes across

conditions. By contrast, in the Target-Detection task, conditions clearly differed in terms of

sensory-motor constraint but not in terms of value. In the Force Discounting task, the

difference in behavior across conditions suggests that it involved both sensory-motor and

cognitive constraints, but the relative weight of the two remains difficult to evaluate.

Neurophysiology:

We recorded single LC units based on previously described methodologies (see

materials and methods for details). In the Delay-discounting task, we recorded the activity of

75 single units from the LC (n=52 in Monkey T and n=23 in Monkey L). In the Force-discounting

task, we recorded 92 LC units (in 2 monkeys, n=63 in Monkey D and n=29 in monkey A). In the

Target-detection task, we recorded 26 neurons (in one monkey, Monkey J). The baseline firing

rate of all the recorded neurons remained relatively constant during the course of the

experiments, besides the periods when monkeys were apparently asleep (eyes closed and

motionless) and LC neurons were virtually silent. Within tasks, inspection of the data did not

indicate any difference between neurons recorded from the different monkeys, so the

neurons were pooled by task.

Evoked responses to action onset and action execution

As classically described in previous studies, LC neurons were activated just before

action onset (Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004; Rajkowski et al.,

126

2004). Neuron-by-neuron, we measured firing rates in a [-250;0ms] epoch before action onset

and also in a baseline period, just before the onset of the Go-signal ([-250;0ms] from Go-

signal). We compared these rates using paired t-tests and then used a second-order test (t-

test on T values) to assess the coherence of the changes in rate. Rate significantly differed

from baseline for 62/75 neurons in the Delay-discounting task (all increased, Fig. 3A), 51/92

neurons in the Force-discounting task (all increased, Fig. 3B); and 16/26 neurons in the Target-

detection task (8 decreased and 8 increased, Fig. 3C). At the population level, the activation

only reached significance in the Delay- and Force-discounting tasks (second level analysis:

Delay-discounting: (t(74)=9.56; p<0.001, Fig. 3D; Force-discounting: t(91)=8.88; p<10-13, Fig.

3E). In the Target-detection task, the change in rate did not reach significance at the

population level (t(25)=0.57; p=0.58, Fig. 3F).

In addition to the activation right before action onset, LC neurons were also activated

during the action itself in both tasks involving force production (Force-discounting and Target-

detection). In the Force-discounting task, in the [0;600ms] epoch after action onset,

corresponding to the time during which the force was exerted (Fig. S1), 50/92 neurons showed

a significant change in rate compared to pre-go signal rate (10 decreased and 40 increased,

Fig. 3B) and the activation was clearly significant at the population level (t(91)=15.29; p<10-26,

Fig. 3E). In the Target-detection task, in the [0;500ms] after action onset during which force

was exerted (Fig. 2G), 20/26 neurons showed a significant change in rate during force

production (6 decreased and 14 increased) and the activation was significant at the population

level (t(25)=7.38; p<10-7, Fig. 3F). On the contrary, rate decreased after action onset in the

Delay-discounting task, compared to the baseline rate. The change was significant for 36/75

neurons (all decreased, t(74)=-2.14; p=0.035; Fig. 3A&D). In that task, the action was much

shorter and less demanding (a small hand movement).

According to our hypothesis that LC neurons encode instantaneous effort production,

the activation of LC neurons before action should reflect the amount of cognitive effort

associated with the decision to produce the action given the natural tendency to engage or

not in the trial. By contrast, the activation of LC neurons during force production should reflect

the amount of physical effort involved in the action. To test this hypothesis, we compared

firing rate modulation around action onset across conditions and across the three tasks.

127

Figure 3: Evoked responses to action onset in the Delay-discounting, Force-discounting and

Target-detection task.

(A-C) Mean rate (colors) of each of the recorded units (y-axis) around the time of action onset

(x-axis) in (A) the Delay-discounting task, (B) the Force-discounting task, and (C) the signal-

detection task. Clearer colors represent higher rates.

(D-F) Results of the first-order and second-order t-tests on rate in (D) the Delay-discounting

task, (E) the Force-discounting task and (F) the Target-detection task. Density function of the

T-values of the neuron-by-neuron t-tests comparing rate before action onset ([-250;0ms] from

action onset, yellow), during action execution ([0;600ms] from action onset, dark red) or after

action onset ([0;250ms] after action onset, green) to baseline rate (rate before go-signal, [-

250;0ms]).

(D) In the Delay-discounting task, across the population, rate increased before action onset

and decreased after action onset.

(E) In the Force-discounting task, across the population, rate was above baseline before action

onset and during action execution.

128

(F) In the Target-detection task, across the population, rate was not different from baseline

before action onset and was higher than baseline during action execution.

*: p<0.05; ***: p<0.001; n.s.: non-significant.

Modulation of action-onset related activity by cognitive and physical effort

To evaluate the modulation of LC activity associated with the triggering of the action

based on the associated cognitive difficulty and effort, we examined the modulation of firing

rates just before action onset across task conditions. We also examined the influence of RTs,

which, in absence of sensory-motor constraints, capture trial-by-trial modulations of cognitive

difficulty and/or the associated cognitive effort to trigger the response (see above, ‘Behavior’).

To assess the potential relation between LC and physical effort, we computed spike

counts in a [0;600ms] epoch after action onset, based on force profiles in both tasks. We

compared LC activity across conditions both in the Force-discounting task (where the force

was imposed and represented an effort, as measured using WTW) and in the Target-detection

task (where we could not find any evidence of cognitive modulation since WTW was

equivalent across conditions). We also examined the relation between firing rates and force

production, with the hypothesis that the amount of force produced only reflected effort in the

Force-discounting task, but not in the Target-detection task.

LC and cognitive effort: Action triggering-related activity

Delay-discounting task

To examine the influence of task parameters on action triggering-related activity in the

Delay-discounting task in each neuron, we fit spike counts right before action onset with a

GLM using Reward and Delay as parameters. The activity of an example unit encoding task

parameters is shown in Figure 4A&B. 9/75 LC units significantly encoded Reward, negatively.

Even if that number was close to the number of neurons expected by chance (n=10) given a

population of that size (n=75), the negative effect of Reward was significant across the

population (second order t-test: t(74)=-4.18; p<0.001, Fig. 4C). There was also a significant

positive effect of Delay for 20/75 neurons, and this positive effect of Delay was consistent

across the population (t(75)=5.44; p<0.001, Fig. 4C). Thus, when monkeys triggered the action,

the direction of the effects of task parameters on LC activity was opposite to what we reported

for WTW (Fig. 2I&J).

129

Next, we examined the influence of RT on action triggering-related activity by fitting a

GLM. The activity of an example unit encoding RT positively is shown in Figure 4D. The firing

of 12/75 units exhibited a significant effect of RT (positive for 11 of them) and the positive

influence of RT was significant across the population (second-order t test: t(74)=4.18; p<0.001,

Fig. 4E). When task parameters were included as covariates of RT in the regression model,

8/75 neurons still exhibited a significant positive effect of RT. Critically, the positive influence

of RT was still significant at the population level (t(74)=2.21; p=0.024, Fig. 4F). Note that both

Reward and Delay still had significant and opposite effects on LC activity (Reward, significant

for 8/75 neurons, 7 negative and 1 positive, second order test: t(74)=-3.74, p=0.0003 ; Delay:

significant for 17/75 neurons, all positive, second order test: t(74)=5.05, p<10-5).

Finally, we compared the fit of the GLMs for spike count including RT alone, task

parameters and a full model with both RT and task parameters. We fit each of these three

GLMs on the entire dataset (all trials of all sessions of both monkeys, spike count not

normalized by neuron) and compared the Bayesian Information Criteria (BIC) of those three

models. The BIC of the model with Reward, Delay and RT was lowest (BIC=35742), compared

to the BIC of the model with only RT (BIC= 35874) and to the BIC with only task parameters

(BIC=35760). Thus, modulation of LC activity across conditions could be accounted for by both

task parameters, which are related to the difficulty to make the decision (to perform that

action), and to the RT, which increases with the difficulty to trigger the action and reflects the

amount of resources (at least time) invested in triggering the action.

Force-Discounting task

We assessed the encoding of task parameters by LC neurons using a GLM for spike

count before action onset with Reward and Force category as parameters. The activity of

example LC units encoding task parameters is shown in Figure 4G&H. The effect of Reward

was only significant for 6/92 neurons (1 positive, 5 negative), which is less than the number of

neurons expected by chance (n=13) for a sample of 92 neurons. At the population level,

however, the effect of Reward was significantly negative (second order t-test: t(91)=-2.05;

p=0.04, Fig. 4I). Similarly, the effect of Force was only significant for 7/92 neurons (1 negative,

6 positive) but it was significantly positive at the level of the population (t(91)=2.72; p=0.0079,

Fig. 4I). Thus, the influence of task parameters on LC activity at time of action triggering

mirrors their influence on WTW.

130

131

Figure 4: Encoding of proxies for cognitive effort to trigger the action and physical effort to

execute it, before action onset in (A-F) the Delay-discounting task, (G-M) the Force-discounting

task and (N) the Target-detection task.

(A,B) Raster plots of the activity of two representative LC units encoding task parameters

(Delay, (A) and Reward, (B)) recorded in the Delay-discounting task, around action onset

(orange vertical line). Green dots represent the go-signal. Trials are split by (A) Delay levels

(D1,2,3) or (B) Reward size (R1,2,3). Unit (A) encoded Delay positively before action onset, i.e.

was more active before action onset in high-Delay conditions. Unit (B) encoded Reward

negatively, i.e. was less active before action onset in high-Reward conditions.

(C) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike count

before action onset with Reward and Delay as parameters, in the Delay-discounting task.

Across the population, neurons encoded Reward negatively and Delay positively.

(D) Raster plot of the activity of a representative LC unit encoding RT positively around action

onset (orange line) in the Delay-discounting task. This unit was more active when RT (interval

between Go-signal, green dot, and action onset) was longer.

(E,F) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike

count before action onset with (E) RT alone or (F) RT, Reward and Delay as parameters in the

Delay-discounting task. (E) Across the population, neurons encoded RT positively, (F) and this

positive encoding was true over and above the encoding of Reward (negatively) and Delay

(positively).

(G-H) Raster plots of the activity of two representative LC units encoding task parameters

(Force category , (G) and Reward, (H)) recorded in the Force-discounting task, around action

onset (orange vertical line). Green dots represent the go-signal. Trials are split by (G) Force

levels (F1,2,3) or (H) Reward size (R1,2,3). Unit (G) encoded Force positively before action

onset, i.e. was more active before action onset in high-Force conditions. Unit (H) encoded

Reward negatively, i.e. was less active before action onset in high-Reward conditions.

(I) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike count

before action onset with Reward and Force category as parameters, in the Force-discounting

task. Across the population, neurons encoded Reward negatively and Force category positively.

(J) Raster plot of the activity of a representative LC unit encoding RT positively around action

onset (orange line) in the Force-discounting task. This unit was more active when RT (interval

between Go-signal, green dot, and action onset) was longer.

132

(K,L) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike

count before action onset with (K) RT alone or (L) RT, Reward and Force category as

parameters, in the Force-discounting task. (K) Across the population, neurons encoded RT

positively, (L) and this positive encoding was true over and above the encoding of Reward

(negatively) and Force category (positively).

(M,N) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike

count before action onset with (M) the maximum exerted force on the grip alone or (N) the

maximum exerted force on the grip, Reward and Force category as parameters, in the Force-

discounting task. (M) Across the population, neurons marginally encoded the maximum

exerted force, (N) but this effect was not present over and above effects of task parameters

(Force category and Reward).

(O) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike count

before action onset, with RT as parameter, in the Target-detection task. Across the population,

neurons did not encode RT.

*: p<0.05; **: p<0.01; ***: p<0.001; n.s.: non-significant; D: Delay, F: Force, R: Reward; RT:

response time.

We examined the encoding of RT by LC neurons by fitting a GLM for spike count with

RT alone (in addition to the constant term). The encoding of RT by an example unit is shown

in Figure 4J. The effect of RT was only significant for 10/92 neurons (4 negative, 6 positive) but

it was consistently positive at the level of the population (t(91)=2.01; p=0.048, Fig. 4J). To

evaluate the possibility that the relation between spike counts and RT was confounded by a

joint relation with task parameters, we added task parameters (Reward and Force category)

as co-regressors, along with RT. The effect of RT was still significant for the same 10/92

neurons (7 positive and 3 negative) and it remained consistently positive across the population

(t(91)=2.38; p=0.020, Fig. 4K). Thus, LC neurons encoded RT just prior action onset, over and

above task parameters. Note that the effect of Force category was still significant for 6/92

neurons (5 positive, 1 negative) and was consistently positive across the population

(t(91)=2.86; p=0.0052, Fig. 4K). The effect of Reward was still significant for 5/92 neurons (4

negative, 1 positive) and second order statistics confirmed that it remained significant at the

population level, with a significant negative effect (t(91)=-1.88; p=0.048, Fig. 4K).

133

We next examined the relation between LC activity before action onset and the

amount of force exerted during action itself, using a GLM with only the maximum exerted

force as parameter (besides the constant term). The effect was significant for 14/92 neurons

(2 negative, 12 positive) and marginally significant at the population level (second order t-test:

t(91)=1.68; p=0.096, Fig. 4L). When Force category and Reward were included as co-regressors

in the model, the influence of the exerted force on firing rate remained significant for 10/92

neurons (5 positive, 5 negative), but it showed no significant effect at the population level

(second-order t-test: p=0.95, Fig. 4M). Thus, the influence of the amount of force exerted on

the grip during the action on the activity of LC neurons just before action onset was negligible,

compared to that of RT.

Finally, we compared the fit of the GLMs for spike count including RT, the exerted

force, task parameters and combinations of those. To run this model comparison on the entire

population of neurons, as we did for the Delay-discounting task, we pooled all the trials of all

the sessions into a single database. Then we fit five GLMs on the data: one with only RT as

parameter, one with Reward and Force category as parameters, and one with RT, Force

category and Reward, one with the maximum exerted force alone, and one with Force

category, Reward and the maximum exerted force. The BIC of the model with RT alone was

lowest (BIC=54960), compared to the BIC of the model with only task parameters (BIC=54972),

with RT and task parameters (BIC= 54969), with only the peak of force (BIC= 54976), and with

the peak of force and task parameters (BIC=54983). Thus, the activity of LC neurons before

action onset in this task was more strongly influenced by RT than by the amount of force about

to be exerted on the grip.

Target-detection task

In the Delay-discounting and Force-discounting tasks, we found a positive encoding of

RT prior to action onset. In both of these tasks, behavioral analyses indicates that triggering

the action involved distinct levels of cognitive difficulty (and potentially cognitive effort) and

trial-by-trial RT could be taken as a proxy for subjective difficulty and corresponding effort.

But since RT modulations across conditions also included sensory-motor constraints,

especially in the Force-discounting task, we examined LC activity in the Target-detection task

as a control for these effects. Indeed, in the Target-detection task, we could not detect any

evidence for a systematic modulation of subjective difficulty/cognitive effort across conditions

134

but RTs showed clear modulations, that could be interpreted in terms of sensory-motor

constraints (see above ‘Behavior’). To examine the potential modulation of LC activity, we fit

a GLM for spike count in the [-250;0ms] epoch before action onset in the Target-detection

task, with only RT as parameter. Only 1/26 neuron exhibited a significant (positive) encoding

of RT. Moreover, the influence of RT on spike count was not significant at the population level

(p=0.56, Fig. 4N).

Thus, in this task where the behavioral response was modulated by sensory-motor

processes, LC neurons showed no significant modulation of action triggering-related activity

across task conditions and no relation with the RT.

LC and physical effort: force exertion-related activity

To examine the relation between LC activity and force production in the Force-

discounting and Target-detection tasks, we focused on the period of the force exertion, after

action onset. Force exertion usually lasted for about 800ms in the Force-discounting task (Fig.

S1A&B), and 500ms in the Target-detection task (Fig. 2G), with the shortest presses lasting for

about 600ms and 400ms respectively. In order to avoid any overlap with the reward delivery

period, we restricted the analyses to a [0;600ms] post action onset epoch in the Force-

discounting task and a [0;400ms] epoch in the Target-detection task.

Force-discounting task

We first examined the modulation of LC activity by task parameters by fitting a GLM

for spike count after action onset (during force exertion) with Reward and Force category. The

effect of Reward was only significant for 11/92 neurons (3 negative, 8 positive), which is less

than the numbers of neurons expected by chance (n=13) given the size of that population

(n=92). In addition, the influence of Reward was not significant across the population (second

order t-test, p=0.7, Fig. 5A). The effect of the Force category was significant for 38/92 neurons

(1 negative, 37 positive, example unit in Fig. 4G) and was consistently positive across the

population (t(91)=9.43; p<10-14, Fig. 5A).

We then examined the influence of physical force by fitting a GLM for spike count with

the maximum exerted force as parameter. The effect was significant for 38/92 neurons (2

negative, 36 positive) and it was consistently positive across the population (second order t-

test, t(91)=7.39; p<10-10, Fig. 5B). In addition, the influence of exerted force on LC activity

135

during action execution remained significant when task parameters were included as co-

regressors (exerted force effect: still significant for 17/92 neurons, 4 negative, 13 positive;

population effect: t(91)=2.04 ; p=0.044, Fig. 5C). Note that the effect of Reward was significant

for 10/92 neurons (3 negative, 7 positive), and still not significant at the level of the population

(second order t-test: p=0.66, Fig. 5C). The effect of the category of Force was significant for

20/92 neurons (3 negative, 17 positive) and it remained significantly positive at the population

level (second order t-test: t(91)=4.08; p<10-4, Fig. 5C). Thus, the activity of LC neurons during

force production was significantly modulated by the amount of force exerted, over and above

the influence of task parameters.

We then examined the influence of RT on LC activity during force exertion. The effect

of RT was significant at the population level when RT was the only parameter in the GLM for

spike count (significant for 10/92 neurons, 3 positive, 7 negative; second order t-test: t(91)=-

2.94; p=0.0041). However, the effect was no longer significant when task parameters were

added to the model (still significant for 10/92 neurons, second order t-test: p=0.18), indicating

that during force exertion, LC neurons showed little sensitivity to RT.

Target-detection task

We examined influence of the exerted force on LC activity during force exertion in the

Target-detection task. We fit a GLM for spike count in the [0;400ms] post-action onset epoch

with only the maximum exerted force as parameter. The effect was significant for 7/26

neurons (3 negative, 4 positive) but it was not significant across the population (p=0.62, Fig.

5F). Note that the effect of the exerted force was not present either before action onset

(significant for 4/20 neurons, 3 positive, 1 negative, second order t-test: p=0.56), and that

there was no effect of RT on spike count during force exertion (significant for 3/26 neurons, 2

negative, 1 positive, second order t-test: p=0.3, Fig. 5G).

Thus, in this task where there is no evidence that force production involved a

systematic difference in effort across conditions, we found no evidence for a modulation of

LC activity by amount of force produced.

136

Figure 5: Force-exertion-related activity in the Force-discounting and Target-detection tasks.

(A-E) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike

count during action (force exertion) with (A) task parameters (Force category and Reward) or

(B) the maximum exerted force on the grip alone, (C) task parameters and the maximum

exerted force, (D) RT alone, or (E) RT, Reward and Force category as parameters, in the Force-

discounting task.

(A) During the force exertion, LC neurons encoded the Force category positively if the exerted

force was not included in the model, and (B) they encoded the exerted force positively if task

parameters were not included in the model. (C) In the model including task parameters and

the maximum force exerted, there was a positive effect of both the exerted force and Force

category.

(D) During the force exertion, there was a positive effect of RT on spike count if RT was alone

in the model, but (E) this effect disappeared if task parameters were included in the model.

137

(F,G) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike

count during action (force exertion) with (F) the maximum exerted force on the grip alone, or

(G) RT alone, as parameters, in the target-detection task. (F) During force exertion, there was

no effect of the maximum exerted force (G) or of RT in the Target-detection task.

*: p<0.05; ***: p<0.001; n.s.: non-significant; F: Force, R: Reward; RT: response time.

DISCUSSION

Summary

The aim of this study was to clarify the dynamic relation between LC activity and effort

production. We examined the activity of LC neurons in two effort-discounting tasks (Force-

discounting and Delay-discounting) and compared it to activity in a task only manipulating

sensory-motor processes but not effort (target-detection task). First, in both effort-

discounting tasks, triggering the response required more producing cognitive control when

outcome value decreased. The firing of LC neurons at the time of action-triggering scaled with

task parameters in directions compatible with an encoding of the difficulty of triggering the

action, and with RT, which probably scaled with the effort produced to trigger the action.

Second, LC neurons were activated during force production both in the Force-discounting task

and in the control Target-detection task, but the magnitude of this activation only scaled with

the exerted force in the Force-discounting task, where producing the force was actually

effortful (since it decreased trial subjective value, as measured by WTW). In the control task,

where similar sensory-motor processes where involved but where we could not find any

evidence of an effort contrast across conditions, as evidenced by the absence of engagement

differences, LC neurons showed no sign of modulation by the exerted force. Thus, across tasks,

LC neurons were activated at the time when monkeys made an effort, either a cognitive effort,

to trigger a costly behavioral response, or a physical effort, to exert force on the grip. Critically,

the magnitude of these activations scaled with the effort produced to face that difficulty, with

a sub-second precision. Altogether, this work supports the idea that LC activity continuously

monitors the production of effort.

138

Behavior

Manipulating effort in animals is very challenging, since it is a complex notion and

measures remain very indirect. Operationally, effort can generally be defined as a mobilization

of resources to boost performance in the face of a difficulty and meet task goals (Borderies et

al., 2020; Kool and Botvinick, 2018; Shenhav et al., 2017). Therefore, effort can only be

evaluated indirectly, through the relative modulations of performance relative to task

difficulty. Here, we manipulated two types of difficulty: overcoming a natural tendency to

disengage from the task when the expected value was low (cognitive effort), and exerting a

relatively high level of force on the grip (physical effort).

For cognitive effort, we took advantage of the fact that monkeys tend to disengage

from single-option tasks when the outcome value is low, even if erroneous trials are repeated

(Bowman et al., 1996; Hori et al., 2009; Jahn et al., 2018, 2020; Minamimoto et al., 2009b,

2012; Varazzani et al., 2015). In these conditions, failing to engage in a low-value trial carries

disutility, since it only increases the delay until the next reward: from an instrumental

perspective, monkeys should engage in every single trial until their motivation level is low

enough for them to stop performing the task. Conversely, in versions of these tasks where

aborted trials were not repeated, monkeys rapidly learned to skip trials associated with low

value and/or high costs (La Camera et al., 2018). This tendency to abort trials associated with

low benefits and/or high costs implies a greater difficulty to overcome this automatic response

and perform the task despite low short-term utility. When erroneous trials are repeated and

skipping trials is not instrumental, including our tasks, monkeys engage in a majority of trials.

Consequently, in correctly performed trials in our tasks, they must have been exerting

cognitive control to overcome their natural tendency to abort trials associated with low

reward and/or high costs, as classically described in humans and other animals performing

standard discounting tasks (Filevich et al., 2013, 2012; Shenhav et al., 2017; Widge et al.,

2019), although the nature and the dynamics of the underlying mechanisms remains difficult

to assess. Based on that principle, we assumed that the proportion of trials in which monkeys

failed to engage in a specific task condition was a good proxy of the effort required to trigger

the action (subjective difficulty) in that cost-benefit contingency (i.e. the expected outcome

value). In both the Delay-discounting and Force-discounting tasks, the subjective difficulty to

trigger the action (and presumably the associated level of cognitive effort) decreased with

expected benefits (reward size) and increased with expected costs (force or delay). By

139

contrast, we failed to detect any evidence of a difference in WTW across conditions in the

control Target-detection task, which implies that in that task the level of subjective difficulty

and associated cognitive control invested to engage in the task was relatively equivalent

across conditions.

In addition, we used RTs as a proxy for trial-by-trial variability in subjective difficulty to

trigger the action, and potentially to the corresponding amount of cognitive effort expensed.

Obviously, RTs were affected both by decision-related processes and by sensory-motor

constraints (Dmochowski and Norcia, 2015; Gilden, 1997; Hunt et al., 2012; Krajbich et al.,

2010; Krajbich and Rangel, 2011; Milosavljevic et al., 2010; Morris et al., 2019; Philiastides and

Ratcliff, 2013). Still, in tasks in which conditions differed almost exclusively in terms of

outcome value, with virtually no sensory-motor differences, RTs reliably decreased with

outcome value and therefore increased with the subjective difficulty to trigger the response

(Hori et al., 2009; Minamimoto et al., 2009b, 2012). In both of our tasks involving force

production (Force-discounting and Target-detection), RTs were strongly modulated by motor

processes, as indicated by the significant relation with the amount of exerted force. But in the

Delay-discounting task, in which sensory-motor constraints were equivalent across conditions,

the modulations of RTs were in line with an interpretation in terms of subjective difficulty to

trigger the action (to overcome a natural tendency to abort trials when value was low). Since

monkeys remained engaged in the task from cue onset to outcome delivery (especially in the

Force-discounting task in which gaze fixation was imposed), variations of RTs across trials were

equivalent to measures of time on task. Under the assumption of a constant rate of effort

production per unit of time, time on task and therefore RT could be taken as a measure of the

amount of cognitive effort invested in the decision to trigger the action (Griffiths et al., 2015;

Lieder et al., 2014, 2012; Lieder and Griffiths, 2017). On the other hand, one could argue that

for a given level of difficulty (within task conditions), engaging more effort would result in

making decision processes more efficient, and therefore lead to shorter RTs (Navon, 1984;

Shenhav et al., 2017).

In humans studies, pupil dilation often appears to constitute a better measure of

cognitive effort, compared to RT (Alnæs et al., 2014; Hess and Polt, 1964; Kahneman, 1973;

Kahneman and Beatty, 1966; Simpson and Hale, 1969; Wahn et al., 2016; Wright et al., 2013).

But given the complex and slow dynamics of pupil dilation it would have been difficult to use

it here, especially given the strong relation between pupil and physical effort (Varazzani et al.,

140

2015; Zénon et al., 2014). From a theoretical point of view, the strong correlation between an

autonomic measures and cognitive and physical effort suggests that the central nervous

system could be using a common mechanism to mobilize the resources in the face of both

cognitive and physical challenges. In both cases, the activation of the sympathetic system

would not only increase pupil dilation but also heart rate and blood pressure, as well as the

availability of metabolic resources. From a practical point of view, this also implies that pupil

is not a specific measure of cognitive effort. Finally, the use of pupil dilation as a measure of

effort is often interpreted in the light of its strong relation with LC activity (Joshi et al., 2016;

Murphy et al., 2014; Varazzani et al., 2015). But obviously, since the goal of these experiments

was precisely to evaluate the actual relation between effort and LC activity, the interest of

pupilometry would have been limited.

Thus, for the purpose of this study, we feel relatively confident that trial-by-trial

variations of RT provide a good proxy for the subjective difficulty to trigger the responses, and

we can only speculate that they were also related to the amount of cognitive effort associated

with facing the difficulty. Of course, further experiments would be necessary to clarify that

point, since it is unclear how increasing the resources allocated for a decision process could

cause a slowing of responses rather than a speeding (Navon, 1984; Shenhav et al., 2017). Still,

we feel that demonstrating a strong and specific relation with LC activity at the time of the

decision to trigger the action provides a strong basis for such research plans.

LC activity

In both our effort-discounting tasks, LC neurons activated just prior to action onset.

Importantly, such activation was never seen for actions executed outside of tasks (between

trials for example), indicating that it was probably associated to the decision process of

triggering actions rather than movements per se. In coherence with that, pupil diameter has

been repeatedly shown to increase during decisions (Beatty, 1982; de Gee et al., 2014;

Gilzenrat et al., 2010; Lempert et al., 2015; Nassar et al., 2012), even in the context of covert

decisions (Einhäuser et al., 2010). Noticeably, LC neurons seem to only respond to decisions

to trigger actions and not to withhold them (Kalwani et al., 2014), although this contrast could

merely emerge from the difficulty to identify precisely the timing of decisions that are not

materialized by overt actions. Indeed, since the activations of LC neurons usually consist in a

141

few spikes (sometimes even just one), they could easily be missed if the activity is not aligned

correctly.

In our effort-discounting tasks, the coherent but opposite modulations of LC activity

and engagement rates by task parameters as well as the positive relation between LC activity

and RT provide additional arguments for the relation of LC activity with decision processes,

more specifically effort production for decisions to act. Indeed, task parameters (Reward and

Delay or Force) influenced pre-action onset LC activity and WTW in opposite directions.

Additionally, LC neurons reliably encoded RT in that same period across the two effort-

discounting tasks. In our frame, this modulation points towards an encoding of the subjective

difficulty to trigger actions, and possibly the cognitive control produced, although the strong

motor effects on RT in the force-discounting task called for some caution in that

interpretation. Yet, since we controlled for RT encoding in a context in which it virtually only

varied with sensory-motor effects (Target-detection task), our results point towards a specific

relation between LC activity and subjective difficulty. Additionally, our model comparisons

showed that GLMs including RT predicted LC activity better than models with task parameters

alone, and model fitting confirmed that RT was encoded over and above task parameters in

both effort-discounting tasks. Provided that task parameters represent trial difficulty, this

result would be in favor of an encoding of the allocated effort given the difficulty, or at least

of the invested time.

Similarly, during the sustained actions of the Force-discounting task (lasting a few

hundred milliseconds), LC neurons encoded the force exerted on the grip. Importantly, it was

not the case in a task in which the exerted force had no impact on the tendency to disengage

from trials (WTW in Target-detection task), i.e. in which the force was not perceived as

effortful. Despite the necessary caution in interpreting this negative result, our findings would

be in favor of a specific encoding of physical effort during its exertion. Besides, our

interpretation resonates with the finding that pupil dilation correlates with physical effort,

especially, if not only, in high effort conditions (Zénon et al., 2014).

Overall, our data establishes that LC neurons positively encode the subjective difficulty

of ongoing processes, and possibly the effort allocated, in case of both cognitive control

production and physical effort exertion. In other words, LC neurons as a population exhibit a

domain-general encoding of subjective difficulty and probably effort production.

Complementarily, our recent pharmacological studies are compatible with the idea that LC

142

activation supports the production of effort face of upcoming challenges, at least physical

effort (Borderies et al., 2020; Jahn et al., 2018). Indeed, decreasing NA levels induced an

increase in effort sensitivity and a decrease in force exertion, which could be expected if the

LC/NA system was involved in the mobilization of resources. Conversely, these results are

incompatible with a role of the LC in reporting subjective difficulty and effort costs to effort-

allocating structures. Indeed, in that frame, one would have expected the perturbation of LC

activity to decrease effort sensitivity, as can be seen for reward sensitivity when dopamine

depletions weaken the signaling of rewards (Le Bouc et al., 2016; Martínez-Horta et al., 2014).

We showed that during periods of decreased LC activity, monkeys still processed the effort

dimension of offers but act as if its cost was greater (Borderies et al., 2020). In the same vein,

modifications of brain NA levels and LC activity have been shown to affect cognitive control

production and specifically inhibitory control (Arnsten, 2006; Bari et al., 2020; Chamberlain et

al., 2009; Faraone et al., 2005; Fernando et al., 2012; Navarra et al., 2008; Robinson, 2012;

Robinson et al., 2008), working memory (Chamberlain et al., 2006b; Coradazzi et al., 2016;

Ellis and Nathan, 2001; Gamo et al., 2010; Li et al., 1999; Mao et al., 1999; Tzavara et al., 2006;

Wang et al., 2007), and set-shifting (Lapiz et al., 2007; Lapiz and Morilak, 2006; McGaughy et

al., 2008; Tait et al., 2007), although only a few of these studies used very specific methods.

In light of the current data, it is tempting to suggest that with a decreased NA tone, facing

upcoming challenges is even harder, and vice versa, regardless of the type of challenge

(mental or physical).

The timing of the modulations of LC activity by subjective difficulty and possibly effort

production reported here is striking, as we found that neurons encoded RT very late in the

action-triggering process (at its very end, around 200ms before action onset) and the exerted

force during the execution of the action. LC axons have a very slow conduction velocity due to

their absence of myelin, such that action potentials reach cortical areas in about 130ms

(Aston-Jones et al., 1985). Although we cannot exclude that the dynamics observed in the

present study result from the artificial separation in time of the decisions to engage and the

triggering of the action (demonstrated by Jahn et al., 2020), they would only be compatible

with an a posteriori influence on effort production. In other words, LC activity may promote

effort production following effortful processes, proportionally to their subjective difficulty and

possibly to the effort produced. For instance, the pre-action onset activation may serve to

facilitate action execution while the peri-action activation may enhance action consequence

143

monitoring. In coherence with that idea, some indirect evidence point towards the

involvement of LC activity in action monitoring, and especially post-error adjustments of RTs

(post-error slowing), as pupil diameter around errors predict the strength of post-error

slowing (Murphy et al., 2016) and non-selective enhancements of LC activity enhance it

(Sellaro et al., 2015). LC activity has also been proposed to serve as a network reset signal

(Bouret and Sara, 2005; Corbetta et al., 2008). The timing of modulations observed here could

be compatible with a role in promoting connectivity adaptations at the transition between

processes (e.g. action-triggering to action execution). Additionally, adjustments in brain

connectivity patterns, and more specifically enhancements of connectivity in task-relevant

networks and decreases in task-irrelevant ones (Brink et al., 2016; Guedj et al., 2017b; Hernaus

et al., 2017), could participate in the facilitation of effort production. These ideas remain quite

speculative, and further investigations would be required to clarify the delayed effects of LC

activity on effort production and the underlying mechanisms.

Another unresolved element here concerns the origin of the inputs of the LC that could

provide information about effort production, as well as the targets of the LC responsible for

its effort facilitation effects. Mental and physical effort perception correlates with pupil

diameter (Alnæs et al., 2014; Kahneman, 1973; Wahn et al., 2016; Wright et al., 2013; Zénon

et al., 2014) and is associated to changes in heart rate and metabolic activity (Andrade et al.,

2020; Carroll et al., 1986a; Da Silva et al., 2020; Sims et al., 1988; Sims and Carroll, 1990; Tibana

et al., 2019; Turner et al., 1983; Turner and Carroll, 1985), indicating that effort production is

associated to autonomic activation. The modulations of LC activity by subjective difficulty

could merely be the consequence of its connections with the nucleus Paragigantocellularis

(PGi, Aston-Jones et al., 1991, 1986; Ennis and Aston-Jones, 1988, 1986; Van Bockstaele and

Aston-Jones, 1995), a key component of the sympathetic system. They could also arise from

and/or target cortical and subcortical components of the brain effort system. For instance, in

the thalamus, a subcortical structure widely connected to the LC (“Ascending projections of

the locus coeruleus in the rat. II. Autoradiographic study,” 1977; “Demonstration of extensive

brainstem projections to medial and lateral thalamus and hypothalamus in the rat,” 1990;

McBride and Sutin, 1976; Schwarz et al., 2015), the centromedian-parafascicular complex

(Cm-Pf) exhibits task-evoked activity reminiscent of LC patterns of responses to relevant cues

and actions, with regards to trial value (Matsumoto et al., 2001; Minamimoto et al., 2009a,

2005; Minamimoto and Kimura, 2002). The Cm-Pf, possibly via interplay with the LC, could

144

play a similar role in regulating effort production. Another region considered as a crucial

component of the effort system is the medial pre-frontal cortex, and in particular its

subcomponent the anterior cingulate cortex (ACC), although the anatomical definition of that

area remains quite volatile across studies, and might at least overlap with the supplementary

motor area (SMA). The ACC-SMA is critical for physical effort perception (White et al., 2013;

Zénon et al., 2015), effort-based decision-making (Rudebeck et al., 2006; Rushworth et al.,

2004; Shenhav et al., 2013; Walton et al., 2009, 2004, 2003) and error monitoring (Botvinick

et al., 2004; Dehaene et al., 1994; Gehring and Knight, 2000; Luu and Tucker, 2001; van Veen

et al., 2001; van Veen and Carter, 2002). In that sense, these roles could result from an

interplay with the LC/NA system, as suggested by computational studies (Silvetti et al., 2018).

Still, since most of these results were obtained using long-term modifications of ACC activity

(lesions, transcranial magnetic stimulation) or techniques with low temporal precision

(functional magnetic resonance imaging), the organization and dynamics of the crosstalk

between the LC and the ACC remain hypothetical.

Altogether, our work refines the relation between LC activity and effort production and

its dynamics. Before actions, at the end of the action-triggering process, LC neurons encoded

the subjective difficulty of that process, and possibly the cognitive control allocated. During

sustained actions requiring force exertion, LC neurons encoded the physical effort associated.

Taken together, these modulations suggest a domain-general encoding of effort production,

regardless of the nature of the effort. Considering their timing, these modulations may

promote the mobilization of resources for monitoring the consequences of triggering the

action, both from a motor and executive perspective, such that courses of action requiring

higher effort to be triggered are more likely to succeed. Thus, this works supports a key role

of the LC in effort processing.

145

References Alnæs, D., Sneve, M.H., Espeseth, T., Endestad, T., van de

Pavert, S.H.P., Laeng, B., 2014. Pupil size signals mental

effort deployed during multiple object tracking and predicts

brain activity in the dorsal attention network and the locus

coeruleus. J. Vis. 14. https://doi.org/10.1167/14.4.1

Andrade, L.S., Kanitz, A.C., Häfele, M.S., Schaun, G.Z., Pinto,

S.S., Alberton, C.L., 2020. Relationship between Oxygen

Uptake, Heart Rate, and Perceived Effort in an Aquatic

Incremental Test in Older Women. Int. J. Environ. Res. Public.

Health 17. https://doi.org/10.3390/ijerph17228324

Arnsten, A.F.T., 2006. Stimulants: Therapeutic actions in

ADHD. Neuropsychopharmacol. Off. Publ. Am. Coll.

Neuropsychopharmacol. 31, 2376–2383.

https://doi.org/10.1038/sj.npp.1301164

Aston-Jones, G., 2005. Brain structures and receptors

involved in alertness. Sleep Med. 6, S3–S7.

https://doi.org/10.1016/S1389-9457(05)80002-4

Aston-Jones, G., Ennis, M., Pieribone, V.A., Nickell, W.T.,

Shipley, M.T., 1986. The brain nucleus locus coeruleus:

restricted afferent control of a broad efferent network.

Science 234, 734–737.

https://doi.org/10.1126/science.3775363

Aston-Jones, G., Foote, S.L., Segal, M., 1985. Impulse

conduction properties of noradrenergic locus coeruleus

axons projecting to monkey cerebrocortex. Neuroscience 15,

765–777. https://doi.org/10.1016/0306-4522(85)90077-6

Aston-Jones, G., Shipley, M.T., Chouvet, G., Ennis, M., van

Bockstaele, E., Pieribone, V., Shiekhattar, R., Akaoka, H.,

Drolet, G., Astier, B., Charléty, P., Valentino, R.J., Williams,

J.T., 1991. Afferent regulation of locus coeruleus neurons:

anatomy, physiology and pharmacology, in: Progress in Brain

Research. Elsevier, pp. 47–75.

https://doi.org/10.1016/S0079-6123(08)63799-1

Bari, A., Xu, S., Pignatelli, M., Takeuchi, D., Feng, J., Li, Y.,

Tonegawa, S., 2020. Differential attentional control

mechanisms by two distinct noradrenergic coeruleo-frontal

cortical pathways. Proc. Natl. Acad. Sci. U. S. A. 117, 29080–

29089. https://doi.org/10.1073/pnas.2015635117

Beatty, J., 1982. Task-evoked pupillary responses, processing

load, and the structure of processing resources. Psychol.

Bull. 91, 276–292. https://doi.org/10.1037/0033-

2909.91.2.276

Borderies, N., Bornert, P., Gilardeau, S., Bouret, S., 2020.

Pharmacological evidence for the implication of

noradrenaline in effort. PLOS Biol. 18, e3000793.

https://doi.org/10.1371/journal.pbio.3000793

Bouret, S., Ravel, S., Richmond, B.J., 2012. Complementary

neural correlates of motivation in dopaminergic and

noradrenergic neurons of monkeys. Front. Behav. Neurosci.

6. https://doi.org/10.3389/fnbeh.2012.00040

Bouret, S., Sara, S.J., 2005. Network reset: a simplified

overarching theory of locus coeruleus noradrenaline

function. Trends Neurosci. 28, 574–582.

https://doi.org/10.1016/j.tins.2005.09.002

Bowman, E.M., Aigner, T.G., Richmond, B.J., 1996. Neural

signals in the monkey ventral striatum related to motivation

for juice and cocaine rewards. J. Neurophysiol. 75, 1061–

1073. https://doi.org/10.1152/jn.1996.75.3.1061

Brink, R.L. van den, Pfeffer, T., Warren, C.M., Murphy, P.R.,

Tona, K.-D., Wee, N.J.A. van der, Giltay, E., Noorden, M.S.

van, Rombouts, S.A.R.B., Donner, T.H., Nieuwenhuis, S.,

2016. Catecholaminergic Neuromodulation Shapes Intrinsic

MRI Functional Connectivity in the Human Brain. J. Neurosci.

36, 7865–7876. https://doi.org/10.1523/JNEUROSCI.0744-

16.2016

Buffalari, D.M., Grace, A.A., 2007. Noradrenergic modulation

of basolateral amygdala neuronal activity: opposing

influences of alpha-2 and beta receptor activation. J.

Neurosci. Off. J. Soc. Neurosci. 27, 12358–12366.

https://doi.org/10.1523/JNEUROSCI.2007-07.2007

Carroll, D., Turner, J.R., Hellawell, J.C., 1986. Heart Rate and

Oxygen Consumption during Active Psychological Challenge:

The Effects of Level of Difficulty. Psychophysiology 23, 174–

181. https://doi.org/10.1111/j.1469-8986.1986.tb00613.x

Carstens, E., Leah, J., Lechner, J., Zimmermann, M., 1990.

Demonstration of extensive brainstem projections to medial

and lateral thalamus and hypothalamus in the rat.

Neuroscience 35, 609–626. https://doi.org/10.1016/0306-

4522(90)90333-Y

Carter, M.E., Yizhar, O., Chikahisa, S., Nguyen, H.,

Adamantidis, A., Nishino, S., Deisseroth, K., de Lecea, L.,

2010. Tuning arousal with optogenetic modulation of locus

coeruleus neurons. Nat. Neurosci. 13, 1526–1533.

https://doi.org/10.1038/nn.2682

Cassens, G., Kuruc, A., Roffman, M., Orsulak, P.J.,

Schildkraut, J.J., 1981. Alterations in brain norepinephrine

metabolism and behavior induced by environmental stimuli

previously paired with inescapable shock. Behav. Brain Res.

2, 387–407. https://doi.org/10.1016/0166-4328(81)90020-6

Chamberlain, S.R., Hampshire, A., Müller, U., Rubia, K., Del

Campo, N., Craig, K., Regenthal, R., Suckling, J., Roiser, J.P.,

Grant, J.E., Bullmore, E.T., Robbins, T.W., Sahakian, B.J.,

2009. Atomoxetine modulates right inferior frontal

activation during inhibitory control: a pharmacological

146

functional magnetic resonance imaging study. Biol.

Psychiatry 65, 550–555.

https://doi.org/10.1016/j.biopsych.2008.10.014

Chamberlain, S.R., Müller, U., Blackwell, A.D., Robbins, T.W.,

Sahakian, B.J., 2006. Noradrenergic modulation of working

memory and emotional memory in humans.

Psychopharmacology (Berl.) 188, 397–407.

https://doi.org/10.1007/s00213-006-0391-6

Clark, C.R., Geffen, G.M., Geffen, L.B., 1987. Catecholamines

and attention. I: Animal and clinical studies. Neurosci.

Biobehav. Rev. 11, 341–352. https://doi.org/10.1016/s0149-

7634(87)80006-4

Coradazzi, M., Gulino, R., Fieramosca, F., Falzacappa, L.V.,

Riggi, M., Leanza, G., 2016. Selective noradrenaline

depletion impairs working memory and hippocampal

neurogenesis. Neurobiol. Aging 48, 93–102.

https://doi.org/10.1016/j.neurobiolaging.2016.08.012

Corbetta, M., Patel, G., Shulman, G.L., 2008. The Reorienting

System of the Human Brain: From Environment to Theory of

Mind. Neuron 58, 306–324.

https://doi.org/10.1016/j.neuron.2008.04.017

Coull, J.T., Nobre, A.C., Frith, C.D., 2001. The Noradrenergic

α2 Agonist Clonidine Modulates Behavioural and

Neuroanatomical Correlates of Human Attentional Orienting

and Alerting. Cereb. Cortex 11, 73–84.

https://doi.org/10.1093/cercor/11.1.73

Cousins, M.S., Salamone, J.D., 1994. Nucleus accumbens

dopamine depletions in rats affect relative response

allocation in a novel cost/benefit procedure. Pharmacol.

Biochem. Behav. 49, 85–91. https://doi.org/10.1016/0091-

3057(94)90460-x

Curtis, A.L., Leiser, S.C., Snyder, K., Valentino, R.J., 2012.

Predator stress engages corticotropin-releasing factor and

opioid systems to alter the operating mode of locus

coeruleus norepinephrine neurons. Neuropharmacology 62,

1737–1745.

https://doi.org/10.1016/j.neuropharm.2011.11.020

Da Silva, D.F., Mohammad, S., Hutchinson, K.A., Adamo, K.B.,

2020. Cross-Validation of Ratings of Perceived Exertion

Derived from Heart Rate Target Ranges Recommended for

Pregnant Women. Int. J. Exerc. Sci. 13, 1340–1351.

de Gee, J.W., Knapen, T., Donner, T.H., 2014. Decision-

related pupil dilation reflects upcoming choice and individual

bias. Proc. Natl. Acad. Sci. 111, E618–E625.

https://doi.org/10.1073/pnas.1317557111

Dmochowski, J.P., Norcia, A.M., 2015. Cortical Components

of Reaction-Time during Perceptual Decisions in Humans.

PLOS ONE 10, e0143339.

https://doi.org/10.1371/journal.pone.0143339

Doya, K., 2002. Metalearning and neuromodulation. Neural

Netw. 15, 495–506. https://doi.org/10.1016/S0893-

6080(02)00044-8

Einhäuser, W., Koch, C., Carter, O.L., 2010. Pupil Dilation

Betrays the Timing of Decisions. Front. Hum. Neurosci. 4, 18.

https://doi.org/10.3389/fnhum.2010.00018

Ellis, K.A., Nathan, P.J., 2001. The pharmacology of human

working memory. Int. J. Neuropsychopharmacol. 4, 299–

313. https://doi.org/10.1017/S1461145701002541

Ennis, M., Aston-Jones, G., 1988. Activation of locus

coeruleus from nucleus paragigantocellularis: a new

excitatory amino acid pathway in brain. J. Neurosci. 8, 3644–

3657. https://doi.org/10.1523/JNEUROSCI.08-10-

03644.1988

Ennis, M., Aston-Jones, G., 1986. Evidence for self- and

neighbor-mediated postactivation inhibition of locus

coeruleus neurons. Brain Res. 374, 299–305.

https://doi.org/10.1016/0006-8993(86)90424-5

Epstein, S., 1994. Integration of the cognitive and the

psychodynamic unconscious. Am. Psychol. 49, 709–724.

https://doi.org/10.1037/0003-066X.49.8.709

Faraone, S.V., Biederman, J., Spencer, T., Michelson, D.,

Adler, L., Reimherr, F., Seidman, L., 2005. Atomoxetine and

Stroop Task Performance in Adult Attention-

Deficit/Hyperactivity Disorder. J. Child Adolesc.

Psychopharmacol. 15, 664–670.

https://doi.org/10.1089/cap.2005.15.664

Fernando, A.B.P., Economidou, D., Theobald, D.E., Zou, M.-

F., Newman, A.H., Spoelder, M., Caprioli, D., Moreno, M.,

Hipólito, L., Aspinall, A.T., Robbins, T.W., Dalley, J.W., 2012.

Modulation of high impulsivity and attentional performance

in rats by selective direct and indirect dopaminergic and

noradrenergic receptor agonists. Psychopharmacology

(Berl.) 219, 341–352. https://doi.org/10.1007/s00213-011-

2408-z

Filevich, E., Kühn, S., Haggard, P., 2013. There Is No Free

Won’t: Antecedent Brain Activity Predicts Decisions to

Inhibit. PLOS ONE 8, e53053.

https://doi.org/10.1371/journal.pone.0053053

Filevich, E., Kühn, S., Haggard, P., 2012. Intentional inhibition

in human action: the power of “no.” Neurosci. Biobehav.

Rev. 36, 1107–1118.

https://doi.org/10.1016/j.neubiorev.2012.01.006

Foote, S.L., Bloom, F.E., Aston-Jones, G., 1983. Nucleus locus

ceruleus: new evidence of anatomical and physiological

specificity. Physiol. Rev. 63, 844–914.

https://doi.org/10.1152/physrev.1983.63.3.844

147

Gallagher, K.M., Fadel, P.J., Strømstad, M., Ide, K., Smith,

S.A., Querry, R.G., Raven, P.B., Secher, N.H., 2001. Effects of

partial neuromuscular blockade on carotid baroreflex

function during exercise in humans. J. Physiol. 533, 861–870.

https://doi.org/10.1111/j.1469-7793.2001.t01-1-00861.x

Gamo, N.J., Wang, M., Arnsten, A.F.T., 2010.

Methylphenidate and Atomoxetine Enhance Prefrontal

Function Through α2-Adrenergic and Dopamine D1

Receptors. J. Am. Acad. Child Adolesc. Psychiatry 49, 1011–

1023. https://doi.org/10.1016/j.jaac.2010.06.015

Gilden, D.L., 1997. Fluctuations in the Time Required for

Elementary Decisions. Psychol. Sci. 8, 296–301.

https://doi.org/10.1111/j.1467-9280.1997.tb00441.x

Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M., Cohen, J.D.,

2010. Pupil diameter tracks changes in control state

predicted by the adaptive gain theory of locus coeruleus

function. Cogn. Affect. Behav. Neurosci. 10, 252–269.

https://doi.org/10.3758/CABN.10.2.252

Griffiths, T.L., Lieder, F., Goodman, N.D., 2015. Rational use

of cognitive resources: levels of analysis between the

computational and the algorithmic. Top. Cogn. Sci. 7, 217–

229. https://doi.org/10.1111/tops.12142

Guedj, C., Monfardini, E., Reynaud, A.J., Farnè, A., Meunier,

M., Hadj-Bouziane, F., 2017. Boosting Norepinephrine

Transmission Triggers Flexible Reconfiguration of Brain

Networks at Rest. Cereb. Cortex N. Y. N 1991 27, 4691–4700.

https://doi.org/10.1093/cercor/bhw262

Hernaus, D., Casales Santa, M.M., Offermann, J.S., Van

Amelsvoort, T., 2017. Noradrenaline transporter blockade

increases fronto-parietal functional connectivity relevant for

working memory. Eur. Neuropsychopharmacol. 27, 399–

410. https://doi.org/10.1016/j.euroneuro.2017.02.004

Hess, E.H., Polt, J.M., 1964. Pupil Size in Relation to Mental

Activity during Simple Problem-Solving. Science 143, 1190–

1192. https://doi.org/10.1126/science.143.3611.1190

Hori, Y., Minamimoto, T., Kimura, M., 2009. Neuronal

Encoding of Reward Value and Direction of Actions in the

Primate Putamen. J. Neurophysiol. 102, 3530–3543.

https://doi.org/10.1152/jn.00104.2009

Hull, C.L., 1943. Principles of behavior: an introduction to

behavior theory, Principles of behavior: an introduction to

behavior theory. Appleton-Century, Oxford, England.

Hunt, L.T., Kolling, N., Soltani, A., Woolrich, M.W.,

Rushworth, M.F.S., Behrens, T.E.J., 2012. Mechanisms

underlying cortical activity during value-guided choice. Nat.

Neurosci. 15, 470–476. https://doi.org/10.1038/nn.3017

Jahn, C.I., Gilardeau, S., Varazzani, C., Blain, B., Sallet, J.,

Walton, M.E., Bouret, S., 2018. Dual contributions of

noradrenaline to behavioural flexibility and motivation.

Psychopharmacology (Berl.) 235, 2687–2702.

https://doi.org/10.1007/s00213-018-4963-z

Jahn, C.I., Varazzani, C., Sallet, J., Walton, M.E., Bouret, S.,

2020. Noradrenergic But Not Dopaminergic Neurons Signal

Task State Changes and Predict Reengagement After a

Failure. Cereb. Cortex N. Y. N 1991 30, 4979–4994.

https://doi.org/10.1093/cercor/bhaa089

Jones, B.E., Moore, R.Y., 1977. Ascending projections of the

locus coeruleus in the rat. II. Autoradiographic study. Brain

Res. 127, 23–53. https://doi.org/10.1016/0006-

8993(77)90378-X

Joshi, S., Li, Y., Kalwani, R.M., Gold, J.I., 2016. Relationships

between Pupil Diameter and Neuronal Activity in the Locus

Coeruleus, Colliculi, and Cingulate Cortex. Neuron 89, 221–

234. https://doi.org/10.1016/j.neuron.2015.11.028

Kahneman, D., 1973. Attention and effort. Prentice-Hall,

Englewood Cliffs, N.J.

Kahneman, D., Beatty, J., 1966. Pupil diameter and load on

memory. Science 154, 1583–1585.

https://doi.org/10.1126/science.154.3756.1583

Kalwani, R.M., Joshi, S., Gold, J.I., 2014. Phasic Activation of

Individual Neurons in the Locus Ceruleus/Subceruleus

Complex of Monkeys Reflects Rewarded Decisions to Go But

Not Stop. J. Neurosci. 34, 13656–13669.

https://doi.org/10.1523/JNEUROSCI.2566-14.2014

Koob, G.F., 1999. Corticotropin-releasing factor,

norepinephrine, and stress. Biol. Psychiatry 46, 1167–1180.

https://doi.org/10.1016/s0006-3223(99)00164-x

Kool, W., Botvinick, M., 2018. Mental labour. Nat. Hum.

Behav. 2, 899–908. https://doi.org/10.1038/s41562-018-

0401-9

Kool, W., Botvinick, M., 2014. A labor/leisure tradeoff in

cognitive control. J. Exp. Psychol. Gen. 143, 131–141.

https://doi.org/10.1037/a0031048

Kool, W., McGuire, J.T., Rosen, Z.B., Botvinick, M.M., 2010.

Decision making and the avoidance of cognitive demand. J.

Exp. Psychol. Gen. 139, 665–682.

https://doi.org/10.1037/a0020198

Krajbich, I., Armel, C., Rangel, A., 2010. Visual fixations and

the computation and comparison of value in simple choice.

Nat. Neurosci. 13, 1292–1298.

https://doi.org/10.1038/nn.2635

148

Krajbich, I., Rangel, A., 2011. Multialternative drift-diffusion

model predicts the relationship between visual fixations and

choice in value-based decisions. Proc. Natl. Acad. Sci. 108,

13852–13857. https://doi.org/10.1073/pnas.1101328108

La Camera, G., Bouret, S., Richmond, B.J., 2018.

Contributions of Lateral and Orbital Frontal Regions to

Abstract Rule Acquisition and Reversal in Monkeys. Front.

Neurosci. 12, 165.

https://doi.org/10.3389/fnins.2018.00165

Lapiz, M.D.S., Bondi, C.O., Morilak, D.A., 2007. Chronic

treatment with desipramine improves cognitive

performance of rats in an attentional set-shifting test.

Neuropsychopharmacol. Off. Publ. Am. Coll.

Neuropsychopharmacol. 32, 1000–1010.

https://doi.org/10.1038/sj.npp.1301235

Lapiz, M.D.S., Morilak, D.A., 2006. Noradrenergic

modulation of cognitive function in rat medial prefrontal

cortex as measured by attentional set shifting capability.

Neuroscience 137, 1039–1049.

https://doi.org/10.1016/j.neuroscience.2005.09.031

Le Bouc, R., Rigoux, L., Schmidt, L., Degos, B., Welter, M.-L.,

Vidailhet, M., Daunizeau, J., Pessiglione, M., 2016.

Computational Dissection of Dopamine Motor and

Motivational Functions in Humans. J. Neurosci. 36, 6623–

6633. https://doi.org/10.1523/JNEUROSCI.3078-15.2016

Lempert, K.M., Chen, Y.L., Fleming, S.M., 2015. Relating Pupil

Dilation and Metacognitive Confidence during Auditory

Decision-Making. PloS One 10, e0126588.

https://doi.org/10.1371/journal.pone.0126588

Li, B.-M., Mao, Z.-M., Wang, M., Mei, Z.-T., 1999. Alpha-2

Adrenergic Modulation of Prefrontal Cortical Neuronal

Activity Related to Spatial Working Memory in Monkeys.

Neuropsychopharmacology 21, 601–610.

https://doi.org/10.1016/S0893-133X(99)00070-6

Lieder, F., Griffiths, T.L., 2017. Strategy selection as rational

metareasoning. Psychol. Rev. 124, 762–794.

https://doi.org/10.1037/rev0000075

Lieder, F., Griffiths, T.L., Goodman, N.D., 2012. Burn-in, bias,

and the rationality of anchoring. Presented at the NIPS.

Lieder, F., Plunkett, D., Hamrick, J.B., Russell, S.J., Hay, N.J.,

Griffiths, T.L., 2014. Algorithm selection by rational

metareasoning as a model of human strategy selection, in:

Proceedings of the 27th International Conference on Neural

Information Processing Systems - Volume 2, NIPS’14. MIT

Press, Cambridge, MA, USA, pp. 2870–2878.

Mao, Z.-M., Arnsten, A.F.T., Li, B.-M., 1999. Local infusion of

an α-1 adrenergic agonist into the prefrontal cortex impairs

spatial working memory performance in monkeys. Biol.

Psychiatry 46, 1259–1265. https://doi.org/10.1016/S0006-

3223(99)00139-0

Martínez-Horta, S., Riba, J., de Bobadilla, R.F.,

Pagonabarraga, J., Pascual-Sedano, B., Antonijoan, R.M.,

Romero, S., Mañanas, M.À., García-Sanchez, C., Kulisevsky,

J., 2014. Apathy in Parkinson’s Disease: Neurophysiological

Evidence of Impaired Incentive Processing. J. Neurosci. 34,

5918–5926.

Matsumoto, N., Minamimoto, T., Graybiel, A.M., Kimura, M.,

2001. Neurons in the thalamic CM-Pf complex supply striatal

neurons with information about behaviorally significant

sensory events. J. Neurophysiol. 85, 960–976.

https://doi.org/10.1152/jn.2001.85.2.960

McBride, R.L., Sutin, J., 1976. Projections of the locus

coeruleus and adjacent pontine tegmentum in the cat. J.

Comp. Neurol. 165, 265–284.

https://doi.org/10.1002/cne.901650302

McGAUGHY, J., ROSS, R.S., EICHENBAUM, H., 2008.

NORADRENERGIC, BUT NOT CHOLINERGIC,

DEAFFERENTATION OF PREFRONTAL CORTEX IMPAIRS

ATTENTIONAL SET-SHIFTING. Neuroscience 153, 63–71.

https://doi.org/10.1016/j.neuroscience.2008.01.064

Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., Rangel, A.,

2010. The Drift Diffusion Model Can Account for the

Accuracy and Reaction Time of Value-Based Choices Under

High and Low Time Pressure. SSRN Electron. J.

https://doi.org/10.2139/ssrn.1901533

Minamimoto, T., Hori, Y., Kimura, M., 2009a. Roles of the

thalamic CM–PF complex—Basal ganglia circuit in externally

driven rebias of action. Brain Res. Bull. 78, 75–79.

https://doi.org/10.1016/j.brainresbull.2008.08.013

Minamimoto, T., Hori, Y., Kimura, M., 2005. Complementary

process to response bias in the centromedian nucleus of the

thalamus. Science 308, 1798–1801.

https://doi.org/10.1126/science.1109154

Minamimoto, T., Hori, Y., Richmond, B.J., 2012. Is Working

More Costly than Waiting in Monkeys? PLOS ONE 7, e48434.

https://doi.org/10.1371/journal.pone.0048434

Minamimoto, T., Kimura, M., 2002. Participation of the

thalamic CM-Pf complex in attentional orienting. J.

Neurophysiol. 87, 3090–3101.

https://doi.org/10.1152/jn.2002.87.6.3090

Minamimoto, T., La Camera, G., Richmond, B.J., 2009b.

Measuring and modeling the interaction among reward size,

delay to reward, and satiation level on motivation in

monkeys. J. Neurophysiol. 101, 437–447.

https://doi.org/10.1152/jn.90959.2008

149

Morris, R., Bornstein, A., Shenhav, A. (Eds.), 2019. Goal-

directed decision making: computations and neural circuits.

Academic Press, Elsevier, London San Diego, CA Cambridge.

Murphy, P.R., Moort, M.L. van, Nieuwenhuis, S., 2016. The

Pupillary Orienting Response Predicts Adaptive Behavioral

Adjustment after Errors. PLOS ONE 11, e0151763.

https://doi.org/10.1371/journal.pone.0151763

Murphy, P.R., O’Connell, R.G., O’Sullivan, M., Robertson, I.H.,

Balsters, J.H., 2014. Pupil diameter covaries with BOLD

activity in human locus coeruleus: Pupil Diameter and Locus

Coeruleus Activity. Hum. Brain Mapp. 35, 4140–4154.

https://doi.org/10.1002/hbm.22466

Nassar, M.R., Rumsey, K.M., Wilson, R.C., Parikh, K., Heasly,

B., Gold, J.I., 2012. Rational regulation of learning dynamics

by pupil-linked arousal systems. Nat. Neurosci. 15, 1040–

1046. https://doi.org/10.1038/nn.3130

Navarra, R., Graf, R., Huang, Y., Logue, S., Comery, T.,

Hughes, Z., Day, M., 2008. Effects of atomoxetine and

methylphenidate on attention and impulsivity in the 5-

choice serial reaction time test. Prog.

Neuropsychopharmacol. Biol. Psychiatry 32, 34–41.

https://doi.org/10.1016/j.pnpbp.2007.06.017

Navon, D., 1984. Resources--a theoretical soup stone?

Psychol. Rev. 91, 216–234. https://doi.org/10.1037/0033-

295X.91.2.216

Nieuwenhuis, S., Aston-Jones, G., Cohen, J.D., 2005. Decision

making, the P3, and the locus coeruleus-norepinephrine

system. Psychol. Bull. 131, 510–532.

https://doi.org/10.1037/0033-2909.131.4.510

Pessiglione, M., Vinckier, F., Bouret, S., Daunizeau, J., Le

Bouc, R., 2018. Why not try harder? Computational approach

to motivation deficits in neuro-psychiatric diseases. Brain

141, 629–650. https://doi.org/10.1093/brain/awx278

Philiastides, M.G., Ratcliff, R., 2013. Influence of Branding on

Preference-Based Decision Making. Psychol. Sci. 24, 1208–

1215. https://doi.org/10.1177/0956797612470701

Phillips, P.E.M., Walton, M.E., Jhou, T.C., 2007. Calculating

utility: preclinical evidence for cost-benefit analysis by

mesolimbic dopamine. Psychopharmacology (Berl.) 191,

483–495. https://doi.org/10.1007/s00213-006-0626-6

Reynaud, A.J., Froesel, M., Guedj, C., Ben Hadj Hassen, S.,

Cléry, J., Meunier, M., Ben Hamed, S., Hadj-Bouziane, F.,

2019. Atomoxetine improves attentional orienting in a

predictive context. Neuropharmacology 150, 59–69.

https://doi.org/10.1016/j.neuropharm.2019.03.012

Robinson, E.S.J., 2012. Blockade of noradrenaline re-uptake

sites improves accuracy and impulse control in rats

performing a five-choice serial reaction time tasks.

Psychopharmacology (Berl.) 219, 303–312.

https://doi.org/10.1007/s00213-011-2420-3

Robinson, E.S.J., Eagle, D.M., Mar, A.C., Bari, A., Banerjee, G.,

Jiang, X., Dalley, J.W., Robbins, T.W., 2008. Similar effects of

the selective noradrenaline reuptake inhibitor atomoxetine

on three distinct forms of impulsivity in the rat.

Neuropsychopharmacol. Off. Publ. Am. Coll.

Neuropsychopharmacol. 33, 1028–1037.

https://doi.org/10.1038/sj.npp.1301487

Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman,

D.M., Rushworth, M.F.S., 2006. Separate neural pathways

process different decision costs. Nat. Neurosci. 9, 1161–

1168. https://doi.org/10.1038/nn1756

Rushworth, M.F.S., Walton, M.E., Kennerley, S.W.,

Bannerman, D.M., 2004. Action sets and decisions in the

medial frontal cortex. Trends Cogn. Sci. 8, 410–417.

https://doi.org/10.1016/j.tics.2004.07.009

Schwarz, L.A., Miyamichi, K., Gao, X.J., Beier, K.T.,

Weissbourd, B., DeLoach, K.E., Ren, J., Ibanes, S., Malenka,

R.C., Kremer, E.J., Luo, L., 2015. Viral-genetic tracing of the

input-output organization of a central noradrenaline circuit.

Nature 524, 88–92. https://doi.org/10.1038/nature14600

Sellaro, R., van Leusden, J.W.R., Tona, K.-D., Verkuil, B.,

Nieuwenhuis, S., Colzato, L.S., 2015. Transcutaneous Vagus

Nerve Stimulation Enhances Post-error Slowing. J. Cogn.

Neurosci. 27, 2126–2132.

https://doi.org/10.1162/jocn_a_00851

Shea-Brown, E., Gilzenrat, M.S., Cohen, J.D., 2008.

Optimization of decision making in multilayer networks: the

role of locus coeruleus. Neural Comput. 20, 2863–2894.

https://doi.org/10.1162/neco.2008.03-07-487

Shenhav, A., Botvinick, M.M., Cohen, J.D., 2013. The

Expected Value of Control: An Integrative Theory of Anterior

Cingulate Cortex Function. Neuron 79, 217–240.

https://doi.org/10.1016/j.neuron.2013.07.007

Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T.L.,

Cohen, J.D., Botvinick, M.M., 2017. Toward a Rational and

Mechanistic Account of Mental Effort. Annu. Rev. Neurosci.

40, 99–124. https://doi.org/10.1146/annurev-neuro-

072116-031526

Shiffrin, R.M., Schneider, W., 1977. Controlled and automatic

human information processing: II. Perceptual learning,

automatic attending and a general theory. Psychol. Rev. 84,

127–190. https://doi.org/10.1037/0033-295X.84.2.127

Silvetti, M., Vassena, E., Abrahamse, E., Verguts, T., 2018.

Dorsal anterior cingulate-brainstem ensemble as a

reinforcement meta-learner. PLOS Comput. Biol. 14,

e1006370. https://doi.org/10.1371/journal.pcbi.1006370

150

Simpson, H.M., Hale, S.M., 1969. Pupillary changes during a

decision-making task. Percept. Mot. Skills 29, 495–498.

https://doi.org/10.2466/pms.1969.29.2.495

Sims, J., Carroll, D., 1990. Cardiovascular and Metabolic

Activity at Rest and During Psychological and Physical

Challenge in Normotensives and Subjects With Mildly

Elevated Blood Pressure. Psychophysiology 27, 149–156.

https://doi.org/10.1111/j.1469-8986.1990.tb00366.x

Sims, J., Carroll, D., Turner, J.R., Hewitt, J.K., 1988. Cardiac

and Metabolic Activity in Mild Hypertensive and

Normotensive Subjects. Psychophysiology 25, 172–178.

https://doi.org/10.1111/j.1469-8986.1988.tb00982.x

Sloman, S.A., 1996. The empirical case for two systems of

reasoning. Psychol. Bull. 119, 3–22.

https://doi.org/10.1037/0033-2909.119.1.3

Stevens, J.R., Rosati, A.G., Ross, K.R., Hauser, M.D., 2005. Will

travel for food: spatial discounting in two new world

monkeys. Curr. Biol. CB 15, 1855–1860.

https://doi.org/10.1016/j.cub.2005.09.016

Sved, A.F., Cano, G., Passerin, A.M., Rabin, B.S., 2002. The

locus coeruleus, Barrington’s nucleus, and neural circuits of

stress. Physiol. Behav. 77, 737–742.

https://doi.org/10.1016/s0031-9384(02)00927-7

Tait, D.S., Brown, V.J., Farovik, A., Theobald, D.E., Dalley,

J.W., Robbins, T.W., 2007. Lesions of the dorsal

noradrenergic bundle impair attentional set-shifting in the

rat. Eur. J. Neurosci. 25, 3719–3724.

https://doi.org/10.1111/j.1460-9568.2007.05612.x

Tibana, R., Manuel Frade de Sousa, N., Prestes, J., da Cunha

Nascimento, D., Ernesto, C., Falk Neto, J., Kennedy, M.,

Azevedo Voltarelli, F., 2019. Is Perceived Exertion a Useful

Indicator of the Metabolic and Cardiovascular Responses to

a Metabolic Conditioning Session of Functional Fitness?

Sports 7, 161. https://doi.org/10.3390/sports7070161

Tsunematsu, S., 2001. Effort- and time-cost effects on

demand curves for food by pigeons under short session

closed economies. Behav. Processes 53, 47–56.

https://doi.org/10.1016/S0376-6357(00)00147-9

Turner, J.R., Carroll, D., 1985. Heart Rate and Oxygen

Consumption during Mental Arithmetic, a Video Game, and

Graded Exercise: Further Evidence of Metabolically-

Exaggerated Cardiac Adjustments? Psychophysiology 22,

261–267. https://doi.org/10.1111/j.1469-

8986.1985.tb01597.x

Turner, J.R., Carroll, D., Courtney, H., 1983. Cardiac and

Metabolic Responses to “Space Invaders”: An Instance of

Metabolically-Exaggerated Cardiac Adjustment?

Psychophysiology 20, 544–549.

https://doi.org/10.1111/j.1469-8986.1983.tb03010.x

Tzavara, E.T., Bymaster, F.P., Overshiner, C.D., Davis, R.J.,

Perry, K.W., Wolff, M., McKinzie, D.L., Witkin, J.M., Nomikos,

G.G., 2006. Procholinergic and memory enhancing

properties of the selective norepinephrine uptake inhibitor

atomoxetine. Mol. Psychiatry 11, 187–195.

https://doi.org/10.1038/sj.mp.4001763

Valentino, R.J., Page, M.E., Curtis, A.L., 1991. Activation of

noradrenergic locus coeruleus neurons by hemodynamic

stress is due to local release of corticotropin-releasing factor.

Brain Res. 555, 25–34. https://doi.org/10.1016/0006-

8993(91)90855-p

Van Bockstaele, E.J., Aston-Jones, G., 1995. Integration in the

Ventral Medulla and Coordination of Sympathetic, Pain and

Arousal Functions. Clin. Exp. Hypertens. 17, 153–165.

https://doi.org/10.3109/10641969509087062

Van Bockstaele, E.J., Bajic, D., Proudfit, H., Valentino, R.J.,

2001. Topographic architecture of stress-related pathways

targeting the noradrenergic locus coeruleus. Physiol. Behav.

73, 273–283. https://doi.org/10.1016/s0031-

9384(01)00448-6

Varazzani, C., San-Galli, A., Gilardeau, S., Bouret, S., 2015.

Noradrenaline and Dopamine Neurons in the Reward/Effort

Trade-Off: A Direct Electrophysiological Comparison in

Behaving Monkeys. J. Neurosci. 35, 7866–7877.

https://doi.org/10.1523/JNEUROSCI.0454-15.2015

Wahn, B., Ferris, D.P., Hairston, W.D., König, P., 2016. Pupil

Sizes Scale with Attentional Load and Task Experience in a

Multiple Object Tracking Task. PloS One 11, e0168087.

https://doi.org/10.1371/journal.pone.0168087

Walton, M.E., Bannerman, D.M., Alterescu, K., Rushworth,

M.F.S., 2003. Functional Specialization within Medial Frontal

Cortex of the Anterior Cingulate for Evaluating Effort-Related

Decisions. J. Neurosci. 23, 6475–6479.

https://doi.org/10.1523/JNEUROSCI.23-16-06475.2003

Walton, M.E., Devlin, J.T., Rushworth, M.F.S., 2004.

Interactions between decision making and performance

monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–

1265. https://doi.org/10.1038/nn1339

Walton, M.E., Groves, J., Jennings, K.A., Croxson, P.L., Sharp,

T., Rushworth, M.F.S., Bannerman, D.M., 2009. Comparing

the role of the anterior cingulate cortex and 6-

hydroxydopamine nucleus accumbens lesions on operant

effort-based decision making. Eur. J. Neurosci. 29, 1678–

1691. https://doi.org/10.1111/j.1460-9568.2009.06726.x

Walton, M.E., Kennerley, S.W., Bannerman, D.M., Phillips,

P.E.M., Rushworth, M.F.S., 2006. Weighing up the benefits

of work: behavioral and neural analyses of effort-related

decision making. Neural Netw. Off. J. Int. Neural Netw. Soc.

19, 1302–1314.

https://doi.org/10.1016/j.neunet.2006.03.005

151

Wang, M., Ramos, B.P., Paspalas, C.D., Shu, Y., Simen, A.,

Duque, A., Vijayraghavan, S., Brennan, A., Dudley, A., Nou,

E., Mazer, J.A., McCormick, D.A., Arnsten, A.F.T., 2007. α2A-

Adrenoceptors Strengthen Working Memory Networks by

Inhibiting cAMP-HCN Channel Signaling in Prefrontal Cortex.

Cell 129, 397–410.

https://doi.org/10.1016/j.cell.2007.03.015

White, O., Davare, M., Andres, M., Olivier, E., 2013. The role

of left supplementary motor area in grip force scaling. PloS

One 8, e83812.

https://doi.org/10.1371/journal.pone.0083812

Widge, A.S., Zorowitz, S., Basu, I., Paulk, A.C., Cash, S.S.,

Eskandar, E.N., Deckersbach, T., Miller, E.K., Dougherty, D.D.,

2019. Deep brain stimulation of the internal capsule

enhances human cognitive control and prefrontal cortex

function. Nat. Commun. 10, 1536.

https://doi.org/10.1038/s41467-019-09557-4

Witte, E.A., Marrocco, R.T., 1997. Alteration of brain

noradrenergic activity in rhesus monkeys affects the alerting

component of covert orienting. Psychopharmacology (Berl.)

132, 315–323. https://doi.org/10.1007/s002130050351

Wright, T.J., Boot, W.R., Morgan, C.S., 2013. Pupillary

response predicts multiple object tracking load, error rate,

and conscientiousness, but not inattentional blindness. Acta

Psychol. (Amst.) 144, 6–11.

https://doi.org/10.1016/j.actpsy.2013.04.018

Zénon, A., Sidibé, M., Olivier, E., 2015. Disrupting the

Supplementary Motor Area Makes Physical Effort Appear

Less Effortful. J. Neurosci. 35, 8737–8744.

https://doi.org/10.1523/JNEUROSCI.3789-14.2015

Zénon, A., Sidibé, M., Olivier, E., 2014. Pupil size variations

correlate with physical effort perception. Front. Behav.

Neurosci. 8, 286.

https://doi.org/10.3389/fnbeh.2014.00286

Zipf, G.K., 1949. Human behavior and the principle of least

effort, Human behavior and the principle of least effort.

Addison-Wesley Press, Oxford, England.

Experimental results

152

Supplementary material

Figure S1: Force exertion in the Force-discounting task across time after action onset by (A)

Monkey D and (B) Monkey A, by force categories. The thin lines represent the mean and SEM

of the exerted force at each time point.

Experimental results

153

2. Additional results

The study presented above focused on the activity of LC neurons around action onset,

arguing for a role of the LC/NA system in reporting on the amount of cognitive control that

was invested in overcoming the cost of triggering a costly action. In the following part, I report

a few additional findings regarding the Delay-Discounting task, considering that this dataset

was never the subject of any other publication. In particular, a first subpart will focus on the

activity of LC neurons around cue onset and the outcome. Then, a second subpart will focus

on the temporal dynamics of the responses to the three task events, and of the modulation of

the rate of LC neurons by task parameters and behavioral variables. Third, a last subpart will

detail findings regarding the heterogeneity of LC neurons in the magnitude of their responses

to task events, and in the strength of the encoding of task parameters and behavioral

parameters. Finally, I will briefly discuss such findings, although more extensive discussion will

be provided in the very last part of this manuscript.

a) Activity of LC neurons around cue onset and the

outcome

(1) LC neurons responded to cue onset and the outcome

Besides displaying a strong response to action onset, LC neurons displayed a phasic

response to cue onset and to the outcome (reward delivery). I quantified the changes in firing

of each neuron by comparing spike counts in a fixed window around these two events of

interest ([0;300ms] after cue onset and [0;300ms] after outcome) and baseline activity ([0 ;

300ms] before cue onset). Practically, for each neuron, I performed a paired t-test between

the spike count computed in the window of interest and the spike count in the baseline

window. I then evaluated the reliability of rate changes at the population level using a second-

level analysis (t-test on the T values from neuron-by neuron t-tests on rate). As shown on

Figure 41, the population of LC neurons displayed a significant activation in response to cue

onset and the outcome (cue: t(74)=5.37; p<0.001; outcome: t(74)=6.94; p<0.001). Out of 75

LC units, 23 (30.6%) displayed a significant response to the cue, and 39 (52%) at the trial

outcome. On average, the LC neurons increased their activity (compared to pre-cue baseline

firing) by 21.7% after cue onset and by 25.4% after the outcome.

Experimental results

154

Figure 41: Summary of the mean variations in rate for the 75 recorded neurons around cue

onset (A, B) and the outcome (C, D).

(A,C): Sliding window analyses of rate. Each line of the plot corresponds to the mean rate of

one individual neuron (y-axis: neuron number, N°) across trials at each time point (middle of

the time window used for rate assessment). The color code used is displayed on the right.

(A) Sliding window analysis of rate performed around cue onset. Just after cue onset, rate

increased for most neurons.

(B) Density plot of the T values from the T tests comparing the neuron by neuron rate in the

[0;300ms] epoch after cue onset to baseline rate in the [-300;0] epoch before cue onset. In that

Experimental results

155

epoch, T values are significantly greater than zero, indicating an increase in rate across the

population compared to baseline rate.

(C) Sliding window analysis of rate performed around the outcome (reward delivery). Just after

the outcome, rate increases for most neurons compared to the baseline rate.

(D) Density plot of the T values from the T tests comparing the neuron by neuron rate in the

[0;300ms] epoch after the outcome to baseline rate in the [-300;0] epoch before cue onset. In

the early epoch, T values are significantly greater than zero, indicating an increase in rate

across the population compared to baseline rate.

(2) Modulation of LC responses by task parameters at cue

onset and the outcome

Besides the strong changes in activity related to cue onset and the outcome, the firing

of LC neurons around cue onset and the outcome was also modulated across the 9 task

conditions, defined both by the reward magnitude and the delay duration (Figure 42).

I quantified the parametric effects of Reward and Delay (three levels each) on spike

counts around these events of interest using generalized linear models (GLMs). I counted

spikes in a [0;700ms] window after cue onset and a [0;500ms] window after trial outcome,

based on raster inspection. Spike counts of each unit were fitted with a GLM to evaluate the

relative weight of Reward (βR) and Delay (βD) modulation. For each event, I used a similar

approach as that described above (Bornert and Bouret, in prep) for event-related changes in

firing rate: a neuron-by-neuron analysis to evaluate the strength of the parametric effects in

individual units and a second-level population analysis to capture the coherence of these

effects across the population (Figure 42).

At cue onset, LC neurons displayed a significant modulation by task parameters (Figure

42A). The positive influence of reward was significant for 45/75 neurons. It was also significant

at the population level, since the distribution of beta weights for the Reward parameter (βR)

was significantly positive (t(74)=11.55; p<0.001). Delay had a significant negative effect (βD)

for 33/75 neurons and the second-level analysis confirmed that this effect was significant at

the level of the population (t(74)=-6.57; p<0.001). I used a paired t test to compare the

magnitude of the effects of Reward and Delay (absolute value of βR and βD) and the difference

did not reach significant (p>0.05).

Experimental results

156

Figure 42: Summary of the modulation of rate by task parameters in large time intervals

around cue onset and the outcome

(A) Density plot of the beta values from the GLM for spike count with Reward (beta values in

green) and Delay (beta value in blue) as parameters, in the [0;700ms] epoch after cue onset.

The mean of the beta value for Reward was significantly greater than zero while the mean of

the beta values for Delay was significantly lower than zero i.e. Reward had a positive effect on

spike count and Delay had a negative effect on spike count after cue onset.

(B) Density plot of the beta values from the GLM for spike count with Reward size (beta values

in green) and Delay (beta value in blue) as parameters, in the [0;500ms] epoch after the

outcome. The mean of the beta values for Reward was not significantly different from zero

while the mean of the beta values for Delay was significantly greater than zero i.e. Delay had

a positive effect on spike count after the outcome but we could not find any consistent effect

of Reward.

At the outcome, the population of LC neurons showed only a significant positive

modulation by Delay (t(74)=6.54; p<0.001) but the effect of Reward was not significant

(p>0.05) (Figure 42B). At the level of individual neurons, only 8/75 neurons showed a

significant effect of Reward, and it was negative for 3 of them. By contrast, 41/75 neurons

showed a significant positive effect of Delay. On average, the magnitude of the Delay

modulation was greater than that of Reward modulation (t(74)=8.45; p<0.001).

In summary, the firing of LC neurons around cue onset and the outcome displayed a

significant modulation by task parameters. At the cue, firing rate was modulated positively by

(B)

Experimental results

157

the Reward and negatively by the Delay, with no significant difference between the effects’

magnitude. At the trial outcome, the influence of task parameters on LC activity reversed:

Reward decreased LC firing whereas Delay increased it. Moreover, the relative magnitude of

the Delay modulation increased compared to that of Reward. In that, the modulation of LC

activity around the outcome of the trial resembles that observed just prior to action onset

(Bornert and Bouret, in prep).

(3) Modulation of LC activity by behavior

(a) Cue-evoked activity and willingness to work

At the time of cue onset, LC neurons encoded information about Reward: its size,

positively, and its Delay, negatively. Given the similarity between the influence of task

parameters on LC activity and willingness to work, I tested the direct relation between these

two variables. Note that willingness to work here refers to the proportion of acceptance of

each condition, by session. Indeed, I used acceptance rates, a measure of the reluctance of

the monkeys to engage in the trials, as a proxy for the subjective cost of engaging in specific

conditions, and tested whether LC neurons encoded this measure at all, and over and above

task parameters (Reward and Delay).

First, I regressed the spike count in a large post-cue epoch ([0;700ms]) on willingness

to work alone, using a simple GLM for each individual neuron. I had to exclude 12/75 neurons

recorded during sessions in which the monkeys had reached 100% engagement. Nine of the

remaining 54 neurons exhibited a significant positive effect of willingness to work, and a

second level analysis revealed that this positive effect was consistent across the population (t

test: t(53)=8.756; p<0.001). I then tested whether this effect was merely confounded by a joint

relation with task parameters, by including Reward and Delay as covariates of willingness to

work in the regression. The positive effect of willingness to work was significant for 12/54

units and the second level analysis indicated that the positive effect was still marginally

significant across the population (t test: t(53)=1.943; p=0.057).

Importantly, these effects were not present in the baseline rate i.e. before cue onset,

meaning that the encoding of willingness to work was dependent on the information about

the condition of the trial provided by the cue. Given the marginal result of the statistical test,

it remains difficult to draw a strong conclusion, yet such results are compatible with the idea

that the activity of LC neurons at the cue is related to the monkeys’ willingness to work.

Experimental results

158

(b) Real delay and outcome activity

In our task, Delay condition (1,2 or 3), an information provided by the cue, was only

partially informative of the time that would elapse between action onset and reward delivery.

Indeed, each Delay category corresponded to a time range in which the actual delay could fall.

To assess the influence of the amount of time elapsed between action and reward delivery

(actual delay) on the magnitude of the outcome (reward)-evoked activity, I regressed outcome

evoked firing rates in the [0;500ms] epoch after the outcome using a GLM with the actual

delay (time elapsed). I included Delay category (1, 2 or 3) as co-regressor, to examine the

influence of influence of actual delay (experienced at the outcome) beyond the Delay category

(known from trial onset). In these conditions, 9/75 neurons exhibited a significant effect of

actual delay (5 negative, 4 positive), and the second level analysis indicated that actual delay

did not have a consistent effect at the population level (t test against zero: p>0.05).

b) Dissection of the temporal dynamics of parameter

encoding relative to activity changes

I used a sliding window procedure to capture the dynamics of firing rate changes not

only within a trial (how firing rate changed from baseline in response to individual task event)

but also across trials (how event-related firing rate varied across the 9 task conditions, defined

by levels of Reward and Delay).

Practically, for rate change latencies, I used the rate computed in the sliding window

procedure (see methods section from Bornert and Bouret, in prep) and I performed a

systematic one-sided Wilcoxon test (alternative hypothesis: greater to detect the increase in

rate) between the rate in each window and the baseline rate (10 first windows of the sliding

window procedure). I determined the position of the first three consecutive windows with a

significant result. The latencies reported correspond to the middle of the first of these three

windows. For rate peak latency, I looked for the position of the window of the rate assessment

sliding window procedure with the highest spike count. I report the timing of the middle of

that window. Regarding the latency to the detection of a modulation of rate by task

parameters and behavioral variables, I used the beta value from the GLM sliding window

procedure (see methods section from Bornert and Bouret, in prep). I identified the latency to

Experimental results

159

the effects of each parameter by determining the position of the first three consecutive

windows of time with a significant coefficient for this parameter. I then report the middle of

the first of these windows as latency to the effect. The results of these analyses, i.e. latencies,

are provided in Table 1.

Table 1: Latencies of the firing rate changes and task parameter encoding at cue onset, action

onset and the outcome. The latencies indicated correspond to the median and confidence

interval of the position of the windows in which a change was detected.

Task event Activation

latency

Peak

latency

Reward modulation

latency

Delay modulation

latency

Cue onset 0

(-25; 75)

75

( 50; 175)

150

(125; 225)

350

(250; 450)

Bar release -250

(-325;-200)

-100

(-125;-75)

-512.5

(-612.5; -143.75)

-187.5

(-400; -150)

Outcome 0

(-25; 25)

125

(75; 150)

200

(162.5; 375)

100

(100; 100)

At the cue onset, the modulation of LC activity as a function of expected Reward (Figure

43A&C) and Delay (Figure 43B&C) levels appeared later than the average change in firing rate.

Indeed, the average latency of Reward and Delay effects were significantly longer than the

average peak latency (Wilcoxon signed rank test with continuity correction: V=276, p<0.001

and V=70.5, p<0.001; for Reward and Delay, respectively). In addition, the timing of the

Reward and Delay modulation differed in time, since the average latency of the Reward

modulation was significantly shorter than that of the Delay modulation (V=78.5; p<0.001). This

suggests that the source of LC inputs responsible from the global activation after cue onset

might differ, at least in time, from those responsible for Reward and Delay modulation.

Experimental results

160

Rew

ard

D

elay

Experimental results

161

Figure 43: Summary of the dynamics of the modulation of rate by task parameters (Reward

and Delay) around cue onset (A,B,C), action onset (D, E, F) and the outcome (G, H, I).

(A, B, D, E, G, H) Plot of the betas for Reward (A, D, G) and Delay (B, E, H) in the GLM for spike

count across time at cue onset (A, B), action onset (D, E) and the outcome (G, H), calculated

via a sliding window procedure (window size=200ms, step=25ms). One line corresponds to the

betas for one individual neuron at each time point (middle of the window of analysis). The color

scale is displayed below the graphs. The green dots represent the latency to the onset of the

effects.

(C, F, H) Plot of the mean rate and of the mean of the betas for Reward (green) and Delay (blue)

across neurons across time around cue onset (C), action onset (F), and the outcome (H).

At cue onset, Reward had a positive effect on rate (A,C) and Delay had a negative effect on

rate (A, C). (C) The latency to the peak in rate was shorter than the latency of the effect of

Reward which was shorter than that of the effects of Delay.

Before action onset, Reward had a small negative effect on rate (D,F) and Delay had a positive

effect on rate (E, F). (F) The effects of Reward and Delay appeared earlier in time than the peak

in rate.

At the outcome, Reward had a small negative effect on rate (G,I) and Delay had a strong

positive effect on rate (H, I). (I) The latency to the peak in rate and to the beginning of Delay

effects were not significantly different while the latency to Reward effects was greater than

the latency to the peak in rate.

At the bar release, the latencies of the Reward (Figure 43D&F) and Delay effects (Figure

43E&F) were indistinguishable (p>0.05). In addition, the effects of both Reward and Delay

appeared earlier than the peak latency of the firing rate modulation (Reward: V=16; p=0.0016;

Delay: V=23; p<0.001) but their latency was comparable to that of the onset of the rate effects

(p>0.05 for both Reward and Delay). In other words, the dynamics of the activation of LC

neurons when monkeys initiated the bar release was similar to the dynamics of the

modulation by Reward and Delay factors.

At the trial outcome, the Delay modulation (Figure 43H&I) appeared after the

beginning of the increase in rate (V=1178; p<0.001) and I could not find a difference in

latencies between the beginning of Delay effects and of the peak in rate (p>0.005), indicating

Experimental results

162

that these may be simultaneous. The effects of Reward (Figure 43G&I) began significantly later

that the peak in rate (V=188, p<0.001), but this effect was very small.

In summary, the complex dynamics of cue-evoked activity indicates that LC neurons

received an initial non-specific excitatory input, followed first by a positive modulation by

Reward information, and later by a negative modulation by Delay information. By contrast, at

the bar release and the trial outcome, the increase in firing rate occurred at the same time as

the positive modulation by Delay information and the negative modulation by Reward

information.

c) Intrinsic neuronal excitability shaped the activation

in response to task events and the encoding of task

parameters and behavioral variables

(1) Higher baseline was associated to higher peak of activity

Total and colleagues (2018) have reported a certain level of heterogeneity across LC

neurons regarding their intrinsic excitability in vivo, and such differences were found to relate

to different projection patterns, and therefore differential implications in cognitive processes

(Uematsu et al., 2017, 2015). In our dataset, I looked for heterogeneity in baseline rate across

neurons, and related differences in baseline rate to differences in activation strength around

task events of interest (cue, action, outcome), and to differences in task parameter encoding

strength.

Even if the response pattern of the population of LC neurons was relatively

homogenous (Figure 41), response magnitude to each of the task events varied across the 75

recorded neurons. I reasoned that if the variability in response magnitude across neurons was

simply related to their global excitability, the magnitude of evoked responses should be

predicted by the neuron baseline activity, across the population (Figure 44A,E,I).

In practice, for each neuron, I identified the moment at which rate was maximal around

the onset of the three task events, using a sliding window procedure or spike counting (see

methods of Bornert and Bouret, in prep). I computed the mean spike count of each neuron at

the peak of its firing rate (i.e. in the time window of the sliding window procedure with the

highest mean spike count), for all three task events, to evaluate the strength of the excitatory

responses. For each neuron, I evaluated baseline activity using spike counts before cue onset

Experimental results

163

([-300;0ms] epoch before cue onset). After computing the strength of the excitatory responses

for each neuron, I evaluated their correlation with baseline rate across the population of all

75 LC units. Note that similar results were obtained if the window used for evaluating baseline

rate was systematically taken before the activation in response to the event of interest, rather

than taking the same reference for all events in the inter-trial interval. For simplicity, results

reported below use a single mean baseline rate value per neuron, computed before cue onset.

Figure 44: Summary of the correlations between baseline rate and the strength of the response

to task events and of modulation of rate by task parameters around task events.

(A, E, I) Correlations between baseline rate (pre-cue rate) and post cue (A), pre-action (E) and

post outcome (I) peak rate. Baseline rate and peak rate are positively correlated.

Experimental results

164

(B, C, D, F, G, H, J, K) Correlations between baseline rate (pre-cue) and the maximum of the

absolute value of betas for Reward (B, F, J), Delay (C, G, K), willingness to work (D), and RT (H)

across time by neuron. For all regressors, the maximum of their effect size is positively

correlated to baseline rate.

(L) Correlation between maximum strength of effects of Reward and Delay on rate after cue

onset. The maximum strength of effect of Reward was positively correlated to the maximum

strength of Delay effects across neurons.

R: Reward; E: Delay; WTW: willingness to work

At cue onset, the rate at the peak was both positively correlated with the baseline rate

(R=0.826; t(73)=12.549; p<0.001). I also found positive correlations between pre-action peak

rate and baseline rate (R=0.740; t(73)=9.41; p<0.001) and between baseline rate and post

outcome peak rate (R=0.888; t(73)=16.514; p<0.001). These results imply that neurons with

higher baseline rates had higher firing rates during periods of event related activation.

(2) Higher baseline was associated to higher signal-to-noise

ratio

Classical theories of tonic versus phasic LC activity modes state that high tonic activity,

i.e. high LC baseline rate is generally associated to low signal to noise ratio in LC activity, i.e.

small differences between baseline rate and evoked responses, with the opposite pattern in

the phasic mode (Aston-Jones and Cohen, 2005). This theory focuses on within-neuron

changes in baseline firing. Here I question whether it can apply across neurons, i.e. whether

high-baseline neurons tend to have a smaller difference between baseline rate and rate at the

maximum of evoked activation. The opposite would be predicted from excitability accounts

of neuron-by-neuron excitability differences. If neurons that are more excitable have a higher

baseline rate, they should also respond more strongly to events. This analysis is

complementary to the one presented in the previous paragraph. Indeed, the direct correlation

between baseline rate and peak rate could have been driven by baseline differences, and

cannot exclude the possibility that beyond this baseline effect, neurons with a higher baseline

display a smaller increase in rate than those with a lower baseline. This is illustrated by Figure

45.

Experimental results

165

(A) (B)

Figure 45: Two alternatives for the relationship between baseline rate and signal-to noise ratio,

represented using schematic representations of the mean rate of virtual example units across

time around an event of interest.

(A) Excitability account: increased baseline leads to increased signal to noise ratio, i.e.

increased difference between peak rate and baseline. Peak rate (without subtracting baseline

rate) and baseline rate are positively correlated.

(B) Gain theory account: higher baseline corresponds to lower signal-to-noise ratio.

Nonetheless, peak rate and baseline rate are correlated.

I therefore examined whether there was a correlation between baseline rate and the

amplitude of the evoked change in firing from baseline. I measured the absolute value of the

difference between baseline rate and peak rate, and evaluated the strength of the correlation

between each of these values and baseline firing across neurons. The magnitude of excitatory

responses displayed a significant positive correlation with baseline rate across neurons

(respectively R=0.51; t(73)=5.1; p<0.001). This indicates that besides having generally higher

rates throughout the task, neurons with higher baseline rates responded to task events more

strongly, exhibiting stronger activations.

Rate Rate

High baseline

Middle

baseline

Low baseline

Time Time

Signal/noise Signal/noise

Experimental results

166

(3) Higher baseline was associated to stronger encoding of

task parameters and behavioral variables

Finally, I examined the relation between baseline rate and the encoding of task

parameters (Reward-Delay conditions, Figure 44B,C,F,G,J,K) as well as event-related

behavioral measures (willingness to work at the cue, Figure 44D, RT at the action, Figure 44H)

across this sample of LC neurons. In order to relate the effect sizes of Reward, Delay, RT and

willingness to work to baseline rate and to each other, I determined for each neuron the

window of the GLM for spike count sliding window analysis corresponding to the extremal

value (maximum of the absolute value) of the coefficients for each of these parameters. In

other words, I measured the relation between baseline rate and strength of encoding, by

taking the maximum of the absolute value of each regression coefficient. The results of this

analysis are reported in Table 2.

Table 2: Results for the Pearson’s correlations of baseline rate with effect sizes (maximum of

the absolute values of coefficients from the spike count GLM) for reward, delay, RT and

willingness to work, at cue onset, action onset and the outcome.

Correlation of

baseline rate

with …

… the maximum of

absolute beta

reward

… the maximum of

absolute beta delay

… the maximum of

absolute beta RT

… the maximum of

absolute beta WTW

Cue onset R = 0.75

t(72) = 9.71

p < 0.001

R = 0.79

t(72) = 10.86

p < 0.001

R = 0.27

t(67)=2.32

p=0.023

Action onset R = 0.56

t(72) = 5.84

p < 0.001

R = 0.64

t(72) = 11.65

p < 0.001

R = 0.69

t(72) = 4.50

p < 0.001

Outcome R = 0.54

t(72) = 5.40

p<0.001

R = 0.45

t(72) = 4.90

p < 0.001

Experimental results

167

Around all three studied task events, effect sizes i.e. encoding strengths were positively

correlated to baseline rate. This indicates that the variability in baseline rate across neurons

could account for some of the variability in sensitivity to task parameters and behavior.

Beyond confirming the previous result that neurons with higher baseline rates are more

sensitive to activating influences, this demonstrates that they are also providing a stronger

input about task events and behavioral responses.

(4) Encoding of Reward discounted value

Finally, I examined whether Reward discounted value, approximated by the difference

between Reward and Delay, was reliably encoded by the LC neurons at the time of the cue.

Specifically, I looked for a negative correlation between the neuron-by-neuron maximal

effects of Reward and of Delay (extrema of the regression coefficients across time around the

three task events, with its sign) (Figure 44I). I indeed found a significant negative correlation

(R=-0.75; t(73)=-9.56; p<0.001), yet this correlation could have been artificially created by the

positive correlations between regression coefficients and rate. Therefore, I then fit a GLM for

the neuron-by-neuron maximal effect of Delay, with the neuron-by-neuron maximal effect of

Reward and baseline rate as parameters. Over and above the effect of baseline rate (β=0.11;

p=0.012), the coefficient for the maximal effect of Reward on the maximal effect of Delay was

significantly negative (β=-0.55; p<0.001). Such results were not replicated at action onset and

the outcome (p>0.05 for both correlations).

3. Discussion of additional results and of study limitations

LC neurons displayed a transient response to the cues signaling the onset of a trial and

providing information about that trial (here, Reward and Delay). As in previous studies, the

strength of the response to cue onset was relatively modest, compared to the strong

activation at the bar release, but the magnitude of this response was strongly modulated by

task parameters (Bouret and Richmond, 2009; Varazzani et al., 2015). In line with previous

work, the intensity of the initial phase of the activation was equivalent across conditions, and

LC neurons only started to discriminate across reward conditions about 100ms after the onset

of the activation (Bouret and Richmond, 2015). Interestingly, the modulation by the

Experimental results

168

information about delay duration appeared even later (about 200ms after the beginning of

reward modulation). Although Varazzani and colleagues (2015) did not examine this directly

in their study on the effort/reward trade-off, the encoding of reward after cue onset did

emerge earlier than that of effort (Varazzani et al., 2015, fig. 5C).

The distinct timing of Reward- and Delay-related effects on activity suggests that the

corresponding modulation of LC activity at the cue could be triggered by separate inputs. But

since the dynamics with which such information is processed remains unclear, LC neurons may

still be contributing to the integration of these parameters, for instance assuming temporal

summation in target areas. This ability to integrate Reward and Delay inputs in a coherent

fashion may account for the relation between LC activity and willingness to work (also

described in Jahn et al., 2020). Indeed, since the effect of condition value, approximated using

engagement probabilities, on cue-evoked activity decreased when Reward and Delay were

added as covariates in the model, LC activity might very well play a role at the interface

between processing task-related information and modulating engagement.

At the time of the outcome of trials, after the delay, when the reward previously

announced by the cue was delivered, LC neurons also displayed a strong activation.

Noticeably, they strongly encoded the Delay category, positively, but not Reward size.

Conversely to what was observed at cue onset, this pattern excludes an encoding of the

discounted value of the Reward. Along the same lines, there is no reason to believe (but no

reason to exclude either) an interpretation in terms of prediction error. This increased LC

activation for longer delays could be related to the general level of salience of the outcome,

in relation to the level of surprise (Aston-Jones and Bloom, 1981; Bouret and Sara, 2004;

Vankov et al., 1995). Indeed, assuming that estimating the timing of the outcome becomes

more difficult as delay increases, longer delays could make reward delivery more surprising,

and therefore more salient. Again, further experiments would be necessary to clarify that

point.

In line with previous studies, the baseline firing rate of the recorded LC neurons

remained stable during the course of the experiments, as long as monkeys remained awake

(Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004; Clayton et al., 2004; Kalwani et al.,

2014). Interestingly, the difference in baseline firing rate across LC neurons was strongly

Experimental results

169

predictive of the differences in response magnitude, both in terms of event-related changes

in firing rate (within trials) and in terms of parametric modulation by task parameters or

behavioral responses (across trials). Neurons displaying the highest firing rates also displayed

the strongest responses, suggesting that individual LC neurons could readily be defined in

terms of global excitability i.e. general susceptibility to excitatory inputs. These results are at

odds with the intuition that baseline activity is noise and has a negative influence on evoked

activity. In the light of the recent findings in rats by Chandler et al. (2014), who demonstrated

that LC neurons projecting to frontal cortices were more excitable than neurons projecting to

the motor cortex, it would be crucial to verify if this relationship is also valid in monkeys. If it

were the case, that would provide a way to identify LC neurons putatively projecting to the

frontal cortex based on their firing and assess other functional properties, such as the relation

with executive functions, and thereby further investigate the functional heterogeneity of the

LC.

More generally speaking, even if this study clearly lacks that level of anatomo-

functional characterization, these results clearly support the emerging idea that the LC is

functionally heterogeneous (e.g. Chandler et al., 2014; Schwarz and Luo, 2015). Moreover, LC

units seem to be grouped into distinct functional modules displaying context-dependent

activity (Likhtik and Johansen, 2019; Uematsu et al., 2017). In that frame, any

neurophysiological feature that could be used to characterize individual LC units in behaving

monkeys might be useful to assess the relation between groups of LC neurons (based on these

features) and behavioral/cognitive contexts and thereby provide a useful complement to

experiments conducted in rodents.

Despite such heterogeneity, LC neurons analyzed as a population seem to provide a

reliable encoding of both task parameters (Delay and Reward) and of behavioral parameters

(RT, willingness to work). For instance, around cue onset, the strength of the encoding of the

task parameters necessary for computing reward value (size and delay) were encoded in a

coherent fashion by LC neurons, ruling out the possibility that Delay and Reward information

were encoded by distinct neuronal populations, putatively sending axons to different brain

areas. Such a pattern is in contrast with what has been observed in the single-unit activity of

cortical areas believed to participate in value processing, such as the ventro-medial prefrontal

cortex (vmPFC, e.g. Hosokawa et al., 2013; Kennerley and Wallis, 2009b; San-Galli et al., 2018).

Experimental results

170

Despite providing valuable insight into the role of the LC in decision-making, and

especially control allocation into triggering costly actions, this study had several limitations.

First, decisions consisted in binary choices between engaging in trials or refusing them. Such

decisions could have been automatic instead of goal-directed. Moreover, placing cue

presentation during a period of waiting did not allow measuring the duration of decisions, at

least when the monkey decided to engage, as Jahn and colleagues (2020) have shown that

decisions to forego a trial can be timed using eye fixation breaks. These limitations are of

particular interest as LC neurons seemed to be activated by decisions to abandon trials (eye

fixation breaks, Jahn et al., 2020). Responses to cue onset could in fact be aligned onto

decisions whether to engage, and this could account for the relatively more variable latencies

of post-cues responses compared to per-action responses (see Fig. 3 in Bornert and Bouret, in

prep, and Figure 41A&C). The second study was designed in order to fill those gaps and

therefore complement previous results.

Experimental results

171

B. Study 2: LC neurons facilitate delayed effort

mobilization and monitoring in a two-alternative choice

task

1. Introduction

In the first study, we reported that LC neurons encoded the subjective difficulty of

triggering actions at the time of triggering them, and the difficulty of executing them while

executing them. We interpreted these modulations as a domain-general encoding of effort

production. Pharmacological studies pointed towards a beneficial effect of LC activity on effort

production, at least in the context of physical effort (Borderies et al., 2020; Jahn et al., 2018).

The dynamics of LC activity relative to effort production observed in our first study, combined

with the slow conduction velocity of LC axons, was incompatible with immediate

consequences on action-triggering and action execution. We suggested that LC activity around

effortful processes may have short-term (sub-second) but not immediate facilitative effects

on effort production, regardless of the nature of the challenge faced (physical or mental).

Moreover, based on studies showing a potential relation between LC activity and adjustments

of effort allocation (at least following errors, Murphy et al., 2016; Sellaro et al., 2015), we

proposed that LC activity may serve as a feedback signal regarding effort production costs and

consequences, causing adaptations of effort-allocating decisions. In order to test our

hypotheses, we recorded the activity of LC neurons in two-alternative effort-reward choice

tasks.

Besides confirming the findings regarding the domain-general encoding of effort by LC

neurons, our second study first aimed at testing the relation between LC activity around effort

production and later effort production. When presented with two alternatives differing in

terms of required effort and associated rewards, subjects choose between offers by making a

tradeoff between effort and reward. Consequently, if effort production was facilitated, the

effect of effort on binary choices should decrease, and vice versa, as was observed in Borderies

et al. (2020). The tasks that we designed required physical effort production (differential force

exertion on a grip). Physical effort facilitation should lead to an increased probability to choose

high-physical effort offers. Additionally, these paradigms allowed combining the assessment

of effort sensitivity of choices to an evaluation of the force exerted during the execution of

Experimental results

172

the chosen action. Within force conditions, increased exerted force should indicate facilitated

effort production, especially if it was combined to decreased effort sensitivity of choices.

Cognitive control exertion optimizes the decision process (Botvinick and Cohen, 2014;

Cohen et al., 1990; Missier et al., 2012). In that sense, in tasks offering overt decisions between

several alternatives such as our two-alternative choice tasks, if LC activity facilitated effort

production in general and therefore cognitive control exertion, we would expect that

following high LC activity, choices would be more accurate. Several studies have

complementarily demonstrated that cognitive control is crucial for decisions that go against a

bias (Epstein, 1994; Shenhav et al., 2013; Shiffrin and Schneider, 1977; Sloman, 1996).

Monkeys have a natural tendency towards avoiding offers requiring high effort production, as

shown by trial omissions in one-option tasks and two-alternative choice tasks (respectively:

Jahn et al., 2020; Varazzani et al., 2015; and Borderies et al., 2020), and binary choices in two-

alternative choice tasks (Borderies et al., 2020). In that sense, facilitated cognitive control

exertion could alternatively cause decreases in the sensitivity of choices to effort, i.e. increase

the probability to choose high-effort options.

Second, this study aimed at exploring the role of LC activity, especially its modulations

by effort production, as a feedback for the effort allocating system. We theorized that LC

activity around the time of effort production in one trial would cause adjustments of decisions

to allocate effort in the next trial. Practically, in our two-alternative choice tasks, we expected

action-related LC activity to predict adjustments in effort-based binary choices in the next trial.

We expected that if LC neurons activated more around the action of trial N, signaling high

effort production, the next choice would have increased effort sensitivity (decreased

probability to choose the high-effort offer).

In a pilot study, we recorded LC neurons in a two-alternative choice task quite similar

to the one used in Borderies et al. (2020). It consisted in presenting two offers, i.e. cues

indicating two force-reward combinations, during a waiting period, and then asking the

monkeys to execute the chosen action to obtain the chosen reward. One monkey was trained

on that task, and two dozen LC neurons were recorded, providing some preliminary

indications of the pattern of activity of LC units in such tasks.

Experimental results

173

Our initial paradigm, however, required optimization for several reasons. First, the

monkey had to materialize his choices by pressing a grip located on the side of presentation

of the chosen cue at the required force. Consequently, in each trial, the monkey could either

choose to keep his hands on the same grip as the one used in the previous trial or choose to

switch. He developed a strong bias towards keeping his hands on the same grip, thus

frequently ignoring cue information to make his choice.

As in the first study (Bornert and Bouret, in prep), we planned on using RT as a measure

of the subjective difficulty of triggering actions. Since our task involved a choice between

alternatives, RTs should also vary with choice difficulty and probably with the cognitive control

exerted to make the choice (Hunt et al., 2012; Krajbich et al., 2010; Krajbich and Rangel, 2011;

Milosavljevic et al., 2010; Morris et al., 2019; Philiastides and Ratcliff, 2013). These effects

should be additive (Dmochowski and Norcia, 2015). However, in the pilot task, RTs were hardly

interpretable: in addition to effects of side of press (postural effects), RTs varied depending

on the previous hand position (previous grip pressed) relative to the current grip chosen. For

instance, if the monkey chose the left side option at trial N, he was much faster if he had

already used that grip at trial N-1, since he did not need to change hand position, compared

to trials in which he had used the right side grip at trial N-1. Additionally, the cues were

presented during a waiting period, which did not allow decision duration to be measured.

Finally, regarding willingness to work, since the task did not impose eye fixation (the

monkey had to examine both options, so gaze fixation was not an option), it was impossible

to assess whether the monkey looked at the cues and decided to refuse to engage, or did not

even look at the cues.

Consequently, we conceived a new paradigm, fixing the imperfections of the previous

one (see materials and methods, part II.B.2). Practically, monkeys were first asked to press on

a middle grip before being presented with a pair of offers, for which they had to press one of

the two side grips. This middle press had several functions: first, it removed the bias of the

monkey to press the same grip at trial N as he had at trial N-1. Second, in most trials, the

monkey would press the middle grip without knowing what the subsequent offers would be,

allowing us to know when he was willing to look at the cues, and made an actual decision

whether to engage in the rest of the trial. Third, since the LC is believed to participate in effort

production for action triggering, execution and action consequences monitoring, designing a

Experimental results

174

task with several steps allowed testing whether LC activity at the first step (middle press) could

influence decisions to engage and binary choices and action performance at the second step

(choice between offers and associated press). Additionally, in this new task, the cue was no

longer presented during a waiting period, i.e. the cue with the two offers was also the go-

signal and the monkey could trigger the chosen option as soon as he had reached a choice.

This would allow timing the duration of the RT consisting of the sum of the durations of the

action selection and the action triggering periods. Gaze position was recorded throughout the

trials and I designed an eye-tracking procedure allowing identification of the timing of the end

of the decision process (see materials and methods, part II.B.2).

2. Materials and methods

a) Animals

One rhesus macaque (Macaca mulatta), monkey J (male, 12kg), was included in this

study. During testing days, the monkey received water as reward, and on non-testing days, he

received amounts of water matching his physiological needs. The experimental procedures

were designed in association with the veterinarians of the ICM (Brain and Spine institute),

approved by the Regional Ethical Committee for Animal Experiment (CREEA IDF n°3) and

performed in compliance with the European Community Council Directives (86/609/EEC).

b) Behavioral paradigm

The monkey was extensively trained on a two-alternative forced choice task (Figure

46A). During sessions, he squatted in a primate chair, in front of a computer screen. The

monkey was head restrained, to allow eye-tracking and electrophysiological recordings. Three

force-sensitive grips (middle, left and right grips, M2E, France) were installed on the anterior

part of the chair, within reach from the hands of the monkey. A metal straw delivering water

rewards was placed between his lips.

Trials begun with the presentation of a central red dot (wait-signal) of variable duration

(100±50ms), after which it turned green and remained green for 3000ms maximum. During

this period, the monkey could press on the middle grip, and any small force exertion was

sufficient. If the monkey pressed the grip, the dot turned blue, and remained blue for a

variable duration (1000±100ms). This duration was imposed to be able differentiate middle

Experimental results

175

action-related LC activity from cues onset related activity. After that delay, the central blue

dot disappeared and two cues appeared on the sides of the screen, one on the right side and

one of the left side, as well as one green dot per side. No waiting period was imposed following

cues presentation, and the monkey could virtually immediately trigger the chosen action for

the whole duration of cues presentation (2250±250ms). The action consisted in pressing the

grip located on the side corresponding to the side of presentation of the chosen cue at the

level of force instructed by the cue. Once a press had been detected on one grip, the monkey

had 1000ms to reach the force threshold imposed by the condition. Levels of force had been

calibrated through training to identify combinations that did affect choices but did not

demotivate the monkey. If the force produced reached the correct threshold of force, the

green dot on the chosen side turned blue, and the force had to be maintained for an additional

80ms. Inter-trial intervals were 475±25ms long.

Each cue consisted of a reward cue (number of blue drops) and a force cue (rectangle

of variable size, Figure 46B). Reward and Force were orthogonalized and left and right offers

were picked from a pseudorandom distribution.

(A) (B)

Figure 46: Two-alternative choice task.

(A) The task consisted in two steps: pressing a middle grip, after what two offers were

presented, and choosing between these options. Choice was materialized by pressing on the

grip located on the same side as the chosen option.

(B) Offers consisted in a combination of Reward (1,2 or 4 drops of water) and requested Force

(1,2 or 3). Each combination was associated to a specific cue, which the monkey had learned

to use.

Erroneous press

Omission

Experimental results

176

Failure to produce the requested force or to sufficiently maintain it (erroneous press),

failure to produce a press on the middle grip at the first step, or on any side grip at the second

step (omission), and presses on any grip during the red dot presentation led to the trial being

restarted (from the middle press step). Consequently, the same offers were presented at the

second step. If the monkey had reached the cue presentation in the failed trial, he already

knew what offers he would get, and the cues therefore provided no new information

(repeated trial). If he had made no error at the previous stage or made an error that did not

allow him to see the cues (omission of middle grip press), the cues carried novel information

(non-repeated trial).

c) MRI, surgery and recordings

A 3T MRI of the monkey’s brain was performed under general anesthesia to determine

the position of the LC, based on previously described anatomical criteria (see part I.B.1.a) for

review and Bornert and Bouret, in prep, materials and methods for extensive description). A

few days after the MRI, a headpost and two recording chambers, each located on top of the

theoretical position of the LC in each hemisphere, were placed on the skull of the monkey

during a sterile surgical procedure under general anesthesia. In this study, only the right side

chamber was used, the second had been placed in prevision of bilateral DREADDs injections.

The right side chamber had a 14° angle to the vertical plan. After a few weeks of recuperation,

a sterile craniotomy was performed in the right side chamber, preserving the dura, to allow

electrode descent in that chamber.

Electrophysiological methods were the same as described in Bornert and Bouret, in

prep (materials and methods) and in several other papers (Bouret et al., 2012; Bouret and

Richmond, 2015, 2009; Jahn et al., 2020; Varazzani et al., 2015). In brief, before each recording

session, I cleaned the recording chamber, and placed a grid with holes 1mm apart in the

chamber. I introduced a metal guide tube (CRIST, 49mm long) in the grid hole that was

identified as positioned on top of the body of the LC in the right hemisphere (Figure 47A). In

that tube, I inserted a tungsten micro-electrode (UEWLEHSM3PNM, FHC, Bowdoin, ME). The

electrode was then lowered using an oil hydraulic Microdrive (Narishige).

Experimental results

177

(A)

(B)

Figure 47: Electrophysiological recordings: location in the brain and pharmacological

validation.

(A) MRI of the brain of Monkey J (T2 contrast, visualized in Brainsight software). The grey

cylinders on top of the skull are the recording chambers. In the right side chamber (right side),

a grid had been placed (grey stripes in chamber). The trajectory used for the recordings is

shown in green. The LC was approximately located 63mm below the top of the grid.

(B) Discharge of example LC unit after clonidine injection (20μg/kg, IM, green arrow). After 5

minutes (time in x-axis), rate (y-axis) started decreasing. The monkey was drowsy (red arrow)

after about 15 minutes, and LC neuron rate dropped after 20 minutes. During the period of

Rat

e (H

z)

3

6

Time (s)

Injection

Drowsy

Opened eyes

* *

* *

* *

Experimental results

178

drowsiness, LC activity increased just before increases in arousal (eyes opening, purple stars).

After about an hour, LC activity increased again and the monkey woke up shortly after.

During the descent, the pattern of discharge and of response to events and stimuli

(sounds, touch, movement) of encountered neurons were used in combination of the MRI

scan to determine the position of the tip of the electrode in the brain. LC neurons were

identified using extensively described electrophysiological criteria (low rate of spontaneous

activity (<4Hz), broad waveforms (>0.6ms until the first peak), and characteristic activation-

pause response to brief auditory or tactile stimuli (e.g. calling monkey’s name, knocking on

the door, or clapping hands). I confirmed this identification by performing clonidine injections

(15-20μg/kg, IM, SIGMA). Under clonidine, the monkey became drowsy. Drowsiness preceded

a decrease the activity of concomitantly recorded LC neurons (Figure 47B). Once LC activity

had become low, after about 20 minutes, increases in activity preceded periods of decreased

drowsiness (eye opening, small movements). The activity of neurons increased before the

monkey woke up.

d) Data analysis

All analyses were performed using the Matlab software. For all analyses, offer

parameters (Rewards, Forces) and behavioral parameters (engagement rate, response time,

exerted force) were z-scored before fitting models.

The procedure for RT determination for middle presses and side presses was identical

to the procedure described in the first study (Bornert and Bouret, in prep), presented above.

The time courses of the force exertion on each of the three grips with the RTs determined by

this method are shown in Figure 48.

The procedure for rate assessment across time (sliding window procedure) was the

same as the one used for the first study (Bornert and Bouret, in prep).

(1) Eye-tracking and determination of action-selection and

action-triggering time

In this task, gaze position (position of the right eye) was recorded in 32/34 sessions i.e.

for 50/54 recorded neurons (ISCAN system). I used the signal-detection task of Study 1, in

which the monkey had to detect that small dots switched from red to green and trigger a

Experimental results

179

press, to calibrate the relationship between the recorded signal and actual gaze position.

Practically, considering the small size of the dots and the unpredictable timing of the color

switch, I made the postulate that the monkey was necessarily looking at the dot when he

triggered his response if the response was correctly timed. Using trigonometry, I then inferred

the relation between pupil center vertical and horizontal eccentricity recorded by the eye-

tracking system and vertical and horizontal angles from screen center. Consequently, I could

determine within which intervals the recorded horizontal and vertical eccentricity of pupil

center should be to ensure that the monkey was looking at the left or right side offer. Note

that identical results were obtained by simply determining the position of the middle of the

screen (via calibration or by taking the mean of the horizontal eye position signal throughout

the whole session) and making the approximation that the monkey was looking at the right

side offer when he looked to the right of the screen center, and vice versa.

Figure 48: RT determination and decomposition: forces exerted in the two-alternative choice

task and eye-tracking procedure.

(D) Reaction Time (Go-signal to press)

Gaze alternates between options: action-selection time

Gaze locked on chosen option: action-triggering time

Action triggered

Right side press

G

O

+

C

U

E

S

Experimental results

180

(A-C) Time course of force exertion (baseline subtracted) after middle press onset (pink vertical

line) or side press onset (orange vertical line), on (A) the middle grip, (B) the left side grip and

(C) the right side grip. All sessions were pooled. The thickness of the lines of the plot

corresponds to one SEM above and below the mean exerted force. For the two side grips (B,C),

trials are separated according to the Force condition (force requirement).

(D) Schematic representation of the procedure based on eye-tracking data used to decompose

RT into action-selection and action-triggering time. After the Go-signal (green dot, here

represented as a green rectangle), during RT (red arrow), the monkey (brown ellipse, eyes as

black circles) usually looked alternatively at the two offers presented on screen (grey

rectangles with cues) for a certain amount of time (dashed orange line, gaze direction

extracted from right eye position represented by blue lines). Then, he locked his gaze onto the

option that he would choose (execute) until actually triggering the action (dotted orange line).

We refer to the time between cues onset and gaze locking onto the cue of the chosen offer as

“action-selection time” and to the time between gaze locking onto the cue of the chosen option

as “action-triggering time”.

To decompose RT, after determining it from the force exertion profiles, I first identified

periods of times during which the monkey looked at either cue during RTs. I made the

postulate that the monkey was making his choice (action-selection) for as long as he

alternately gazed at both offers. Then, when his gaze stopped alternating and locked onto one

offer, the monkey would be triggering the action he chose to execute (Figure 48D).

Consequently, I identified the timing of the last saccade onto the chosen option. The time

from cues onset to this saccade corresponded to action-selection time, while the time from

this saccade to action-triggering (detection of the beginning of the press) corresponded to

action-triggering time. One limitation of this technique comes from outlier trials, in which the

monkey either did not look at both cues (kept his gaze onto one cue until he responded), or

did not look at the chosen option when he triggered its execution. These trials were excluded

from analyses requiring the use of action-selection and action-triggering times.

(2) Value inference from choices

I used the choices between offers made by the monkey throughout sessions to

determine the mean subjective value of each offer i.e. combination of Reward and Force for

Experimental results

181

each session. Session-by-session, I fit a logistic regression for choices (0 if the monkey chose

the left side offer, 1 if he chose the right side offer) with right side Forces and Rewards minus

the left side Forces and Rewards as parameters. Indeed, considering the polarization of the

logistic regression, weights for left side offers had to have a change of sign:

Choice ~ logit ( c + βRr . RR + βFr . FR - βRl . RL - βFl . FL )

With c a constant term, RR the Reward of the right side offer, FR the Force of the right side

offer, RL the Reward of the left side offer, FL the Force of the left side offer, and β the associated

coefficients in the logistic regression for choices.

All the trials in which the monkey made a side press were included in the regressions.

For each session, the mean weight of the Forces and of the Rewards on subjective values was

thus determined (β in the previous formula). For each trial, I then inferred the value of the

offers from these weights:

Voffer LEFT = βRl . Roffer + βFl . Foffer

Voffer RIGHT= βRr . Roffer + βFr . Foffer

With Voffer LEFT and Voffer RIGHT the subjective value for the considered session of one specific

offer presented on the left or right side, respectively, as a function of its Reward (Roffer) and

Force (Foffer) characteristics and of the coefficients for Reward and Force on the corresponding

side (left side, respectively βRl and βFl, right side, respectively βRr and βFr).

3. Results

a) Behavior

One monkey was trained on a two-alternative choice task consisting of two steps (see

part II.B.2.b) and Figure 46 for details). The first was pressing on a middle grip, after what two

offers, a combination of Reward size and requested Force, were presented. The second step

was making a choice between the two offers and executing the chosen action, by pressing the

grip placed on the side corresponding to the side of presentation of the chosen offer.

Erroneous trials (pressed at the wrong time, incorrect pressing force, or omission) were

repeated. This means that, if the trial was aborted (omission) or failed (erroneous press) after

cue presentation, the cues had already been seen by the monkey, and until the trial was

Experimental results

182

completed, subsequent cue presentations carried no novel information. Behavior was

analyzed for the 34 recording sessions.

Throughout the analyses of the behavior, I looked for indications that the monkey

invested effort in the task, to decide to engage in actions, choose between offers, trigger

actions, and execute them, and examined the relationship between the different effortful

processes. Regarding cognitive control exertion to engage in actions and action-triggering, I

expected that the time taken to trigger the actions would scale negatively with the tendency

to disengage from the task and would increase throughout sessions. This would indicate that

this time could serve as a measure of the subjective difficulty of triggering actions, and possibly

of the cognitive control exerted.

Regarding binary choices between effort-reward combinations, I expected either one

of two possibilities. The time taken to make choices has been shown to reflect the cognitive

control invested in the decision (Hunt et al., 2012; Krajbich et al., 2010; Krajbich and Rangel,

2011; Milosavljevic et al., 2010; Morris et al., 2019; Philiastides and Ratcliff, 2013). In choice

tasks, behavior usually follows a speed-accuracy tradeoff: over and above choice difficulty, if

subjects take longer to decide, their decisions are usually more accurate (Chittka et al., 2009,

2003; Förster et al., 2003; Fudenberg et al., 2018). I looked for signatures of a speed-accuracy

trade-off in our task, i.e. for evidence that longer action-selection time was associated to more

accurate choices, via increased cognitive control exertion. Alternatively, cognitive control

exertion for the binary choice process may have reduced the strength of decision biases

(Epstein, 1994; Shenhav et al., 2013; Shiffrin and Schneider, 1977; Sloman, 1996). In that

frame, I would have expected that increased time spent on action-selection would lead to an

increased probability to choose high-effort offers, i.e. a decreased bias towards avoiding

them.

Regarding action execution, we defined effort as the process that mediated between

difficulty and performance. In other words, increased effort increases performance. Our task

inposed differential force execution on a grip to obtain a reward. Within force categories,

increasing force exertion increases the probability of success. In that sense, exerting effort to

execute actions should increase force exertion especially in high-force conditions. We

questioned the relation between physical effort production for action execution and cognitive

control exertion for engaging in the actions and triggering them. I expected that if engaging

in the actions and triggering them was subjectively more difficult, executing the action would

Experimental results

183

also be more difficult, and that less physical effort would be produced, resulting in decreased

force exertion and performance.

(1) Engagement in costly actions

(a) RT to the middle press represented the cost of

triggering the middle action and starting a trial

First, I analyzed behavior associated to triggering the middle press: engagement

(decisions whether to press the grip) and RT (time from go-signal to press onset). First, I aimed

at verifying the validity of RT as a proxy for the subjective difficulty of triggering the press on

the middle grip to start a trial. The distribution of middle press RTs, across all sessions, is

shown in Figure 49A. If middle press RT scaled with the difficulty to engage, it should have

increased throughout the session, as the monkey got tired, and in periods of time when the

monkey was less willing to engage.

To confirm that the cost of engaging in the middle press increased throughout the

session, I first verified that engagement in the middle press decreased throughout the session.

I pooled all the trials that were not repeated (in which the monkey did not see the cue before,

to avoid any potential value-based effects) of all the sessions and fit a logistic regression for

engagement with only trial number as parameter. The effect of trial number was significantly

negative (β=-0.15; p<10-144, Figure 49B), indicating that triggering a new trial was probably

more costly as the session went by. Consequently, if RT increased with increased cost to

trigger the middle action, I expected RT to increase throughout the session and to increase if

the current engagement rate was lower.

I then pooled all the trials in which the monkey pressed the middle grip during the

display of the go-signal, that were non-repeated trials. First, I fit a GLM for middle press RT

with only trial number as parameter. The effect of trial number was significantly positive

(β=0.11; p<10-22, Figure 49C). Second, I split sessions in halves (beginning and end), and

computed the engagement rate in the middle press in non-repeated trials for each half. Each

trial was therefore associated to the probability of engagement in its half of the session. I then

pooled all the non-repeated trials in which the monkey pressed the middle grip from all the

sessions, and fit a GLM for middle press RT with only block-wise engagement rate as

Experimental results

184

parameter. There was a negative effect of block-wise engagement rate on middle press RT

(β=-0.028; p=0.014, Figure 49D). These results provide two indications that RT to the middle

press could readily be used as a measure of the cost of triggering the middle action and

starting a trial in this task.

Figure 49: Behavior associated to the middle press was compatible with a positive relationship

between RT and the cost of triggering the middle press.

(A) Density function of middle press RT (bandwidth = 10). Most RTs are below 1 second.

(B) Logistic regression for engagement (whether the monkey pressed on the middle grip) with

trial number as parameter. Trial number had a negative effect on engagement, indicating that

the cost of engaging increased throughout the session.

Experimental results

185

(C,D) GLM for middle press RT with (C) trial number as parameter or (D) block-wise

engagement rate. (C) Trial number had a positive effect on RT, while (D) current engagement

rate had a negative effect, indicating that higher costs of triggering the action led to increased

RT.

*: p<0.05, ***: p<0.001, ctt: constant.

Overall, our results indicate that middle press RT scales with the subjective difficulty to

trigger the middle press, and therefore initiate new trials. Given that I focused on trials for

which the monkey did not know about the cues, this effect cannot be accounted for by effects

of the values of the offers from the choices.

(b) Decisions to engage based on cue information

Second, I focused on the second step requiring engagement: the two-alternative

choice. This novel paradigm allowed clearly identifying trials in which the monkey intended to

look at the cues (offers), and pressed the middle grip. After cues were presented, the monkey

could decide to make a choice (pick any option by pressing on any side grip) or to abort the

trial (no press, omission). I define choice engagement as the decisions whether to choose any

option offered (versus do nothing).

(i) Choice engagement increased with

increased sum of Rewards of options

I first studied the influence of offer parameters (offered Rewards and Forces) on choice

engagement in trials, trial-by-trial. In that aim, I compared the fit of different logistic

regressions for choice engagement (0, no press on side, 1, press on any side). Obviously, only

trials in which the monkey had made the middle press and therefore visualized the cues were

considered. Non-repeated trials from all the sessions in which the monkey pressed on the

middle grip and in which cues appeared were pooled. I expected Forces to have a negative

effect on choice engagement and Rewards to have a positive effect on choice engagement,

i.e. for the monkey to engage more in choices with high-value options. I compared three

models: a model with Rewards (left and right separately) and Forces (left and right separately)

as parameters, a model with Force minus Reward (left and right, separately), and a model with

the sum of Forces (left + right) and the sum of Rewards (left + right). I then compared their

Experimental results

186

Bayesian Information Criteria (BIC). The model including the sum of Forces and sum of

Rewards had the lowest BIC (BIC=1562.3), compared to the model including all the parameters

separately (BIC=1576.3) and to the model including the difference between Forces and

between Rewards on both sides (BIC=1574.7). The sum of the Rewards had a significant

positive effect on choice engagement on a trial-by-trial basis (β=0.71; p<10-13), and the sum of

Forces did not have any effect on choice engagement (β=-0.089; p=0.25, Figure 50A).

Figure 50: Decisions to engage in the choice in the two-alternative choice task.

(A) Coefficients in the logistic regression for choice engagement. Besides a strong bias towards

engaging, materialized by the significant constant (ctt) term, choice engagement was affected

positively by the sum of Rewards offered.

(B) Coefficients in the logistic regression for choice engagement, with RT of the middle press

added. Middle press RT had a significant negative effect on choice engagement.

ctt: constant term ; F: Force; R: Reward; *** : p<0.001; n.s.: non-significant, error bars

represent SEM.

Considering that these results suggest that the absolute weight of the sum of Rewards

on choice engagement was stronger than that of the sum of Forces, I then performed the same

logistic regression on trial-by-trial choice engagement, but session-by-session. Two sessions

had to be removed for that analysis due to a low proportion of trials in which the monkey did

not engage in choices. I compared the magnitude (absolute value of coefficients) for the sum

of Forces and the sum of Rewards, across sessions, using a paired t-test. This analysis

demonstrated that the magnitude of the effects of the sum of Rewards was larger than that

***

Experimental results

187

of the effects of the sum of Forces (t(31)=-3.68; p=0.001). In other words, the monkey placed

more weight on the total benefits (Rewards) offered in the trial than in the total costs (Forces)

when deciding to engage in choices.

Overall, these results indicate that the monkey engaged in trials more often when the

overall value, and especially the overall Rewards, of the offers were high, suggesting that

decreased overall value of offers increased the difficulty of engaging in the choice.

(i) Action-triggering time scaled with

difficulty to engage in the choice

Second, I examined the modulations of the time that the monkey took to trigger the

chosen action, and its relationship with the difficulty to engage in the choice. In this paradigm,

I recorded gaze position throughout trials. Using a calibration procedure, after cues onset, I

were able to identify periods of times during which the monkey looked at the right or left side

cue (see methods in part II.B.2.d)(1)). I define RT as the time elapsed between cues onset

(associated to the go-signal) and action onset (determined by finding the moment when the

force signal slope started changing, see procedure described in Bornert and Bouret, in prep,

materials and methods). Based on the literature on subcomponents of RT (Dmochowski and

Norcia, 2015), I made the hypothesis that RT was decomposable into two subparts: an action-

selection period and an action-triggering period, including the motor preparation (hand

positioning). I used eye-tracking to decompose RT into those two parts. After a period during

which gaze position alternated between the two offers (action-selection time), in most trials,

the monkey locked his gaze onto the chosen option. I refer to the period between gaze locking

and the beginning of the press on the chosen grip as the action-preparation period. The

duration of this period was highly correlated, negatively, to the duration of the action-

selection period, over and above effects of offer parameters and side of choice. For trials in

which the monkey never stopped looking at either option, or was not looking at the chosen

option while executing it, RT could not be decomposed into subcomponents. Additionally,

eye-tracking data was only available for 32/34 sessions. For the rest of this manuscript, every

analysis using action-triggering time and/or action-selection time therefore implicitly uses

Experimental results

188

only sessions and trials in which these two times could be determined. The distribution of RTs

and of their two subcomponents is shown in Figure 51A.

Figure 51: Action-triggering time increased with difficulty to engage in the choice

(A) Density plots of RTs (full line), action-selection times (dashed line) and action-triggering

time (dotted line).

(B) Coefficients in the GLM for action-triggering times. The chosen Reward had a negative

effect, while the chosen Force and the unchosen Force and Reward had positive effects. There

was a negative effect of action-selection time, meaning that the longer the monkey took to

decide on a choice, the faster he would be to trigger the chosen action. Additionally, Side had

Experimental results

189

a positive effect, indicating a longer duration of action-triggering time when the right side was

chosen.

(C) Coefficients in the GLM for action-triggering times. Besides the effects of offer parameters,

side and action-selection time, the choice engagement rate in the current trial block had a

negative effect on action-triggering time, indicating that increased difficulty to engage in the

choices led to increased time to trigger the chosen action.

ctt: constant term ; F: Force; R: Reward; *** : p<0.001; ** : p<0.01; * : <0.05 ; n.s.: non-

significant, error bars represent standard error of the mean (SEM).

I expected action-triggering time to increase with the difficulty of triggering the chosen

action. In order to examine that, I first fit a GLM for action-triggering time with task

parameters (chosen and unchosen Forces and Rewards), Side of press, and action-selection

period. All the trials in which the monkey made a choice, had looked at both cues, looked at

the chosen option at the time of the action, and in which the cue was unknown until

presentation (non-repeated trials) were included. The effect of the duration of the action-

selection period on the action-triggering period duration was significantly negative (β=-0.77;

p<10-100, Figure 51B), indicating that the monkey was faster to trigger the chosen action when

he had taken longer to choose between offers. Apart from that, the chosen Reward had a

quite strong negative effect (β=-0.23; p<10-119), and the chosen Force had a small positive

effect (β=0.028; p=0.0018). The unchosen Force and the unchosen Reward had a positive

effects (Force: β=0.030; p=0.0017, Reward: β=0.051; p<10-7), as well as Side (longer duration

for right side choices, β=0.089; p<10-6).

In order to confirm a link between action-triggering time and difficulty to engage in the

choice, I tested for a negative relationship between current engagement rate in choices and

action-triggering time. In an identical manner as what was done for middle press RT and

middle press engagement rate, sessions were cut in halves and choice engagement rate in

each half was determined. Each trial was therefore associated to a certain engagement rate

in its half of the session). Trials from all the sessions (same selection as previous GLM for

action-triggering time) were pooled. I performed the same GLM for action-triggering time as

before (with offer parameters, action-selection time and Side of press) but added block-wise

choice engagement rate. The effect of current choice engagement rate was significantly

Experimental results

190

negative (β=-0.044; p<10-6, Figure 51C), over and above the effects of offer parameters and of

side10. This indicates that the difficulty to engage in the choice scaled with action-triggering

time negatively, i.e. the monkey triggered the chosen action faster when deciding to engage

in the choice was less difficult.

(ii) Difficulty to engage in the choices scaled

with difficulty to engage in the middle press

Next, I tested whether the cost of triggering the action initiating the trial (middle press)

was related to the cost of engaging in the choice, over and above effects of task parameters.

Practically, I first examined whether current middle press engagement rate (within half of

session, see above for details) was correlated to current choice engagement rate. I pooled all

the non-repeated trials in which cues appeared and fit a logistic regression for choice

engagement with the sum of offered Forces, the sum of offered Rewards, and current middle

press engagement rate. Besides identical effects of the sum of Forces and of Rewards as

observed before (respectively β=-0.087; p=0.26; β=0.71; p<10-13), there was a positive effect

of current middle press engagement rate on choice engagement (β=0.16; p=0.043, Figure

52A). This provides evidence for the positive relationship between the difficulty to engage in

the middle press and in the choice.

Additionally, I expected middle press RT to have a positive effect on choice

engagement over and above effects of task parameters. I fit a logistic regression for choice

engagement (1 if the monkey made a choice i.e. pressed on one side grip, 0 otherwise) with

the sum of offered Forces, the sum of offered Rewards, and middle press RT as parameters.

Besides identical effects of the sum of Forces and of Rewards (respectively β=-0.087; p=0.27;

β=0.72; p<10-13), there was a negative effect of middle press RT (β=-0.25; p<10-4, Figure 52B)

on choice engagement. In other words, if the monkey perceived engaging in the middle press

as more difficult, i.e. was less willing to engage in it and spent more time triggering it, he also

had a lower probability to engage in the choice.

10 Effects of offer parameters, side and action-selection time remained unchanged: Chosen Reward: β=-0.23; p<10-120; Chosen Force: β=0.031; p=0.00062; Unchosen Reward: β=0.032; p=0.00099; Unchosen Force: β=0.049; p<10-7; Side: β=0.106; p<10-8; Action-selection time: β=-0.78; p<10-150.

Experimental results

191

Figure 52: The subjective difficulty to engage in the choice scaled with the subjective difficulty

to engage in the middle press.

(A,B) Logistic regressions for choice engagement. Over and above effects of the sum of Forces

and of Reward offered, (A) the current engagement rate in the middle press (engagement rate

in the current half of the session) had a positive influence on choice engagement, and (B)

middle press RT had a negative effect. This indicates that if the monkey was less willing or had

taken more time to trigger the middle press, he would be less willing to engage in the choice.

(C,D) GLMs for action-triggering time. Over and above effects of the sum of Forces and of

Rewards offered, (C) the current engagement rate in the middle press had a negative influence

on action-triggering time, and (B) middle press RT had a positive effect. This indicates that if

Experimental results

192

the monkey was less willing or had taken more time to trigger the middle press, he would take

more time to trigger the chosen action.

ctt: constant term ; F: Force; R: Reward; *** : p<0.001; * : <0.05 ; n.s.: non-significant, error

bars represent the SEM.

Next, I tested whether this relationship also applied to action-triggering time, i.e.

whether the monkey spent more time triggering the chosen action when he perceived

engagement in the middle press as more difficult. First, I expected that action-triggering time

would scale negatively with the current engagement rate in the middle press. I fit a GLM for

action-triggering time on all the non-repeated trials in which the monkey pressed on the

middle grip then on one side grip, and in which I could decompose RT into its subcomponents

(valid eye-tracking, monkey did look at both options and looked at chosen one during

execution). All the parameters found to have effects on action-triggering time (chosen and

unchosen Forces and Rewards, Side of press, action-selection time) as well as current

engagement rate for the middle press were included. Besides the same effects as what I

described before11, there was a negative effect of middle press engagement rate on action-

triggering time (β=-0.033; p=0.00012, Figure 52C), indicating that the monkey was faster at

triggering the chosen action if he had found engaging in the middle press less difficult and

would engage more often. Note that this effect remained true even if including middle press

RT and choice engagement rate to the model.

In order to confirm this result, I then looked for a relation between middle press RT

and action-triggering time, expecting a positive relationship. Again, I fit a GLM for action-

triggering time (same trials, sessions pooled) with chosen and unchosen Forces and Rewards,

Side of press, action-selection time, and middle press RT as parameters. Besides previously

described effects12, there was a positive effect of middle press RT on action-triggering time

(β=0.064; p<10-13, Figure 52D). This relationship remained true over and above engagement

rates in the middle press and in the choice.

11 Chosen Reward: β=-0.23; p<10-120; Chosen Force: β=0.031; p=0.00062; Unchosen Reward: β=0.049; p=0.00099; Unchosen Force: β=0.049; p<10-7; Side: β=0.095; p<10-6; Action-selection time: β=-0.78; p<10-150. 12 Chosen Reward: β=-0.24; p<10-128; Chosen Force: β=0.031; p=0.00040; Unchosen Reward: β=0.030; p=0.0015; Unchosen Force: β=0.048; p<10-7; Side: β=0.095; p<10-6; Action-selection time: β=-0.78; p<10-150.

Experimental results

193

Overall, these results indicate that the subjective difficulty to engage in the first step

of the task (middle press) was positively related to the subjective difficulty to engage in the

second step of the task (choice).

(2) Choosing between offers

In this task, in each trial, the monkey had to make a choice between two offers. I first

tested whether the monkey followed a speed-accuracy tradeoff when deciding and

questioned whether increased subjective difficulty to engage in the costly actions was

associated to changes in the speed-accuracy tradeoff, i.e. whether the current motivational

state of the monkey had effects on the resources he was willing to invest in the choices.

(a) Influence of Forces and Rewards on binary

choices between offers

First, I examined the effects of offer parameters (Forces and Rewards) on choices (left

or right option chosen), in order to be able to identify the value of offers based on choice

behavior. Overall, I expected the Forces to have a negative effect on choice probability and

the Rewards to have a positive effect on choice probability. Practically, I pooled the non-

repeated trials in which the cues appeared and the monkey made a choice (pressed on one

side grip) of all the sessions, and fit logistic regressions on choices (1: chose right side option,

0: chose left side)13. I compared three models: one with Rewards (left and right separately)

and Forces (left and right separately), one with the difference in Rewards (right–left) and the

difference in Forces (right-left), and one with the Reward minus the Force for both sides (left

and right, separately). I compared the fit of these regressions by comparing their BICs. The

model including all the offer parameters of both sides, separately, fit the data better

(BIC=3926.4) than the model using the differences between Forces (right-left) and Rewards

(right-left, BIC=3966.8), and than the model using the differences between Reward and Force

by side (BIC=4371.2).

In the best model, the right side Reward had a positive effect on right offer choice

probability (β=2.73; p<10-25), as well as the left side Force (β=1.15; p<10-12). Conversely, the

13 Identical results were obtained when including non-repeated trials too.

Experimental results

194

left side Reward (β=-3.05; p<10-29) and the right side Force (β=-1.04; p<10-10) had negative

effects on right offer choice probability (Figure 53A). Additionally, the constant, indicating a

potential side bias, was not significant (p=0.45). In other words, the probability to choose the

right side offer increased if it had a greater Reward and a smaller Force, i.e. a greater value,

and decreased if the left side option had a greater Reward and a smaller Force, i.e. a greater

value. Moreover, the monkey had no bias towards choosing one side above the other.

Figure 53: Binary choices between offers, and action-selection times.

(A) Coefficients for the logistic regression for choices (1 if the monkey chose the right side, 0

otherwise). The Reward offered on the right side had a positive influence on the probability to

choose the right side offer while the Force offered on the right side had a negative influence.

The opposite pattern was found for parameters of the left side offer, with a negative effect of

Experimental results

195

the Reward offered on the left side and a positive effect of the Force offered on the probability

to choose the right side option.

(B) Coefficients of the GLM for action-selection times. The chosen Reward and the unchosen

Force had negative effects, while the unchosen Reward and the chosen Force had positive

effects, indicating that a lower difference in value of the offers would lead to a longer action-

selection time.

(C) Coefficients of the GLM for action-selection times. There was a marginally significant

negative effect of the difference in the values of the chosen and the unchosen options (value

difference).

ctt: constant term ; F: Force; R: Reward; ***: p<0.001; **: p<0.01; *: <0.05 ; n.s.: non-

significant, error bars represent SEM.

These results would additionally indicate that, just like what I observed for choice

engagement, choices were more strongly influenced by Rewards than by Forces. In order to

properly assess that, I fit session-by-session logistic regressions for choice (1 if right side

chosen, 0 if left side chosen) on trials in which the monkey made a choice (pressed any side

grip). Using a two-way ANOVA, I compared the magnitude (absolute value) of the effects (β

values) depending on the side of the parameter (left or right) and the nature of the parameter

(Force or Reward). There was a significant effect of Force versus Reward, with stronger effects

associated to Rewards (F(1)=316.77; t-test on means of absolute values of β values, t(67)=-

2.3253; p<0.0001). There was no significant effect of side (F(1)=2.21; p=0.14). In other words,

Rewards had stronger effects on choices than Forces, and this pattern is reminiscent of what

I observed for choice engagement.

Overall, our results show that the monkey computed a tradeoff between Rewards and

Forces to choose between offers.

(b) Action-selection time had a tendency to increase

with choice difficulty

Second, I examined whether the pattern of action-selection times was compatible with

a speed-accuracy tradeoff. Humans and animals have been shown to increase the duration of

their decisions when difficulty (amount of information to process, conflict…) increases (Chittka

Experimental results

196

et al., 2009, 2003; Förster et al., 2003; Fudenberg et al., 2018). The notion of speed-accuracy

tradeoff stems from the postulate that accurate decisions require longer computations, and

are therefore more costly, at least in opportunities (Boureau et al., 2015). Practically, in this

task, during RTs, in most trials, gaze usually first alternated between the two cues, before

locking onto the chosen one. I posited that the period during which gaze was mobile between

cues corresponded to the action-selection period. Consequently, I looked at the relationship

of action selection duration with offer parameters and choice difficulty.

In order to investigate the relation between action-selection times and offered Forces

and Rewards, I pooled all the trials in which the monkey had looked at both options and had

looked at the chosen option during execution, from all the sessions in which I had valid eye-

tracking recordings (32/34 sessions). Additionally, I only selected trials in which the monkey

did not know about the offers displayed by the cues before they appeared, i.e. non-repeated

trials. Indeed, in repeated trials, the monkey probably did not need to select the chosen action,

as he may have done it before, and merely triggered it14. I fit a GLM on action-selection times

with chosen and unchosen Forces and Rewards as parameters, as well as the Side of press.

The effect of the chosen Reward (β=0.20; p<10-48) and of the unchosen Force (β=0.11; p<10-12)

were positive, and the effects of the unchosen Reward (β=-0.13; p<10-20) and of the chosen

Force (β=-0.09; p<10-9) were negative (Figure 53A). Additionally, action-selection time was

longer for right side choices (Side effect, β=-0.50; p<10-72). In other words, action-selection

was longer when the chosen option had a high value and when the unchosen option had a

low value. Thus, I compared the fit of the model presented above and that of a model including

only the difference in offer value (determined using choices, see methods) or the absolute

value of the difference between the values of the chosen and unchosen options and the side

of the chosen offer. I found that the model including all the parameters separately fit the data

better (Model with all the parameters: BIC=13 254; model with value difference: BIC=13 528,

model with absolute difference of values: BIC=13 529).

Additionally, to be able to compare the magnitude of the effects of Forces and Rewards

on action selection time, I fit a GLM per session for action selection time with chosen and

14 This hypothesis was confirmed by fitting a GLM for action selection times in repeated trials, with chosen and unchosen Rewards and Forces, and chosen side as parameters. No parameter had a significant effect (p>>0.05). This provides additional evidence that monkey selected his course of actions during the period of gaze position alternation between offers.

Experimental results

197

unchosen Forces and Rewards. I selected trials in which the monkey saw the cues, made a

choice, had looked at both options, and was looking at the chosen cue when triggering the

action. I then performed a two-way ANOVA (analysis of variance test) on the GLM coefficients

with chosen versus unchosen parameters and nature of parameters (Force or Reward) as

factors. The ANOVA failed to detect any effect of the “Force versus Reward” parameter

(F(1)=1.06; p=0.31) or of the “chosen versus unchosen” parameter (F(1)=2.77, p=0.09). In

other words, contrary to what was observed for choices between offers for which Rewards

had a stronger effect, action-selection duration was affected by Rewards and Forces with

virtually equivalent magnitudes.

Although the model explaining action-selection time with the value difference

between the chosen and unchosen option was not the model fitting the data best, there was

a marginally significant positive effect of the value difference on action-selection time

(β=-0.024; p=0.082, Figure 53B). This provides some indication that the monkey had a

tendency to be faster when the difference in value between the options was higher. These

results indicate that the monkey took more time to choose between options when the choice

was more difficult, with closer values of the two options.

(c) Action-selection time did not influence choice

accuracy

Next, I looked for evidence that increased time spent on the choice led to more

accurate decisions, which could have indicated that longer action-selection times were

associated with increased exertion of cognitive control for the decision. Using the subjective

values of the offers inferred from choices (see method in II.B.2.d), I determined whether the

monkey had chosen according to the average value that he had attributed to each offer, i.e.

whether he had made an accurate choice. First, I fit a logistic regression for choice accuracy (1

if option with highest subjective value was chosen, 0 otherwise) with only action-selection

time as parameter, on all the non-repeated trials in which the monkey made a choice and in

which action duration time could be determined. Action-selection time had a negative effect

on choice accuracy (β=-0.19; p<10-6), indicating that the longer the monkey took to reach a

choice, the less accurate his choice would be (Figure 54A).

Experimental results

198

In order to remove any difficulty-related effects, I then added the absolute value

difference (value of best option value minus value of the worst option, ΔVbest-worst) and offer

parameters (Forces and Rewards of best and worst options). The effect of action-selection

time was no longer significant (β=-0.07; p=0.13, Figure 54B). ΔVbest-worst had a positive effect

on choice accuracy (β=4.65; p<10-4). Regarding individual offer parameters, the Reward of the

best option (β=2.30; p<10-64) and the Force of the worst option (β=1.05; p<10-57) had positive

effects, while the Force of the best option (β=-0.97; p<10-55) and the Reward of the worst

option (β=-1.95; p<10-63) had negative effects. These results indicate that action selection time

had no effects on choice accuracy over and above effects of choice difficulty, i.e. the

relationship between action-selection time and accuracy was due to the fact that action-

selection time was longer in case of difficult choices (see Figure 53B&C), associated to lower

choice accuracy.

(d) Absence of relationship between engagement

difficulty and choice accuracy

Earlier, I established that difficulty, i.e. required effort, to engage, probably reflected a

global motivational state with effects on all engagement-related processes throughout the

trial. Next, I questioned whether this motivational state had effects on the cognitive effort

invested in choices between offers. Specifically, I would have expected that periods of time

during which the monkey found it more difficult to engage were associated to lesser choice

accuracy. Therefore, I tested whether choice accuracy varied with current middle press and

choice engagement rates, middle press RT, and action-triggering time (side press). I fit a

logistic regression for choice accuracy as described above, and added either one of these four

parameters to the regression. In none of the cases did I find a significant relationship between

choice accuracy and such proxies for the effort required to engage (p>0.55 in all cases). This

result indicates that the motivational state that influences the engagement of the monkey and

more specifically the subjective difficulty of engaging was probably not related to the

mobilization of resources for the choices.

Experimental results

199

(e) Increased action-selection time is associated to

decreased sensitivity of choices to Forces

One major assumption of the previous analyses was that choices were considered to

be accurate, and therefore possibly associated to more control exertion, if they were

consistent with the mean value of the offers. Cognitive control is required to inhibit decision

biases, and avoid default (automatic) courses of action, in order to promote strategies

compatible with the goals, usually reward rate maximization (part I.A.2.d)(1) for review). In

that task, the monkey perceived the high-Force conditions as aversive, thus making his choices

by avoiding high-Force options. Moreover, there was a tendency for combinations of offers

with a high sum of Forces to be refused more often. In that sense, the default course of action

would be choosing the low-force options. Thus, it is possible that increasing cognitive control

investment for decisions would have facilitated high-Force choices. If that were true,

considering the extensive literature on the relationship between the time spent on decisions

and the cognitive control allocated to the decision process (Hunt et al., 2012; Krajbich et al.,

2010; Krajbich and Rangel, 2011; Milosavljevic et al., 2010; Morris et al., 2019; Philiastides and

Ratcliff, 2013), I posited that increased action-selection time should be associated with

decreased effects of Forces on the probability to choose the high-Force option (Force

sensitivity), but should not affect Reward sensitivity.

To test this hypothesis, I categorized the choices made by the monkeys in two ways:

high- versus low-Force choices (if he chose the option associated to the highest Force or not)

and high- versus low-Reward choices (if he chose the option associated to the highest Reward

or not). I pooled all the non-repeated trials in which the monkey made a choice (pressed on

one side grip after cues onset). First, I split trials depending on whether the high-Force option

had been chosen or not (excluding trials in which the Forces offered on both sides were

identical). I then fit a logistic regression for high-Force choices (1 if the high Force option was

chosen, 0 otherwise) with the difference between the high and low Forces and difference of

Rewards of the high-Force and low-Force options. Additionally, based on our hypothesis, I put

the interactions between action-selection time and the differences in Forces and in Rewards

of the offers.

Experimental results

200

Figure 54: Relationship between action-triggering time and choice accuracy: tenuous evidence

for a speed-accuracy tradeoff, and evidence for a “speed-cognitive control production”

tradeoff.

(A,B) Logistic regressions for choice accuracy (1 if the monkey chose the option with the highest

value, 0 otherwise). (A) When action-selection time was alone in the model, there was a

negative relation between action-selection time and choice accuracy. (B) This relationship did

not stand over and above choice difficulty (Difference in value between best and worst option,

Delta V best-worst) and offer parameters (Forces and Rewards).

Experimental results

201

(C) Logistic regression for high-Force choices (1 if the monkey chose the option with the highest

Force offered, 0 otherwise). Over and above effects of the difference in Forces (F difference,

high – low force) and in Rewards (R difference, Reward of high force option – Reward of low-

Force option), there was negative effect of the interaction between action-selection time and

the difference in Forces.

(D) Interaction plot showing the effects of the interaction between action-selection time (AST)

and the Force of the high-Force option (y-axis) on the probability to choose the high-Force

option. Action-selection times were split between those above (dotted line) or below (full line)

the median (m). The effect of the difference of Force was smaller when action-selection time

was longer.

ctt : constant term ; F: Force; R: Reward; ***: p<0.001; error bars represent SEM.

Expectably, there was a positive effect of the difference in Rewards (β=-0.61; p<10-13)

and a negative effect of the difference in Forces (β=4.26; p<10-124, Figure 54C). This indicates

that the monkey was more willing to choose the high-Force option if it was associated to a

higher Reward, and was and less willing to choose it if the alternative was associated to a

higher Reward. Moreover, the monkey was more willing to choose the high-Force offer if the

highest Force proposed was close to the lowest force proposed. Additionally, there was a

negative effect of the interaction between the difference in Forces and action-selection time

(β=-0.16; p=0.041). Specifically, longer action selection-times were associated to decreased

effects of the difference in Forces (Figure 54D). There was no effect of the interaction between

action-selection times and Rewards (β=-0.085; p=0.56). Note also that there was no main

effect of action-selection time if it was added to the model, indicating that longer action-

selection times did not merely bias the choice process towards high-choice options, but

decreased the sensitivity to Forces, possibly because longer action-selection times were

associated to increased cognitive control production.

I repeated the procedure, this time splitting trials between ones in which the highest

offered Reward was chosen (high-Reward option chosen) and ones in which the lowest

offered Reward was chosen (low-Reward option). I put the difference in Rewards (biggest –

smallest Reward) and the difference in Forces (Force of the high-Reward option – Force of the

low-Reward option) as well as the interactions between action-selection time and Force and

Experimental results

202

Reward difference as parameters. There was a negative effect of the difference in Forces (β=-

1.88; p<10-38) and a positive effect of the difference in Rewards (β=1.52; p<10-24) but no effects

of their interactions with action-selection time (respectively: β=0.13; p=0.24 and β=-0.002;

p=0.99).

These results indicate that increased time spent on the decisions was specifically

associated with decreased sensitivity of the choices to the Force parameter of the offers,

making the monkey more likely to engage in courses of action requiring high physical effort

when he had taken longer to selection a course of action. Considering that higher cognitive

control is associated weaker biases towards responding according to the default (reviewed in

part I.A.2.d)(1)), and since the default response would be avoiding high-Force offers, this result

would indicate that longer decision times were associated to increased control exertion and

lesser biases to avoid high-Force options.

(3) Execution of costly actions

In this task, two costly actions had to be completed: a press on the middle grip, with a

minimal amount of force, and a press on one of the side grips, with a force above the threshold

requested by the cue for at least 80ms. I tested whether I could find evidence of a relationship

between the subjective difficulty of engaging in actions and the physical effort produced for

the execution of the actions. Specifically, I expected that when the monkey perceived

engagement and action-triggering as more difficult, he would also perceive action execution

as more difficult and would exert less force and have lower performance.

(a) Middle press exerted force was negatively

related to difficulty to engage in middle press

First of all, I examined the relationship between the force exerted on the middle grip

and the subjective difficulty of engaging in the middle press, approximated by engagement

rates in middle presses and middle press RTs. I fit GLMs for the maximum force exerted on the

middle grip with either only the block-wise engagement rate in the middle press or only the

middle press RT. There was a positive effect of middle press current engagement rate (β=0.08;

p<10-12), or a negative effect of middle press RT (β=-0.12; p<10-25) on the maximum force

exerted on the middle grip (Figure 55A&B respectively). This indicates that faster RTs, i.e.

Experimental results

203

lower subjective difficulty to engage, was also associated to higher force exertion. In other

words, the monkey exerted less force on the middle grip if he perceived engaging in the middle

grip press as more difficult.

Figure 55: Negative relationship between maximum exerted force and engagement difficulty.

(A,B) GLMs for maximum exerted force on middle grip. The monkey exerted more force on the

middle grip if (A) current engagement rate was higher and (B) RT to the middle press had been

shorter, suggesting that increased difficulty to trigger the middle press led to decreased

exerted force.

(C) GLM for maximum exerted force on side grip. Action-triggering time had a negative effect

on the force exerted on the side grip, suggesting that increased difficulty to trigger the side

press led to decreased force exertion.

Experimental results

204

(D) Logistic for performance of side press. Action-triggering time had a negative effect on

performance over and above effects of offer parameters and side.

(b) Side press exerted force was negatively related

to difficulty to engage in side press

Second, I examined whether increased subjective difficulty to engage in the chosen

action was also associated to decreased exerted force for the chosen action (side grip). I fit a

GLM for the maximum exerted force on the side grip at the moment of the execution of the

chosen action, with offer parameters (chosen and unchosen Rewards and Forces), side of

press, and action-triggering time. I used all the trials15 in which I could identify action-

triggering time, and in which the monkey had pressed on the side grip.

There was a negative relation between action-triggering time and the maximum force

exerted on the side grip (β=-0.040; p<10-5, Figure 55C). Additionally, the chosen Reward had

a positive effect (β=0.042; p<10-4), as well as the chosen Force (β=0.70; p<10-150), the

unchosen Force (β=0.049; p<10-7) and side of press (β=0.44; p<10-123). The unchosen Reward

had a negative effect (β=-0.057; p<10-8).

Note that middle press RTs, middle press engagement rate and choice engagement

rate were not correlated to side press exerted force over and above the positive relationship

between middle press exerted force and side press exerted force.

(c) Performance was negatively related to action-

triggering time

Over all sessions, the monkey’s success rate was 99.81% chosen Force 1, 94.8% for

chosen Force 2 and 71.7% for chosen Force 3. Based on the effect of action-triggering time on

exerted force, I could expect either one of two different effects on performance: no effect if

the exerted force decreased but was maintained above threshold, i.e. if force was optimized

to be minimized without changing performance, or a decrease in performance if the exerted

force was decreased regardless of the instruction. To examine this, I pooled all the trials in

15 Identical results were obtained using only non-repeated trials.

Experimental results

205

which the monkey made a side press, with a valid eye-tracking signal and in which decision

time was determinable, and fit a logistic regression for performance, with chosen Forces and

Rewards, side of press, and action-triggering time as parameters. There was a negative effect

of action-triggering time on performance (β=-0.23; p<10-5, Figure 55D), as well positive effects

of the chosen Reward and of the unchosen Force (β=0.13; p=0.035 and β=0.22; p<10-4,

respectively), and negative effects of the chosen Force and the unchosen Reward

(respectively: β=-1.80; p<10-92; and β=-0.14; p=0.026). The Side of the press had no significant

effect (β=0.15; p=0.16). This indicates that increased time taken to trigger the action was

associated to decreased performance, meaning that the observed decrease in exerted force

did not correspond to an optimization of resource mobilization for the action, but rather an

overall decrease in effort production.

Overall, the results described above indicate that increased subjective difficulty of

engaging in actions and triggering them led to decreased force exertion during actions, and

decreased performance, when the force was instrumental for the action. This suggests a

decrease in physical effort production for action execution when engaging in the action was

more costly. This could suggest either that engaging when subjective difficulty was high was

overall associated to a low motivational state, and that this state also affected physical effort

production. Alternatively, it could indicate that making the effort to engage when subjective

difficulty was high depleted a certain resource critical for physical effort, resulting in a lower

capacity to exert force.

(4) Summary of behavioral results

To summarize, in this task, three categories of processes potentially required effort

exertion. First, engaging in costly actions probably required cognitive control exertion, as the

default (automatic) response would have been avoiding the high-physical effort options (see

part I.A.3 for relationship between engaging in actions requiring physical effort and cognitive

control). I showed that the subjective difficulty of engaging could be approximated on a trial-

by-trial basis by the time taken to trigger each action (middle press RT and action-triggering

time for side press). Indeed, the monkey took longer to trigger both actions in periods of

overall low willingness to engage, i.e. high subjective difficulty of engaging. The subjective

Experimental results

206

difficulty to engage in both steps of the task was strongly correlated, suggesting that the

subjective difficulty of engaging in each step depended on a global effort variable.

Second, making choices between offers probably required cognitive control (see part

I.A.2.d)(1) for the role of cognitive control on optimizing decisions). I showed that the monkey

maybe followed a speed-accuracy tradeoff, as the time spent deciding increased for difficult

choices but choice accuracy still decreased, suggesting that the time spent deciding was not

increased enough to cope with the increase in difficulty. Moreover, I could not find any

consistent relation between the subjective difficulty of engaging in actions and the accuracy

of the choices of the monkey. Conversely, increased time spent on choices led to decreased

Force sensitivity, but did not alter Reward sensitivity. In that sense, action-selection time may

reflect the cognitive control invested in the decision to avoid choosing default courses of

actions associated to low physical effort.

Third, executing the forces probably represented a physical effort, at least for the side

press, since monkeys discounted offered Rewards with required Forces in their choices. I

showed that increased subjective difficulty of engaging in actions led to decreased force

exerted on the grips. This indicates either that states of lower motivation were associated with

lower physical effort exertion or that exerting cognitive control to trigger the chosen action

led to decreased capacity for physical effort (see part I.A.3 for negative effects of cognitive

control exertion on physical effort).

b) Neurophysiology of the LC

54 LC single-units were recorded in the two-alternative choice task. LC neurons were

identified prior to task onset based on anatomical criteria (located just ventrally to the motor

part of the mesencephalic trigeminal nucleus which is sensitive to face and mouth

movements) and well-described electrophysiological criteria (low baseline rate <5Hz, large

waveforms and activation-inhibition responses to sudden sounds or other stimuli, Bouret and

Richmond, 2009; Grant et al., 1988). The position of the electrode in the LC was confirmed in

several sessions using clonidine injections (Bouret and Richmond, 2009; De Sarro et al., 1987;

Gatti et al., 1988; Varazzani et al., 2015), which caused a strong decrease of LC neurons’ firing

rate (Figure 47B). Moreover, if the monkey became lethargic or fell asleep, the firing of the

recorded LC neurons strongly decreased. The average firing rate of all 75 neurons was

Experimental results

207

4.78±0.73 spikes/s over the course of recording sessions, and the baseline firing rate of the

neurons remained quite constant when the monkey was not drowsy. I never observed any

change in firing mode (e.g. tonic vs phasic), but LC neurons tended to exhibit a decrease in

rate at the beginning of the task, compared to pre-task rate.

Throughout analyses of LC activity in that task, I first looked for evidence of an

encoding of the effort produced to engage in actions and trigger them, choose between offers,

and execute them, or at least of the subjective difficulty of these processes. I expected LC

neurons to display a domain-general encoding of effort production throughout the task.

Regarding cognitive control exertion to engage in actions and trigger them, prior to middle

press onset and prior to the chosen action, I expected LC neurons to encode the instantaneous

subjective difficulty of triggering the middle press and the chosen action, respectively.

Specifically, I expected LC neurons to encode middle press RT before the middle press, and

action-triggering time before the chosen action. Importantly, I did not expect LC neurons to

encode the current engagement rates in the middle press and the choice, respectively. Indeed,

despite their correlations with middle press RT and action-triggering time, these variables

represented long-term motivational states rather than instantaneous subjective difficulty. In

other words, I expected LC neurons to encode the cost of ongoing processes with a fine time

scale. Regarding physical effort production to execute the presses, I expected LC neurons to

encode the exerted force on the grip during force exertion.

Second, I expected LC activity to have delayed effects on effort production. In other

words, I expected increased LC activity early in the trials (e.g. around the middle press) to have

a facilitative effect on effort production later in the trial (e.g. engaging in the choice and

choosing between options), regardless of the challenge at hand at that moment. I

systematically examined the relation between LC activity at each step of the task (middle press

onset and execution, cues onset, chosen action onset) and effort-related behavior later on

(engaging, choosing between offers, triggering and executing the chosen action).

Third, I hypothesized that the signal regarding effort production constituted by LC

activity around chosen action execution could serve as a feedback to the effort allocating

system. Practically, this means that I expected the strength of the activation of LC neurons

around chosen action execution (over and above task parameters and the exerted force) to

predict the effort sensitivity of the binary choice made in the next trial. Additionally, in order

Experimental results

208

to correctly adjust effort allocation, the system would need to receive feedback on errors.

Since commitment error signaling by LC neurons had already been reported in the past (Jahn

et al., 2020), in order to confirm and extend this finding I looked for commitment and

execution error signaling by LC neurons in my paradigm.

(1) Middle press-related activity: encoding of middle press

RT and enhancement of engagement-related processes and of

Force exertion

In the previous part, I showed that the subjective difficulty of triggering the middle

press varied across sessions and could be approximated on a trial-by-trial basis using middle

press RT. Given the results of the previous study (Bornert and Bouret, in prep) regarding the

positive relationship between LC activity before the onset of a costly action and RT, I expected

LC neurons to encode middle press RT before middle press onset. Additionally, I looked for

short-term (force exerted on the middle grip, immediately after triggering the press) and

longer-term (choice engagement, choice accuracy, chosen force exertion) effects of such LC

activity on behavior, given our hypothesis that LC activity energizes behavior, and specifically

facilitates effort production after the activation.

(a) LC neurons increased their rate before and

during middle press

First of all, I assessed rate changes from baseline around the middle press. To quantify

the proportion of neurons that displayed evoked responses around the press, and test for a

population-wise modulation of rate, I computed the neuron-by-neuron spike count per trial

in a pre-press ([-300;0ms] from press onset) and a post-press ([0;300ms] from press onset)

epoch, and in reference windows (baseline, [-300;0ms] from middle red dot onset). I then

compared the spike count in the baseline window and the spike count in the two windows of

interest with a paired t-test. At the level of the population, coherence of evoked responses

was tested by performing a t-test on the neuron-by-neuron t values previously obtained, and

comparing their mean against zero.

Before press onset, rate changed from baseline for 27/54 neurons (20 increased and 7

decreased), and there was a coherent increase in rate at the level of the population (second

order: t(53)=2.56; p=0.0132, Figure 56A). After press onset, rate changed from baseline for

Experimental results

209

36/54 neurons (30 increased and 6 decreased), and there was a coherent increase in rate at

the level of the population (second order: t(53)=7.71; p<10-9, Figure 56B).

During the press, I questioned whether LC activity was better aligned onto the onset

of the press or the attainment of the maximum exerted force on the grip. In that aim, in all

the non-repeated trials in which the monkey made a side press, for all 54 neurons, I identified

the time of the attainment of the maximum in exerted force on the middle grip, and for each

spike in the press-onset to maximum of force exerted epoch, I computed the absolute value

of the time elapsed from the spike to the onset of the press and to the peak of the press.

Neuron-by-neuron, I then calculated the density of the distribution of these times to either

event, and identified the maximum of each density function (method adapted from Bouret et

al., 2012). Across neurons, I then performed a t-test on the maxima of the density functions.

These density functions included the same number of data points, since I used the same spikes

only aligned on different events. Therefore, a higher maximum of the density function would

signal a narrower distribution, and therefore a better alignment of the timing of spikes. Across

the population, spikes were better aligned onto the peak of the press than its onset (t(53)=-

2.64; p=0.011).

(b) LC neurons encoded middle press RT before

middle press onset

LC neurons activated just prior to middle press onset, which, in non-repeated trials,

signaled engagement in trials independently of value evaluation. I reasoned that the

frequency of engagement in middle presses across trials as well as the middle press RT (time

elapsed between Go signal - central green dot - and initiation of middle press) should

represent the cognitive control invested in triggering the middle press. Under the hypothesis

that LC neurons signal resource mobilization for costly actions, or at least the difficulty of

triggering costly actions, I questioned whether LC neurons encoded engagement rate in

middle presses, and middle press RT, just before middle press onset. Although I showed earlier

that these two measures were correlated negatively, their encoding by LC neurons would

reflect slightly different intuitions: while current engagement rate (computed over several

trials), represents a motivational state impacting the subjective difficulty of triggering actions,

trial-by-trial RT may represent instantaneous difficulty or resource mobilization for triggering

action initiating the trial.

Experimental results

210

Figure 56: Activity of LC neurons around middle press onset: LC neurons activated around the

middle press and encoded middle press RT before the middle press.

(A) Sliding window representation of the mean rate across time of all the LC units recorded (y-

axis, 1 neuron per line), around middle press onset. Rate is z-scored by neuron. The green dot

indicates the mean timing of the onset of the green dot (based on the mean middle press RT

of the session).

Pre action onset

Post action onset

Experimental results

211

(B) Density plots (bandwidth=5) of t values of the comparison of rate in the pre (yellow) and

post (dark red) middle action onset epochs, with baseline (pre red dot onset) rate. Second order

t-test on rate indicated that, at the level of the population, around middle press onset, rate

significantly increased.

(C) GLMs for pre-middle action spike count: mean of individual beta values and SEM across

neurons. There was no significant effect of block-wise engagement frequency at the level of

the population.

(D) Raster plot of example LC unit displaying a positive encoding of RT before middle press

onset. More spikes (black dots) were present before middle press onset (pink line) after long

RTs (longer time during green dots representing go-signal for middle press and pink line

representing middle press onset).

(E) GLMs for spike count: mean of individual beta values and SEM across neurons. LC neurons

encoded RT positively (significant effect across the population) before middle action onset.

*: p<0.05; **: p<0.01; n.s.: non significant, error bars represent SEM.

First, I cut testing sessions in halves and computed the frequency of engagement in the

middle press in non-repeated trials in each block i.e. the proportion of non-repeated trials in

which the monkey made a middle press. I then fit, for each neuron, a GLM for spike count in

a pre-middle action epoch ([-200;0ms] epoch before middle action onset, based on raster

examination) with block-wise frequency of engagement in middle presses as parameter. I had

to exclude 3 neurons recorded in sessions with insufficient differences in engagement rate

between blocks. For 13/51 neurons, block-wise engagement frequency had a significant effect

(7 positive, 6 negative). At the level of the population, there was a no significant effect (second

order t-test on T values: p=0.30, Figure 56C).

Second, I fit a neuron-by-neuron GLM for spike count in the same pre-middle action

epoch, with only RT for the middle press as parameter. I only included trials for which the

monkey did not know about the future offers (cues) i.e. that had not been repeated, and in

which he pressed on the middle grip during the correct period (during go-signal presentation).

9/54 LC neurons encoded RT positively in that epoch (8 positively, 1 negatively). The activity

of an example unit is showed in Figure 56D. The positive encoding of RT was consistent at the

level of the population (second order t-test on T values: t(53)=2.75; p=0.0081, Figure 56E).

Experimental results

212

Note that this encoding was not present earlier before press onset or before red dot onset,

nor was an encoding of current willingness to engage in the middle press.

Overall, these results reproduce the result obtained in the first study (Bornert and

Bouret, in prep). They point towards an encoding of instantaneous subjective difficulty rather

than current (longer-term) motivational state, before the monkey engages in the middle

press.

One of the hypotheses of this work concerns the fact that LC neurons encode

instantaneous effort production. If the monkey produced an effort to execute the middle

press, the force exerted at that time should represent the physical effort associated to

pressing the middle grip. I therefore examined whether LC neurons encoded the maximum

force exerted on the middle grip, during force exertion. I fit a neuron-by-neuron GLM for spike

count in a [0;300ms]16 epoch after press onset (during force exertion) with only the maximum

exerted force on the middle grip as parameter, on non-repeated trials in which the monkey

engaged in the middle press during the dedicated time interval. The effect of the maximum

exerted force on the middle grip was significant for 8 neurons (2 negative, 6 positive) and was

not consistent at the population level (second order: p=0.32, Figure 56E). Note that, during

this epoch, RT was no longer encoded, and the exerted force on the middle grip was not

consistently encoded prior to action onset either (p>>0.05 at the population level). Similarly

to what we discussed in Study 1 (Bornert and Bouret, in prep), the minimal force that had to

be applied on the middle grip may not have required effort production.

(c) Middle press activity and engagement throughout

the trial: higher pre-middle press activity was associated

to higher probability to engagement in the choice

Prior to middle press onset, I showed that LC neurons encoded RT, which I hypothesize

to represent the effort made, or at least required, to trigger the action, and therefore engage

in a trial. Next, I questioned if this activity around the beginning of trials could influence

behavior later on during trials. Specifically, I hypothesized that increased LC activity would

16 Identical results were obtained when using a time window selectively including the period before the peak in force (see Figure 48A for force signals across time), around 200ms from action onset (spike count in a [0;200ms] epoch) or selectively the period after the peak in force ([200;400ms] after action onset).

Experimental results

213

lead to increased effort production later on, possibly a few seconds later. In the particular case

of our task, I hypothesized that pre-middle press activity, putatively reflecting the effort

invested in starting the trial, would have beneficial effects on the next processes requiring

effort production, regardless of the nature of the challenge (mental or physical): force exertion

on the middle grip and choice engagement. Since I questioned whether LC neurons as a

population could have effects on behavior, the general method I used for testing such effects

was pooling the relevant trials from all the sessions, i.e. from all the neurons, before fitting

regressions on the behavioral variable of interest, with spike count in windows of interest as

one of the parameters. Based on results from the previous study regarding the positive

correlations between the strength of encoding of task parameters and of behavioral variables

and baseline as well as evoked firing rate (see part II.A.2.c)(3)), I expected more active neurons

to have stronger effects on behavior, and therefore did not normalize spike count per neuron

before pooling the trials.

I looked for a link between pre-middle press onset activity and choice engagement,

over and above effects of the overall difficulty to engage and of offer parameters. I pooled

trials from all the sessions and fit a logistic regression for choice engagement with the sum of

Forces and the sum of Rewards, current engagement rate in middle press or middle press RT,

and pre-middle press onset spike count ([-200;0ms] epoch, in which RT is encoded by LC

neurons) as parameters. Only non-repeated trials in which the monkey pressed on the middle

grip were included.

Over and above the effects of the sum of Rewards (β=0.71; p<10-13), of Forces (β=-0.09;

p=0.25), and of the current rate of engagement in the middle press (β=0.13; p=0.07), there

was a positive effect of spike count on choice engagement (β=0.20; p=0.04, Figure 57A). The

same was true when including middle press RT in the model as a proxy for the difficulty to

engage (sum of Rewards: β=0.71; p<10-13, sum of Forces: β=-0.08; p=0.26, middle press RT:

β=-0.24; p=0.00014, spike count: β=0.20; p=0.04, Figure 57B). This indicates that increased

spiking before middle press onset was associated to a higher probability of engagement in the

choice. Importantly, these effects were not present in baseline activity (pre-middle red dot

activity) or in post-middle press onset activity (during middle press).

Experimental results

214

Figure 57: Increased LC activity before middle press onset was associated to decreased

difficulty to engage in the choice, as choice engagement increased and action-triggering time

decreased.

(A,B) Logistic regressions for choice engagement, on all non-repeated trials in which the

monkey pressed on the middle grip. Spike (spk) count in the [-200;0ms] epoch before middle

press onset had a positive effect on choice engagement, over and above the positive effect of

the sum of Rewards (Sum R) and the negative effect of middle press RT, and (A) the current

engagement rate in the middle press or (B) middle press RT.

(C) GLM for action-triggering time. Spike count in the [-200; 0ms] epoch had a negative effect

on action-triggering times, over and above effects of task parameters, chosen side and over

Experimental results

215

and above the strong negative correlation between action-selection and action-triggering

times. This indicates that increased LC activity led to faster responses.

*: p<0.05; **: p<0.01; ***: p<0.001, n.s.: non significant, Spk: spike, F: Force, R: Reward

Such effects of LC activity around the middle press could indicate either that engaging

in the choice and therefore triggering the chosen action was perceived as less difficult, or that

the monkey engaged more effort despite equal perceived difficulty when the LC neurons had

spiked more. We therefore fit a GLM for action-triggering time with pre-middle press activity

as parameter, as well as chosen and unchosen Forces and Rewards, action-selection time, and

side of press as parameters. Besides identical effects of offer parameters, action-selection

time and side as described earlier17, spike count before middle press onset had a negative

effect on action-triggering time (β=-0.022; p=0.011, Figure 57C), indicating that increased

spike count before middle press onset led to decreased time taken to trigger the chosen

action. In other words, after increased spiking of LC neurons around the middle press, the

monkey was both more willing to engage in the choice and faster to trigger the chosen action,

confirming the relation between middle press LC activity and subjective difficulty to engage in

the choice.

This effect was also present in the post-middle press onset epoch ([0;200ms] from

middle press onset, β=-0.041; p<10-5), indicating that spike count in an epoch around middle

press onset ([-200;200ms] from middle press onset) had a negative influence on action-

triggering time. I confirmed that by fitting a GLM for action-triggering time with spike count in

the [-200;200ms] epoch around middle press onset, chosen and unchosen Forces and

Rewards, action-selection time and side of press as parameters. The effect of spike count was

significantly negative (β=-0.036; p=0.0004).

Note that there was also a negative effect of baseline spike count on action-triggering

time, but that effect did not remain significant if middle press spike count ([-200;200ms]

epoch) was included in the model. Additionally, the fit of the GLM with baseline spike count

was not as good as the fit of the model with peri-middle press spike count (BICbaseline=8981.8;

BICmiddle press=8974.1).

17 Action-selection time: β=-0.78; p<10-150; Chosen Reward: β=-0.23; p<10-120; Chosen Force: β=0.030; p=0.00089; Unchosen Reward: β=-0.032; p=0.0010; Unchosen Force: β=0.049; p<10-7; Side: β=0.094; p<10-6.

Experimental results

216

(d) Middle press activity and force exertion: higher

pre-middle press activity was associated to increased

middle press exerted force

Next, I examined the relationship between pre-middle press LC activity and exerted

force throughout the trial. First, I tested the relationship between activity before middle press

onset and middle press exerted force, over and above the negative correlation of middle press

exerted force with middle press RT. I pooled all the non-repeated trials in which the monkey

pressed on the middle grip and fit a GLM for the maximum exerted force on the middle grip

with middle press RT and pre-middle press spike count ([-200;0ms] from middle press onset)

as parameters. Over and above the previously described negative effect of middle press RT

(β=-0.12; p<10-17), there was a positive effect of spike count on middle press exerted force

(β=0.084; p<10-8, Figure 58A).

Figure 58: Pre-middle press spike count had a positive relation with middle press exerted force.

(A,B) GLMs for middle press exerted force. There was a positive effect of pre-middle press spike

count ([-200;0ms] from middle press onset) on middle press exerted force, (B) as well as a

positive effect of the spike count during the press. Baseline rate had no effects.

*: p<0.05; ***: p<0.001, n.s.: non significant, Spk: spike, F: Force, R: Reward

This positive relationship between the middle press exerted force and spike count was

still present during the press, and specifically during the period of the press that preceded the

Experimental results

217

attainment of the peak of force ([0;200ms] after press onset (see Figure 48A for time course

of force exertion), over and above pre-middle press spike count (spike count before action:

β=0.039; p=0.029; spike count during action: β=-0.072; p<10-4; middle press RT: β=-0.12;

p<10-16). Additionally, adding baseline spike count (pre-middle red dot spike, [-300;0ms]

epoch) did not change the effect of pre- and post-middle action onset spike count on exerted

force (spike count before action: β=0.040; p=0.040; spike count during action: β=-0.073;

p<10-4; middle press RT: β=-0.12; p<10-16, Figure 58B). Moreover, the effect of baseline rate

on middle press exerted force was not significant (β=-0.003; p=0.87). These results tend to

indicate that peri-middle press activity was associated to higher middle press exerted force,

when regressing out effects of middle press RT, and considering the neurons as a population.

Overall, across the population, the spike count of LC neurons before middle press onset

scaled positively with middle press RT, a proxy for the instantaneous difficulty to trigger the

middle press. Increased activity in that epoch was associated to increased probability to

engage in the choice regardless of offer parameters. Moreover, increased spike count was also

associated with increased exerted force for the middle press.

(2) Activity around cues onset

Given the indirect (pharmacological or pupilometry studies, see part I.B.5.a)) evidence

for a role of LC activity in action-selection, and cognitive control exertion for choices against

biases, I looked for direct evidence for such roles in the activity of LC neurons and its relation

to behavior. The monkey only knew about the parameters (Forces and Rewards) of the offers

after cues had appeared (in non-repeated trials), and had putatively reached a decision (made

a choice) by the time he locked his gaze onto the chosen option (choice onset). Therefore, I

focused on the activity of LC neurons between cues onset and choice onset, in non-repeated

trials. I first examined the pattern of evoked responses during that period, then analyzed the

modulation of activity by offer parameters, and finally tested for effects of this activity on

choices, and especially on choice accuracy and Force sensitivity.

(a) Responses to cues onset and choice onset

Examination of LC activity during the second step of the task revealed that a certain

proportion of LC neurons displayed bursts of activity (a few spikes) between cue onset and

choice onset (last saccade onto chosen option, i.e. end of action-selection time).

Experimental results

218

I examined whether I could identify if the activity of LC units between cues onset and

choice onset was better aligned onto cues onset or onto choice onset. In that aim, using the

same method as the one used for middle press related activity, I compared the distribution of

the timing of spikes when aligned on cues onset or on choice (last saccade onto chosen option)

onset. I could not find any difference between the distribution of the event-to-spike times for

the activity comprised between those two events, i.e. I could not conclude on a better

alignment of spikes onto cue onset or choice onset (t(49)=1.23; p=0.22).

I then quantified the proportion of neurons that changed their activity after cues onset.

I used the same procedure as the one described for middle press onset responses. In brief,

neuron-by-neuron, I compared rate in windows of interest to rate in a baseline window, using

paired t-tests. Population rate changes were assessed by performing a second-order t-test on

the T values previously obtained. Note that the windows were assigned based on raster

examination. Around cue onset, I quantified activity in a [0;500ms] post-cues onset epoch,

and compared it to a [-500;0ms] epoch preceding cues onset18 (Figure 59A). 29/54 neurons

exhibited a change in rate (rate decreased for 9 neurons and increased for 20 neurons) and

this change was marginally significantly positive at the level of the population (second-order

t-test: t(53)=1.96; p=0.055, Figure 59B).

(A) (B)

Figure 59: Rate of LC neurons around cues onset, and results of first and second order t-tests

assessing rate changes: after cues onset, LC activity increased.

18 This baseline epoch was entirely comprised during the presentation of the middle blue dot, and started at least 400m after the middle press.

Experimental results

219

(A) Activity of the 54 recorded LC units around cues onset. Rate is z-scored by neuron. The

magenta dot indicates the mean RT of the session, i.e. the mean timing of action onset.

(B) Density plots (bandwidth=5) of the T values from the first-order t-tests comparing rate in

the [0;500ms] post-cues onset epoch to baseline rate ([-500;0ms] epoch before cues onset).

*: p<0.05; ***: p<0.001

(b) LC neurons positively encoded immediate

subjective difficulty to engage in choice around cues

onset

Raster examination indicated that some LC neurons displayed higher activity around

cues onset when RT (time from cues onset to action onset) would be longer (Figure 60A). For

a large proportion of the neurons with such a pattern of activity, the effect appeared to start

before cues onset (about 100ms before). I first quantified the effect of total RT on LC activity

in a [-100;200ms] epoch around cues onset. I fit a GLM for spike count in that epoch with only

RT as parameter. This analysis was restricted to non-repeated trials in which the monkey later

engaged in the choice. Additionally, in order to be able to decompose RT into its

subcomponents, I focused on the 50 neurons for which eye-tracking was available. The effect

of RT on peri-cues onset spike count was significant for 9/50 neurons (1 negative, 8 positive),

and was significantly positive at the level of the whole population (t(49)=2.07; p=0.044, Figure

60B).

In order to specify the relationship between LC activity and the subcomponents of RT,

i.e. action-selection and action-triggering time, I then fit a GLM for spike count (same window,

same neurons, same trials) with action-selection and action-triggering times as parameters.

The encoding of action-selection time was significant for 10/50 neurons (9 positive, 1

negative), and was significantly positive at the level of the population (t(49)=2.02; p=0.049,

Figure 60C). The encoding of action-triggering time was significant for 7/50 neurons (6

positive, 1 negative), and was significantly positive at the level of the population (t(49)=2.09;

p=0.042).

To verify whether the effect of the two parts of RT were encoded over and above the

effects of offer parameters (Forces and Rewards), I then added chosen and unchosen Forces

and Rewards to the previous GLM. In that model, the effect of action-selection time was still

significant for 10/50 neurons (1 negative, 9 positive) but there was only a marginally significant

Experimental results

220

tendency for a positive encoding at the level of the whole population (t(49)=1.70, p=0.095,

Figure 60C). The encoding of action-triggering time was still significant for 7/50 neurons (6

positive, 1 negative) and was consistently positive across the population (t(49)=2.74,

p=0.0085). This pattern indicates that neurons encoded the subjective difficulty to engage in

the choice and trigger the chosen action.

Figure 60: LC neurons encoded RT positively, and specifically its subcomponent action-

triggering time, over and above offer parameters, around cues onset.

(A) Raster plot of the activity of an example LC unit around cues onset (green line). Magenta

dots represent the activity of the neuron when the monkey will choose the left side option, and

purple dots represent the activity of the neuron when the monkey will choose the right side

option. Diamond-shaped purple dots represent the timing of the onset of the side press. Around

cues onset, this neuron exhibited increased activity when the monkey would take longer to

respond (longer time between green line and purple diamond).

Experimental results

221

(B,C,D) GLMs for spike count in the [-100;200ms] peri-cues onset epoch, with (B) only RT, (C)

action-selection and action-triggering time, or (D) action-selection, action-triggering time and

chosen and unchosen offer parameters (Forces and Rewards).

(B) LC neurons encoded RT positively around cues onset.

(C) LC neurons encoded action-selection and action-triggering times around cues onset, but (D)

only action-triggering time was significantly encoded over and above effects of Forces and

Rewards.

*: p<0.05, **: p<0.01; n.s.: non significant, F: Force, R: Reward, ctt: constant

In order to confirm that, I tested whether LC neurons encoded the current choice

engagement rate around cues onset. First, I split sessions in halves, determining the choice

engagement rate in each half, and associating each trial with the current choice engagement

rate. Then, selecting only non-repeated trials in which the monkey had engaged in the choice,

I fit neuron-by neuron GLMs for spike count with only the current choice engagement rate as

a parameter. The effect of the current engagement rate was significant for 14/50neurons (6

negative, 8 positive) and the effect was not consistent across the population (t(49)=0.079;

p=0.94)19. Neurons did not encode middle press RT or current engagement rate in the middle

press around that time either20. Importantly, neither side press RT, nor action-triggering time

were encoded in the baseline activity (activity prior to middle red-dot onset or earlier before

cues onset, i.e. before 100ms before cues, during middle blue dot presentation).

Similarly to what I observed around middle press onset, LC neurons therefore encoded

trial-by-trial difficulty to engage in choices (action-triggering time for side press, middle press

RT for middle press). In other words, LC neurons encoded the subjective difficulty of ongoing

action-triggering processes. Conversely, they did not encode the current engagement rates in

middle presses and choices. Considering that engagement rates provide long-term

measurement of the difficulty to engage, i.e. assess motivational states, the fact that they

were not encoded by LC neurons would be in favor of an encoding of the subjective difficulty

of ongoing processes with a fine time scale.

19 Adding action-selection and action-triggering time and chosen Forces and Rewards: current choice engagement significant for 12/50 neurons (6 positive, 6 negative), population: t(49)-0.10; p=0.92. 20 Middle press engagement rate: significant for 14/50 neurons (5 negative, 9 positive), population: t(49)=1.36; p=0.17. Middle press RT: significant for 5/50 neurons (1 negative, 4 positive), population: t(49)=0.69; p=0.49.

Experimental results

222

Around cues onset, contrary to what I found around middle press onset when LC

neurons encoded RT just prior to action onset, and to what we observed in the first study

(Bornert and Bouret, in prep) LC neurons encoded action-triggering time early before action

onset. Indeed, most RTs were longer than 700ms, meaning that the window in which action-

triggering time was encoded anticipated the action by at least 500ms21.

(c) LC neurons encoded Rewards but not Forces

after cues onset

Next, I questioned whether LC neurons encoded any of the parameters (Forces and

Rewards) of the offers displayed on screen, from which the monkey could choose. For activity

to participate in offer value evaluation, LC neurons would have needed to encode both

dimensions of both offers, i.e. the Forces requested and the offered Rewards. I restricted our

analysis to trials in which the monkey later made a choice i.e. pressed on one of the two side

grips. In the previous part, I already showed that LC neurons encoded only the chosen Reward

early after cues onset (Figure 60D).

Based on raster examination (rasters for one example unit displayed in Figure 61A&B),

I assessed effects of chosen and unchosen Rewards and Forces on LC activity in a [200;500ms]

epoch after cues onset. Although this window was late after cues onset, I ensured that I did

not capture pre-action effects by only analyzing trials with a total RT above 700ms (although

identical results were obtained with trials with shorter RTs included), and by verifying that

adding RT to the model did not capture the effects of offer parameters. I fit a neuron-by-

neuron GLM for spike count in that epoch with chosen and unchosen Forces and Rewards as

parameters. Then, I tested for population effects of each parameter by performing a second-

order t-test on the regression coefficients obtained.

The effect of the chosen Reward was significant for 7/54 neurons (6 positive and 1

negative), and was consistently positive at the level of the population (t(53)=3.21; p=0.0022,

Figure 61C). The effect of the chosen Force was significant for 2/54 neurons (1 positive and 1

negative) and was not significant at the level of the population (t(53)=-1.60; p=0.11). As for

21 Note that I could not find any later encoding of side press RT as a whole or of action-triggering time, even just before action onset. [-300;0ms] from side press onset, GLM for spike count with only action-triggering time as parameter: effect of action-triggering time significant for 10/50 neurons (5 positive and 5 negative), not significant across the population: t(49)=0.63; p=0.53.

Experimental results

223

the unchosen parameters, the effect of the unchosen Reward was significant for 5/54 neurons

(2 positive, 3 negative) and was marginally significant at the level of the population

(t(53)=-1.94; p=0.058). The effect of the unchosen Force was significant for 4/54 neurons (all

positive) and was not significant at the level of the population (t(53)=-0.50; p=0.62). These

results indicate that LC neurons did encode some of the information taken into account for

choices between alternatives, but not all of it. Indeed, in our population of LC neurons, I did

not find any consistent encoding of the Force parameters after cues onset, while there was an

encoding of the Rewards.

(A) Trials sorted by chosen Reward (B) Trials sorted by unchosen Reward

(C)

Figure 61: Encoding of offer parameters (Forces and Rewards) in a late epoch ([200:500ms])

after cue onset: LC neurons only encoded Rewards.

(A,B) Rasters of two example LC units showing the contrast of activity between different (A)

chosen or (B) unchosen Reward conditions. Activity is aligned on cues onset (vertical green

line). Magenta dots represent spikes when the monkey later chose the left side offer and purple

Experimental results

224

dots represent spikes when the monkey later chose the right side offer. Purple diamond-shaped

dots represent action onset. Note also that the LC neuron in (A) encoded RT positively after

cues onset, i.e. displayed more spikes just after cues onset when RT (green line to purple

diamond interval) would be longer.

(C) Summary of neuron-by neuron GLMs for spike count in the [200;500ms] epoch after cues

onset. Bars represent the mean coefficients for each parameter, across neurons, with chosen

and unchosen Force and Reward parameters. At the level of the whole population, there was

a significant positive encoding of the chosen Reward and a marginally significantly negative

encoding of the unchosen Reward. There was no significant encoding of the chosen and

unchosen Forces.

**: p<0.01; n.s.: non significant, F: Force, R: Reward, ctt : constant

(d) Increased activity of LC neurons around cues

onset was associated to increased probability to choose

high-Force option

Next, I questioned whether the activity of LC neurons around cues onset influenced

behavior, and especially choices. In Borderies et al., (2020), we showed that decreasing brain

NA led to a stronger effect of Effort on choices and overall less high-Effort choices. Here, I

tested whether I could find any signature of such an effect in LC activity around cues onset,

although I could not verify any causality link. In other words, I tested whether LC activity

around cues onset could predict the sensitivity of choices to Forces and/or the overall

propensity to choose high-Force offers. I conversely expected no relation between LC activity

and Reward-based decisions.

Using the same method as before (see II.B.3.a)(2)(e)), I separated choices between

high-Force versus low-Force or high-Reward versus low-Reward choices and fit logistic

regressions for high-Force or Reward choices. First, I fit a logistic regression for high-Force

choices, with the difference in Force (high minus low Force) and in Rewards (Reward of high-

Force option minus Reward of low-Force option), the interaction between the difference in

Forces and action-selection times, and spike count in the [-100;200ms] epoch around cues

onset. Over and above previously described effects22, there was a positive effect of spike count

22 Difference of Forces: β=-0.62, p<10-14; Difference of Rewards: β=4.26, p<10-124; Interaction between action-selection time and difference in Forces: β=-0.16, p=0.039

Experimental results

225

(β=0.14; p=0.048, Figure 62A). The interactions between spike count, action-selection time

and differences in Forces or Rewards were not significant if added to the model (p>>0.05).

Figure 62: LC activity around cues onset had a positive influence on the probability to choose

high-Force options, on the exerted force, especially in high-chosen Force conditions, and a

positive effect on performance.

Experimental results

226

(A) Logistic regression for high-Force choices (1 if the high-Force option was chosen, 0

otherwise). Over and above effects of the difference of Forces (high – low), of the difference of

Rewards (Reward of the high-Force option – Reward of the low-Force option), and of their

interaction with action-selection time, spike count in the [-100;200ms] peri-cues onset epoch

had a positive effect on the probability of high-Force choices.

(B) GLM for the maximum exerted force on the side grip. Over and above effects of offer Forces

and Rewards, chosen side of press, and action-triggering time, there was a positive effect of

spike count on the exerted force and a positive effect of the interaction between spike count

and the chosen Force.

(C) Logistic regression for performance of the side press. Over and above effects of offer Forces

and Rewards, side of press, and action-triggering time, there was a marginally significant

positive effect of the interaction between spike count and the chosen Force.

(C) Interaction plot showing the effect of the interaction between chosen Force and spike count

at cue onset (x-axis) on performance (y-axis). Spike count is split between below and above the

median (m).

*: p<0.05, ***: p<0.001; n.s.: non significant, F: Force, R: Reward, ctt : constant; Spk: spike.

Using the same procedure, I fit a logistic regression for high-Reward choices, with the

difference in Rewards (high-low Rewards) and in Forces (Force of high-Reward option – Force

of low-Reward option), and spike count in the [-100;200ms] epoch around cues onset. Over

and above previously described effects23, the effect of spike count was marginally significant

(β=0.21; p=0.073). There was no effect of the interactions between spike count and other

parameters (p>>0.05).

Importantly, the main effect of spike count on high-effort choices probability was not

present in the baseline ([-400;-100ms] pre-cue or [-300;0ms] pre-red dot onset), in the pre-or

peri-middle press epoch ([-200;0ms] and [0;200ms]) or in the pre- or post-action onset epoch

of the previous press ([-200;0ms] and [0;600ms] of the last trial with a press). This indicates

that this relation between LC activity and high-Force choices was specifically associated to

cues anticipation and processing. LC activity around the onset of the cues was associated to a

23 Difference of Forces: β=-1.90, p<10-39; difference in Rewards: β=1.51, p<10-24

Experimental results

227

higher probability to choose the high-Force options, but no difference in effort sensitivity. This

suggests that higher LC activity may have facilitated cognitive control exertion for the choice,

decreasing the bias towards avoiding high-Force offers. Conversely, there was no significant

relation between cues onset activity and Force-sensitivity (no interaction between Force

difference and spike count).

(e) Activity around cues onset had a positive effect

on side press exerted force in high-chosen Force

conditions

Next, I questioned whether this relationship between high-effort choices and peri-cues

activity could be accounted for by an increased capacity or willingness to perform the forces

on the side grip and therefore an increase in performance following increased LC activity

around cues onset. More specifically, I expected either one of two results: either increased

cues onset activity would lead to overall stronger presses, regardless of the force category,

i.e. spike count would have a global energization effect on side presses (main effect on exerted

force), or it would optimize the exerted force for the requested level of force (interaction with

chosen Force).

I pooled all the trials from all the recording sessions (with valid eye-tracking, 50

neurons) in which the monkey had pressed on the middle grip then made a choice (pressed

one of the side grips) and in which I could identify the time of the choice between offers (end

of action-selection time). I fit a GLM for the exerted force on the side grip. I used peri-cues

spike count ([-100;200ms]), action-triggering time, chosen and unchosen Forces, Side of press,

and the interaction between spike count and chosen Force as parameters. Besides identical

effects of offer parameters, side of press and action-triggering time as described earlier24,

there was a main effect of spike count (β=0.021; p=0.028) and positive interaction between

spike count and chosen Force25 (β=0.022; p=0.016, Figure 62B). This indicates that pre-middle

press spike count both had non-specific effects on the exerted force during the side press, and

chosen-Force specific effects, increasing the exerted force most strongly in conditions where

stronger force exertion was required.

24 Chosen Reward: β=0.045; p<10-4; Chosen Force: β=0.67; p<10-150; Unchosen Reward: β=-0.054; p<10-6; Unchosen Force: β=0.050; p<10-6; Action-triggering time: β=-0.040; p<10-4; Side of press: : β=0.44; p<10-112. 25 No other interaction was significant when included.

Experimental results

228

Consequently, I looked for effects of peri-cues onset spike count on performance.

Given that performance in case of chosen Force 1 was virtually at ceiling, I did not expect any

main effect of spike count on performance, but only when the monkey chose the Forces 2 or

3. Thus, I fit a logistic regression for performance with chosen and unchosen Forces and

Rewards, action-triggering time, side of press and the interaction between spike count and

the chosen Force as parameters. Over and above effects of offer parameters and action-

triggering time26, there was a marginally significant positive effect of the interaction between

spike count and the chosen Force on performance (β=0.039; p=0.069, Figure 62C&D).

The effect of the interaction between spike count and chosen Force on the exerted

force were already present before middle red dot onset ([-300;0ms]), in the baseline rate

(β=0.0099; p=0.013), but not the main effect of spike count (β=0.013; p=0.15). Since this

similarity could have been due to the strong correlation between baseline rate and rate

around cues onset (R=0.48; p<10-280), I tested whether a model using the baseline spike count

rather than the spike count around cues onset fit the data better. The model including the

baseline rate had a higher Bayesian Information Criterion (BIC=9915.3) than the model

including the rate around cues (BIC=9907.5). Moreover, adding the interaction between the

baseline spike count and the chosen Force to the model including the cues onset spike count

and its interaction with the chosen Force did not improve the fit of the model, as its BIC

increased (BIC=9919.4). Consequently, the relationship between cues onset spike count and

the exerted force on the side grip were not merely due to effects of baseline rate on the

exerted Force.

Regarding activity before and during side press ([-300;0ms] and [0;300ms]), the

interaction between spike count and the chosen Force was significantly positive, yet very small

(before press: β=0.0075; p=0.036; during press: β=0.0078; p=0.023) and there was no main

effect of spike count (before: β=-0.012; p=0.21; during: β=-0.0021; p=0.83). However, the fits

of these models were not as good as the fit of the one including cues onset rate instead (BICpre-

action rate=9918.3; BICperi-action rate=9918.4; BICperi-cues rate=9907.5), and adding the interaction

between pre-action rate and the chosen Force to the model with cues onset rate did not

improve model fit (BICpre-action rate=9921.2; BICperi-action rate=9919.7).

26 Chosen Reward: β=0.16; p=0.019; Chosen Force: β=-1.87; p<10-89; Unchosen Reward: β=-0.16; p=0.018; Unchosen Force: β=0.23; p=0.00015; Action-triggering time: β=-0.23; p<10-4; Side of press: : β=0.05; p=0.62.

Experimental results

229

Note also that such effects of the interaction between spike count and chosen Force

on the exerted force were not present for the spike count around the middle press (before,

[-200;0ms] or after, [0;200ms]), earlier during the inter-trial interval ([-450;-150ms] from

middle red dot onset), or later after cues onset ([200;500ms] after cues onset).

Thus, LC activity around cues onset had a chosen-Force dependent positive effect on

the exerted Force. In other words, increased LC activity around the time of offer presentation

predicted increased force exertion for the chosen action, especially in difficult conditions,

resulting in a tendency for an increase in performance in difficult conditions only. This suggests

that increased LC activity around cues onset facilitated physical effort production for the

chosen action, better adjusting the force exerted to the condition of the task. Indeed, in the

small Force condition (Force 1), in which almost every attempt to press the grip was successful,

increasing the exerted force was not instrumental, as it would not be beneficial for

performance. Conversely, in the intermediate Force condition (Force 2), and even more in the

high Force condition (Force 3), increasing the force increased the probability of success (and

of reward delivery).

These findings provides a potential mechanistic account for the relation between LC

activity and force sensitivity (Borderies et al., 2020), as it suggests that producing an effort (at

least physical effort) may have been less difficult after higher LC activity. This idea resonates

with the results presented above (see parts II.B.3.b)(1)(c) and II.B.3.b)(2)(d)) regarding the

relation between LC activity around the middle press and the subjective difficulty to engage

in the choice and trigger the chosen action, as well as the finding that increased LC activity

around cues onset led to a decreased bias against choosing high-Force offers. Indeed, these

findings suggest that following higher LC activity, the monkey found it easier to produce

cognitive control to repress his natural tendency to disengage from the task, and to inhibit the

bias towards avoiding high-effort offers.

Experimental results

230

(3) Side-press related LC activity

(a) Evoked activity around action onset

Various studies have reported that LC neurons activate just prior to action onset

(Bouret et al., 2012; Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004; Clayton et al.,

2004; Varazzani et al., 2015, Study 1, see part II.A), and maintain higher activity during

sustained actions such as presses on force grips (Varazzani et al., 2015). Here, inspection of LC

activity during the second step of the task revealed that LC neurons displayed bursts of activity

(a few spikes) between choice onset (gaze locking onto chosen option) and side press onset,

and/or during side press. I examined whether I could identify if activity between choice onset

and side action onset was better aligned onto choice onset or side action onset, using the

same method as the one used for activity between cues onset and choice onset. There was a

tendency for a better alignment of spikes onto the onset of the press rather that of the choice

(t(49)=1.77; p=0.83). Identically, during the side press, we examined whether we could

identify if LC spikes were better aligned onto the onset of the press or the attainment of the

maximum of exerted force. For all 54 neurons, after identifying the time of the peak of force

for each non-repeated trial in which the monkey made a side press, we compared the timings

between press onset and spikes, or spikes and time of peak of force. Across the population,

the comparison was not significant (t(49)=1.30; p=0.20).

We then quantified the proportion of neurons that responded to side action onset,

using the same technique as the one used for middle press and cues onset (neuron-by-neuron

t-tests to compare rate in window of interest to baseline rate, then second order t-test on T

values obtained for population effects). Before side action onset, in a [-300;0ms] epoch, we

compared rate to activity in a [-300;0ms] epoch before cues onset. 35/54 neurons exhibited a

change in rate (26 increased, and 9 decreased, Figure 63A), and there was a coherent increase

in rate across the population (second-order t-test: t(53)=2.56; p=0.013, Figure 63B). During

action execution, in a [0;300ms] from side action onset, 47/54 neurons exhibited a change in

rate (36 increased and 11 decreased, Figure 63A), and at the level of the whole population

there was a consistent increase in rate (t(53)=3.53; p<10-3, Figure 63B).

Experimental results

231

(A) (B)

Figure 63: Rate of LC neurons around side action onset, and results of first and second order t-

tests assessing rate changes.

(A) Activity of the 54 recorded LC units around action onset. Rate is z-scored by neuron. The

green dots indicate the mean timing of the onset of the cues by session i.e. by neuron.

(B) Density plots (bandwidth=5) of the T values from the first-order t-tests comparing rate in

the [-300;0ms] pre-action onset epoch and in the [0;300ms] post-action onset epoch to

baseline rate ([-300;0ms] epoch before cues onset).

(b) LC neurons encoded the difficulty of the action

during its execution

Based on the literature (Varazzani et al., 2015) and on what we found in our first study,

we expected LC neurons to encode the immediate difficulty to perform the action while the

monkey performed it. Practically, we expected LC neurons to encode the chosen Force and

the maximum exerted force during the press, and such an effect was visible on raster plots

(Figure 64A). We first fit a GLM with only offer parameters (chosen and unchosen Force and

Reward) for spike count in the [0;600ms] epoch after action onset, i.e. during the press (see

Figure 48 for time course of force exertion). Once again, we only used non-repeated trials in

which the monkey made a side press. Since eye-tracking was not required, all 54 neurons were

included. The effect of the chosen Force was significant for 22/54 neurons (20 positive, 2

Experimental results

232

negative) and was consistently positive across the population (t(53)=5.41; p<10-5, Figure 64B).

The effect of the chosen Reward was significant for 4/54 neurons (2 positive, 2 negative) and

was not consistent across the population (t(53)=0.41; p=0.69, Figure 64B). The effect of the

unchosen Reward was significant for 5/54 neurons (3 positive, 2 negative) and was

consistently negative across the population (t(53)=-2.35; p=0.023, Figure 64B). The effect of

the unchosen Force was significant for 4/54 neurons (2 positive, 2 negative) and was not

consistent across the population (t(53)=1.27; p=0.21, Figure 64B). Note that LC neurons

already encoded the chosen Force before action onset27.

We then fit a GLM for spike count in the same epoch with only the maximum exerted

force on the side grip as parameter. The effect of the maximum exerted force was significant

for 19/54 neurons (2 negative, 17 positive) and was consistently positive across the population

(t(53)=4.27; p<10-4, Figure 64C). When we added chosen and unchosen Forces and Rewards

to the model, the effect of the maximum exerted force was still significant for 13/54 neurons

(7 negative, 6 positive) and no longer significant across the population (t(53)=0.97; p=0.33,

Figure 64D).

Finally, in order to provide some conclusion on whether LC neurons encoded the

objective difficulty (chosen Force category) of the action or the actual resource engagement

in the action, we compared the fit of the model including only the chosen and unchosen Forces

and Rewards to the fit of the model including only the exerted force. For 24/54 neurons, the

model with the maximum exerted force alone fit the date better, and for 30/54 neurons, the

model with the chosen Force fit the data better. In these conditions, we cannot properly

conclude on whether LC neurons as a population encode one rather than the other. Our

results only provide an partial indication that LC neurons encode the difficulty of chosen

actions during their execution, at least the objective difficulty (required force).

27 [-250;0ms] epoch before action onset: Chosen Force: significant for 11/54 neurons (3 negative, 8 positive), consistently positive across the population (t(53)=2.56, p=0.013). Chosen Reward: significant for 6/54 neurons (4 negative, 2 positive), not consistent across the population (t(53)=-0.50; p=0.62). Unchosen Reward: significant for 4/54 neurons (1 negative, 3 positive), not consistent across the population (t(53)=1.04; p=0.30). Unchosen Force: significant for 3 neurons (1 negative, 2 positive), not consistent across the population (t(53)=1.24, p=0.22). NB: GLM with only action-triggering time, significant for 10/50 neurons (5 positive and 5 negative), not significant across the population (t(49)=0.63; p=0.53).

Experimental results

233

Figure 64: LC neurons encoded the difficulty of the action during its execution.

(A) Raster plot of the activity of an example LC unit around the time of action execution (around

action onset). Trials are split by chosen Force (F1, 2 or 3, top to bottom). Purple dots indicate

spikes in trials in which the left side offer was chosen and pink dots indicate spikes in trials in

which the right side offer was chosen. The pink line indicates action onset. The green dots

indicate cues onset. This unit was more active during the press when a higher Force was chosen

and executed.

(B-D) Summary (mean and SEM) of the coefficients of the neuron-by-neuron GLMs for spike

count with (B) chosen and unchosen Forces and Rewards, (C) the maximum exerted force, or

(D) all the latter as parameters. (B) LC neurons encoded the chosen Force and the unchosen

Reward negatively. (C) When the maximum exerted force was alone in the model, it had a

positive effect on spike count, but (D) this effect was not significant over and above the effects

of task parameters.

F1

F2

F3

Experimental results

234

(c) Higher press-related LC activity at trial N was

associated to increased Force sensitivity of choices at

trial N+1

Next, we questioned whether the activity of LC neurons around the time of the press,

at the time when they encode the exerted force, and therefore putatively the physical effort,

could have an influence on the choice in the next trial. Specifically, if LC neurons participate in

the monitoring of effort exertion to subsequently adjust effort allocation, we would expect

that increased LC activity at the time of the exertion, putatively reflecting the resources

invested in the action, would lead to increased effort sensitivity in the next trial.

To test that hypothesis, in order to specifically focus on effort-based choices, we

focused on trials in which the monkey made a choice (made a side press) and had made a

choice in the previous trial, and in which the two offers presented had identical Rewards, i.e.

did not differ in the Reward dimension, but differed in the Force dimension. We split trials

between high-Force choices (the offer with the highest Force proposed was chosen) or low-

Force choices (the offer with the lowest Force proposed was chosen). We fit a logistic

regression for high-Force choice (1 if the monkey made a high-Force choice, 0 if he made a

low-Force choice) with the difference in Forces offered (high minus low), action-selection

time, the Force exerted in the previous trial, and the interaction between spike count in the

previous trial around side press ([0;600ms] from press onset) and the difference in offered

Forces as parameters.

There was a significant negative effect of the interaction between spike count during

the previous press and the Force difference (β=-0.30; p=0.037, Figure 65A), indicating that

increased spike count around the previous press increased the Force sensitivity of Force-based

choices in the next trial (Figure 65B). Besides this effect, the difference in Forces had a negative

effect (β=-0.62; p=0.00020) indicating that the monkey was more reluctant to choose the high-

Force option if it had a much higher Force requirement than the alternative option. Moreover,

there was a positive effect of action-selection time (β=0.28; p=0.013, Figure 65A) indicating

that longer action-selection times were associated with higher probability to choose the high-

Force offer. Finally, the maximum exerted force of the previous trial had a positive effect

(β=0.81; p<10-11, Figure 65A). Importantly, the effect of the interaction between spike count

and Force difference was also true when considering only trials following correct execution of

Experimental results

235

the side press, meaning that it was not due to the increased probability of failure for high-

Force conditions.

The effect of the interaction between spike count and the Force difference was also

present when considering spike count around cues onset ([-100;200ms] from cues onset in

the same trial as the choice, β=-0.30; p=0.010), but the model using the spike count related to

the previous press fit the data considerably better (BICcues = 735.49; BICprevious press=484.8). I

could not find any evidence for such an effect at any other time during the trial or in the

baseline (inter-trial interval) rate.

Figure 65: Increased spike count during the previous side press was associated to increased

Force sensitivity of the next Force-based choice.

(A) Coefficients of the logistic regression for Force-based choices (offers with identical Reward

size and different Forces). Over and above effects of the Force difference and action-selection

time of the current trial, and of the exerted Force of the previous press, the interaction between

the spike count of the LC neurons during the execution of the previous chosen action and the

difference of Forces (high minus low Force) had a negative effect on the probability to make

high-Force choices.

(B) Interaction plot showing the interaction between the effect of the Force difference (x-axis)

and the spike count during the previous press (split between spike count above, dotted line, or

Experimental results

236

below median spike count m, full line) on the probability to make high-Force choices (y-axis).

When spike count during the previous side press increased, the effect of the difference in Force

was stronger, indicating that Force sensitivity increased. *: p<0.05; **: p<0.01; ***: p<0.001;

n.s.: non significant; m: median;

Altogether, these results indicate that the spike count of LC neurons during the chosen

action at trial N scaled with the difficulty and possibly the effort produced for action execution.

Moreover, increased spike count around that time was associated to increased Force

sensitivity of choices at trial N+1. In other words, LC activity during the press possibly reflected

the physical effort produced to execute it, and increased activity in that period of time was

associated to increased physical effort avoidance the next time the monkey faced a binary

choice. One of the working hypotheses of this study was that the activity of LC neurons during

effort production could serve as a feedback for the effort allocating system, used to adjust

future effort-based decisions. In order to regulate resource expenditure, following intense

effort production, the system should favor options requiring less effort. Our findings show

that LC activity during actions (at least actions requiring physical effort production) could

provide information about resource expenditure, and is associated to modulations of effort

allocation strategies. Thus, this data suggests that effort-related LC activity could be

responsible for adjusting effort sensitivity for subsequent choices based on the a posteriori

evaluation of effort exertion.

(4) Inter-trial activity of LC neurons

(a) LC activity increased during the ITI following

failures to execute the side press

LC activity has been proposed to participate in action-monitoring and especially error

monitoring. Jahn and colleagues (2020) reported increased LC activity following errors

(fixation breaks and therefore no engagement), scaling with the probability that the monkey

would re-engage in the task, suggesting that LC activity signals errors and subsequently

enhances resource mobilization for engagement. I aimed at extending this finding beyond

participation mistakes (failures to engage). If LC neurons do track mistakes, in this task, I would

expect LC neurons to increase their activity after failures to engage and after failed presses.

Experimental results

237

ITIs were 475±25ms long, and, in the case of successful trials, followed reward delivery. Based

on raster inspection (Figure 66A), I used a [-450;-150ms] epoch from the onset of the next trial

(middle red dot) to analyze ITI activity, in order to avoid any confusion with Reward delivery-

related activity.

(A) Activity during ITI

Figure 66: LC neurons encoded performance of previous side press, but not engagement in

previous side press during the ITI ([-450;-150ms] before middle red dot onset).

Experimental results

238

(A) Raster plot of the activity of an example LC unit during the ITI, showing only ITIs following

engagement in choices, and separated based on the outcome of the side press (failure, top, or

success, bottom).

(B,C,D) Summary of the neuron-by-neuron GLMs for spike count during the ITI.

(A) LC neurons as a population did not significantly encode engagement in the choice in the

previous trial (N-1) during the ITI.

(C) LC neurons encoded performance in the previous trial (N-1) negatively, i.e. were more active

after a failure.

(D) LC neurons encoded performance in the previous trial (N-1) negatively, and there was a

marginally significant effect of the interaction between performance and chosen Reward of

the previous trial.

*: p<0.05; n.s.: non significant, ctt : constant

Regarding ITIs following failure to engage, I focused on ITIs following trials in which the

monkey had made a middle press, and separated them between trials in which the monkey

made a side press (choice) or not. Trials in which no middle press was initiated were excluded.

I fit a GLM for spike count in the ITI window ([-450;-150ms] from middle red dot onset) with

engagement in the previous side press as parameter (0 if no press was made on the side, 1

otherwise, regardless of performance of side press). I had to exclude 7/54 neurons due to

insufficient number of trials in which the monkey did not engage in the choice after

performing the middle press. The effect of engagement in the previous choice was significant

for 10/47 neurons (9 negative, 1 positive). There was no effect of engagement in choice in the

previous trial on ITI activity at the level of the population (second order t-test: t(46)=-0.90;

p=0.37, Figure 66B).

Regarding ITIs following failure to perform the side press, I focused on ITIs following

attempts to perform the side press, and split them between ITIs following failure or success in

performing the side press. I then fit neuron-by-neuron GLMs on ITI spike count with

performance of the side press (1 if success, 0 if failure) as parameter. The effect of

performance of the previous trial was significant for 15/54 neurons (13 negative, 2 positive),

and the effect was consistently negative across the population (second order t-test: t(53)=-

1.87; p=0.047), indicating that LC neurons displayed increased spike count after a failed side

press (Figure 66A&C).

Experimental results

239

This encoding of performance could, in fact, correspond to an encoding of Reward

prediction error. Indeed, there would be a high prediction error in case of failed presses,

especially if the missed Reward was big. To test that, I added the interaction between

performance and Reward at stake for the previous press to the previous model. The effects of

performance was still significant for 15/54 neurons (13 negative, 2 positive) and significant

across the population (second order t-test: t(53)=-2.03; p=0.048, Figure 66D). The effect of the

interaction between the Reward and performance of the previous trial was significant for

17/54 neurons (12 negative, 5 positive) and was marginally significantly negative at the level

of the population (t(53)=-1.85; p=0.071). This indicates that the relation between LC activity

during the ITI and performance was not confounded with an encoding of Reward prediction

error. Considering the marginally significant result obtained for the interaction between

performance and Reward at stake, it cannot be excluded that LC neurons do encode Reward

prediction errors, although this would deserve further investigation.

Overall, despite not replicating the finding by Jahn and colleagues (2020) regarding the

response of LC neurons to commitment errors, I showed that LC neurons discriminated

between correct and incorrect performance, with a higher spike count after incorrect

performance. In other words, in this task, LC neurons responded to execution errors.

Discussion

240

III. Discussion

The work conducted during my PhD aimed at clarifying the role of the LC/NA system in

effort processing. We hypothesized that LC neurons provide an online and domain-general

representation of effort production, and signal some of the consequences of effort allocation,

or at least insufficient effort allocation, such as errors. We proposed that these modulations

of activity could serve as a feedback signal for structures making the decisions to allocate

effort (cognitive control and physical effort). Moreover, we hypothesized that LC activity had

beneficial short-term effects on effort production.

In three tasks involving investing effort, physical effort and/or cognitive control, we

consistently showed that LC neurons encoded the difficulty of engaging in the actions and the

difficulty of executing them. More precisely, before action onset, LC neurons encoded RT, i.e.

the time taken to respond by triggering the actions, which, at least in the Delay-discounting

and two-alternative Force-discounting task, scaled negatively with the subjective cost of

engaging in actions. Additionally, during the action itself, LC neurons encoded the force

exerted by the monkeys in the two tasks involving differential force exertion.

In the second study, I showed that LC activity around costly task events (engaging in

actions and executing them) had delayed effects that could be separated into two

subcategories: short-term within trial effects, and long-term across trials effects. First, in

periods of time in which LC neurons encoded the subjective difficulty to engage in the current

action, LC activity had delayed within-trial facilitating effects on future effort production,

which could be accounted for by a decrease in the subjective cost of engaging in future actions

and executing them. This effect was domain-general, meaning that increased LC activity was

associated to enhancements of cognitive control and physical effort production. Second,

across-trials, increased LC activity around the time of exertion of the physical effort (grip press)

was associated with increased sensitivity of the following choice to the effort dimension,

suggesting that it served as a learning signal for structures participating in effort-based

decision-making.

In this part, I discuss the findings of the two studies detailed in previous parts of this

manuscript, their implications with regards to the literature of effort allocation and production

and of LC neurophysiology, as well as potential perspectives for future studies.

Discussion

241

A. Behavior: signatures of effort in one- and two-

alternative tasks

In the two studies detailed earlier, three different tasks manipulating physical and

mental effort were used: one-alternative Force-discounting and Delay-discounting tasks, and

a two-alternative Force-discounting task. In the one- and two-alternative Force-discounting

tasks, high-Force options were perceived as aversive, i.e. were avoided more often than low-

Force options. Indeed, higher Forces led to decreased acceptance rates (e.g. Jahn et al., 2020;

Varazzani et al., 2015; Zénon et al., 2014) and were chosen less often (e.g. Borderies et al.,

2020; Cousins and Salamone, 1994; Floresco et al., 2008; Salamone et al., 2007; Stevens et al.,

2005; Walton et al., 2006). This pattern followed the law of least effort (Hull, 1943). In the

Delay-discounting task, high-Delay options were refused more often (Falcone et al., 2019;

Minamimoto et al., 2012, 2009b), as would be expected by a discounting of Reward values by

Delay (e.g. Green et al., 2013; Kirby and Maraković, 1996; Madden et al., 2003; McKerchar et

al., 2009; Prévost et al., 2010; Rachlin and Jones, 2008; Shamosh and Gray, 2008). Overall, our

results indicate that monkeys perceived high-cost options as more aversive.

1. Cognitive control exertion for decisions

Goal-directed behavior relies on the representation of the relation between actions

and their outcomes, i.e. of subjective utility (Balleine and Dickinson, 1998; Dickinson and

Balleine, 1994; Valentin et al., 2007; Zwosta et al., 2015). Cognitive control serves to maximize

subjective utility (Kool and Botvinick, 2018). In the particular case of decision-making, exerting

control allows overcoming biases arising from the use of automatic processing (Shiffrin and

Schneider, 1977) and therefore overriding default responses (Epstein, 1994; Sloman, 1996).

This role was particularly emphasized by pupillometry studies showing that increased pupil

diameter, a proxy for the invested cognitive control (Alnæs et al., 2014; Wahn et al., 2016;

Wright et al., 2013), was associated to decreased decision biases (de Gee et al., 2020, 2017,

2014).

In our three effort-discounting tasks, a first step at which behavior had to be adjusted

in order to maximize utility was the engagement in trials. Monkeys could refuse to engage in

trials based on cues information, although rejected options would be offered again until they

Discussion

242

were completed. Refusing trials led to lower reward rates, and therefore lower utility28, as it

led to increased delay to reward delivery. This was especially true in the two-alternative Force-

discounting task, which had longer trials since they included a middle press before offers were

made. Considering that high-cost/low-reward offers were perceived as aversive, the

automatic behavior would have been disengaging from the task when facing low-value offers.

Since cognitive control is required to override decision biases and automatic behavior,

choosing to engage in such trials probably required exerting some cognitive control. Engaged

trials could therefore be seen as trials during which sufficient control was allocated such that

the monkey decided to commit. Nonetheless, in the absence of formal evidence that decisions

to engage were goal-directed, it cannot be considered as certain. Indeed, it could still have

been an automatic Pavlovian approach behavior (Dickinson, 1980; Timberlake, 1994).

In the second study, besides engaging in trials, maximizing utility involved making

choices that minimized force exertion while maximizing rewards. Behavioral analyses showed

that the monkey’s choices were compatible with a cost-benefit trade-off. Still, rather than

engaging cognitive control and performing extensive computations to decide optimally, he

could have relied on simplifying decision strategies (Gigerenzer, 2010; Gigerenzer and

Goldstein, 1996; Sidarus et al., 2019; Tversky and Kahneman, 1974), such as perceptual

strategies (locating the offer with the biggest reward cue), at least for a certain proportion of

the trials. Indeed, the monkey had been extensively trained and knew the cues very well.

Additionally, the monkey could have relied on Pavlovian associations (Dickinson and Balleine,

1994). Such dual strategies of response could account for the bimodal distribution of action-

selection times (Figure 51A), with the fast times corresponding to trials in which the monkey

responded automatically, and slow trials corresponding to more complex computations and

investment of cognitive control. In coherence with that hypothesis, long action-selection times

were associated to lower sensitivity of choices to Forces, indicating that longer action-

selection times may have been associated to cognitive control exertion for the decision

process, facilitating decisions away from the default of choosing low-effort options (Epstein,

28 Note that it cannot be excluded that, in some situations, engaging in the action may have carried disutility (the

expected value of engaging may have been negative), in case of fluctuations in perceived reward values and effort costs (see Figure 11, Shenhav et al., 2013), and that refusals to engage (at least some of them) still allowed maximizing utility. Additionally, such disutility of engaging has been theorized in economics in the case of high-risk situations (Von Neumann and Morgenstern, 1947), and this effect could account for the refusals of high-force trials in the force-discounting tasks, in which high-Force category performance was relatively low compared to the performance in low-Force conditions.

Discussion

243

1994; Sloman, 1996). Note, still, that this assumption remains speculative, as the behavioral

analyses did not provide any additional evidence for the relationship between action-selection

times and cognitive control exertion for choices between offers.

2. Cognitive control for triggering actions

Once the monkeys had reached the decision to engage in courses of action involving

force exertion or delay to the Reward, cognitive control was probably needed to trigger the

action. Action-triggering can be thought of as a separate decision process, lasting from the go-

signal to the onset of the action. Indeed, in the one-option Force-discounting task used in

Study 1 (same dataset), Jahn and colleagues (2020) showed that fixation breaks signaling

disengagement from trials virtually always happened before go-signal onset, suggesting that

monkeys decided whether to engage between cue onset and go-signal onset, and only

triggered the action after the go-signal. We made the assumption that the same was true in

the Delay-discounting task, in which we could not precisely time refusals since gaze fixation

was not imposed. Additionally, in the two-alternative Force discounting task, gaze patterns (in

trials in which the monkey made a choice) were compatible, in most trials, with a sequential

process of choosing between alternatives before triggering the chosen action, a temporal

dissociation also described in EEG data (Dmochowski and Norcia, 2015).

In the context of value-based decision-making, RT had repeatedly been shown to scale

with decision difficulty (Hunt et al., 2012; Krajbich et al., 2010; Krajbich and Rangel, 2011;

Milosavljevic et al., 2010; Morris et al., 2019; Philiastides and Ratcliff, 2013), and to increase

with offer costs (Hunt et al., 2012; McGuire and Botvinick, 2010). In the two-alternative Force-

discounting tasks and the Delay-discounting task, the time taken to trigger the actions29

decreased with the reward and increased with the cost of the triggered option, as well as

during periods of time and in task conditions associated to low engagement rates. This

indicates that the time taken to trigger the actions reflected the difficulty of the decision

process of triggering the action. Note that we could not find any strong evidence for the

relationship between RT and the difficulty of triggering actions in the one-option Force

discounting task, supposedly because of strong motor effects on RT.

29 RT in the one-option tasks, middle press RT and action-triggering time in Study 2.

Discussion

244

Under the assumption that the rate of resource mobilization for cognitive processes is

constant across time, i.e. that longer time taken for any process is associated to increased

resource mobilization, the time taken to trigger the actions would have scaled with the

cognitive control invested in that decision process. Indeed, triggering the action was binary

(either successful or not, no in-between). Therefore, the optimal amount of control invested

in the action-triggering process would be exactly proportional to the difficulty, and therefore

just sufficient to trigger the action. This constitutes a rather speculative reasoning, which could

tend to be circular (Navon, 1984; Shenhav et al., 2017), thus the relationship between the time

taken to trigger actions and the cognitive control invested in the process remains hypothetical.

3. Physical effort to execute presses

In two of the force-discounting tasks used, reward delivery was conditioned by the

exertion of specific amounts of force on a grip. Following a long time taken to trigger the

actions, in the two Force-discounting tasks, monkeys usually exerted less force on the grip.

This means that if the monkey perceived force exertion as aversive, he took longer to trigger

the chosen action and did not press as hard on the grip. In other words, the monkey had a

stronger bias towards avoiding actions perceived as more costly (subjectively more difficult)

and was not as good at executing them i.e. at producing the physical effort required. This

pattern is compatible with the existence of a global computational variable, effort,

representing the resources required for specific courses of action, and encompassing both

their aversiveness when making choices, and therefore the difficulty to trigger them, and the

difficulty of executing them (Borderies et al., 2020). When the cost of specific actions

increased, the monkey was less willing to trigger them and had more difficulties mobilizing

the required resources to execute the action.

However, there could be at least one alternative interpretation for the negative

relationship between the time taken to trigger actions and the force exerted. Mental fatigue

caused by cognitive control exertion has a well-described negative impact on physical effort

performance (e.g. Boksem and Tops, 2008; Martin et al., 2018; McMorris, 2020; Schiphof-

Godart et al., 2018; Van Cutsem et al., 2017; reviewed by Habay et al., 2021). Under the

assumption that the time taken to trigger the action scaled with the cognitive control exerted

Discussion

245

to trigger it, such “ego-depletion” effects could have explained the decrease in force exerted

observed following long RTs.

B. LC activation may signal the end of decision

processes

247 LC neurons from five monkeys were recorded in four tasks: three effort-

discounting tasks (a Delay-discounting task and two Force discounting tasks, one-option and

two-alternative) and a target-detection task. In all three effort-discounting tasks, LC neurons

increased their rate both around cues onset (for one-option force-discounting task, see Jahn

et al., 2020 and Varazzani et al., 2015 for details on cue-related activity) and just before

action(s) onset (bar press(es) in Force-discounting tasks and bar release in Delay-discounting

task). This pattern is highly reminiscent of what was previously found in a variety of tasks (cue

onset: Bouret et al., 2012; Bouret and Richmond, 2009; Jahn et al., 2020; Varazzani et al., 2015;

action onset: Bouret et al., 2012; Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004;

Clayton et al., 2004; Varazzani et al., 2015). The second study extended this finding to a task

with no artificial decomposition of the different steps of decisions, as monkeys were not

forced to wait between cues presentation and action onset, and could produce the chosen

actions as soon as they wanted. Identically, LC neurons displayed increased activity after cues

onset and before action onset, although the decreased temporal separation between these

events in the new paradigm made the distinction more difficult to assess.

LC activity has been proposed in the past to signal decisions (Clayton et al., 2004;

Rajkowski et al., 2004) and specifically their end (Shea-Brown et al., 2008). Moreover, pupil

diameter, a proxy for LC activity (Joshi et al., 2016; Varazzani et al., 2015), is known to increase

around the time of decisions (Beatty, 1982; de Gee et al., 2014; Gilzenrat et al., 2010; Lempert

et al., 2015; Nassar et al., 2012; Figure 6A), even in the absence of immediate responses

materializing them (Einhäuser et al., 2010). In value-based decision tasks, including the Force-

and Delay- discounting tasks presented earlier, the discharge of LC neurons could have been

evoked by the conclusion of two decision processes assumed to happen between cues onset

and press onset: choosing to engage (and between offers in case of multiple offers) after cues

onset, and triggering the (chosen) action. In the two-alternative choice task, the timing of the

end of the action-selection process was approximated using the last saccade onto chosen

Discussion

246

offer. However, I could not identify whether LC spikes occurring between cues and last

saccade onset, or between last saccade and side press onset, were better aligned onto either

one of these events. This negative result must be taken with caution, as it cannot be excluded

that it came from a lack of statistical power, or heterogeneity of the population of neurons.

Still, it could indicate that LC activity was neither evoked by cues onset, by the saccade onto

the chosen offer or by action onset, but by the termination of decision processes. Indeed, LC

neurons could, first, signal the end of the action-selection decisions. These decisions were

probably initiated by cues onset, and their end led to the saccade onto the chosen offer. Then,

before action onset, LC neurons could respond to the end of the action-triggering decisions.

These decision processes probably started around the time of the saccade and their end led

to action onset. The possibility that LC neurons may activate around the end of decisions had

already been suggested by Shea-Brown and colleagues (2008).

If that were true, first, it would explain why latencies of responses after cues onset in

the two-alternative Force-discounting task (Study 2, Figure 59A) seemed to be longer than

those previously observed in one-alternative tasks, in our first study and others (Figure 41A;

Fig. 3G from Varazzani et al., 2015, Fig. 3C from Bouret et al., 2012). Indeed, choosing whether

to engage and which offer to pick would take longer than deciding whether to engage when

facing only one offer, due to differences in the amount of information to process (Bettman et

al., 1990, 1990; Carpenter and Just, 1975; Huber, 1980; Johnson and Payne, 1985; Newell and

Simon, 1972; Payne et al., 1988; Shugan, 1980). Second, LC evoked responses by decision

termination could provide a mechanistic explanation for the generally higher variability in

latencies of cue-related responses compared to action-related responses (Delay-discounting

task: Figure 41A, and Fig. 3A from Bornert and Bouret, in prep; Fig. 3G&H from Varazzani et

al., 2015; Bouret et al., 2012, Fig 3C&D). Identically, the higher variability in the latencies of

pre-action responses in case of triggering complex motor actions compared to simple ones

could result from a higher variability in the time required to generate and transmit complex

motor commands at the end of the action-triggering process (differential force requirement

vs bar release, Bornert and Bouret, in prep, Fig. 3A&B). Furthermore, it could account for the

apparent absence of increase in LC activity after cue onset when cues were previously known

(Jahn et al., 2020), as the decision whether to engage may have been made even before cues

appeared, and may not be sufficiently aligned onto cue onset to detect coherent changes in

LC firing.

Discussion

247

In the target-detection task, LC neurons displayed one peak of activity, during the

action. In tasks involving perceptual decisions (visual or olfactory), LC neurons had repeatedly

been shown to respond only once during trials, with no response to stimulus (cue) onset and

only a response to action onset (Bouret and Sara, 2004; Clayton et al., 2004; Rajkowski et al.,

2004). In the theoretical frame explained above, it would mean that LC neurons only

responded at the end of the action-triggering process, but not to the target-detection process.

In other words, it would suggest that LC activity is in fact only evoked by the termination of

decisions associated to some cognitive control exertion. In that sense, the increase in LC

activity observed around cues onset in value-based decision task probably does not have the

same cause and function as the well-described peak of LC activity observed in response to

surprising stimuli such as sudden sounds (Bouret and Richmond, 2009; Bouret and Sara, 2004;

Grant et al., 1988; Vankov et al., 1995, Figure 20).

The pattern of activation of LC units observed in our target-detection task and other

perceptual tasks however contrasts with findings by Kalwani and colleagues (2014), who

showed that, in a Go/No-Go task, LC neurons responded both to the go signal and to action

onset (saccade). This difference in pattern may in fact reinforce the idea that LC neurons

respond at the end of costly decision processes. Indeed, in the countermanding task used by

Kalwani and colleagues (2014), when the Go-signal was detected, even in the absence of

doubt regarding its presence, there was an uncertainty regarding the correct response to

produce, as the stop-signal could have appeared later on during the trial. Consequently, the

monkeys may have exerted some cognitive control to deliberate and decide whether to

produce the saccade. In other words, in the task used by Kalwani and colleagues (2014), there

may have been an additional decision process to decide whether to engage based on the

probability of the Go-response being correct, compared to tasks used by Bouret and Sara

(2004), Clayton and colleagues (2004) and Rajkowski and colleagues (2014). Consequently, LC

neurons may have responded to the termination of that decision process. Additionally, LC

neurons have been hypothesized to encode uncertainty (Jepma et al., 2018; Jepma and

Nieuwenhuis, 2011; Nassar et al., 2012; Payzan-LeNestour et al., 2013), and this evoked

response to the go-signal in the task by Kalwani and colleagues may have reflected the

uncertainty about the correct response (Go or No-Go).

Discussion

248

Interestingly, in the task used by Kalwani and colleagues (2014) and our target-

detection task, the action-related increase in activity of LC neurons occurred after action

onset. This may have been the consequence of the implementation of the motor command

(action onset) being faster than the onset of the activation of LC neurons evoked by the

termination of the action-triggering process.

In the three tasks involving force exertion on a grip, LC neurons were active during the

press, and especially during the first half of the press. The timing of this activation was

compatible with it happening between press onset and peak of force attainment, as it started

after action onset and lasted about the same time as the onset-to peak time. This report of

increased activity during strenuous actions is coherent with previous reports of LC single unit

activity (Varazzani et al., 2015) and pupil dilation increases during force exertion (Varazzani et

al., 2015; Zénon et al., 2014). For one of the presses (middle press in two-alternative choice

task), I could determine that LC activity following press onset was in fact better aligned onto

the end of the ascending phase of the force exertion, i.e. the attainment of the peak in force.

This could indicate that LC neurons activated at the end of the contraction, i.e. signaled the

end of the motor program. For the other press (chosen action) of that task, I could not identify

whether LC activity was better aligned onto the onset of the press or the attainment of the

peak of force. This could have resulted from the increased diversity of movements required,

which could have introduced variability in the time taken to terminate the motor action after

the onset of the corresponding motor signal. Indeed, the side presses involved the use of two

grips and required different levels of force exertion.

The timing of responses of LC neurons to task events, probably at the very end of

decision processes, is quite incompatible with a direct influence on these decision processes.

For instance, consistently across our tasks and those used across numerous other studies, the

activation of LC neurons evoked by action onset appeared just prior to it (Bouret et al., 2012;

Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004; Clayton et al., 2004; Varazzani et

al., 2015), or even sometimes just after it (Kalwani et al., 2014). Since LC axons are not

myelinated and conduct inputs rather slowly, pre-action LC activity would eventually reach

targets at least 130ms after their generation in LC neurons, i.e. too late to participate in the

decision process itself, especially if action-evoked activity did not precede the action. Bouret

Discussion

249

and Sara (Bouret and Sara, 2005; Sara and Bouret, 2012) have proposed that LC activity could

serve as a reset signal, facilitating or even causing the transition from one cognitive state to

another by provoking network reorganization. Although the evidence for an evoked response

of LC neurons to the end of decision processes (for instance selecting courses of actions and

triggering them) or motor processes (presses) is at this point mostly indirect and

circumstantial, it would be compatible with that theoretical framework. Still, additional

studies would be needed to provide more precise dissections of the timing of LC activity with

regards to decisions. Additionally, recording simultaneously in candidate target areas for

network reconfiguration and the LC could provide a direct confirmation of the network reset

theory.

C. LC neurons encode subjective difficulty, and

possibly effort, to engage in actions and execute them

Reliably across three tasks mobilizing cognitive effort to trigger costly actions, LC

neurons encoded the time taken to trigger the actions, before they were triggered, over and

above the encoding of task parameters influencing the difficulty of the action. As discussed

before, this time probably reflected the subjective difficulty to engage in actions, and may

have scaled with the cognitive control invested in triggering the action. Additionally, the fact

that the encoding of RT (or more specifically action-triggering time in Study 2) was present

even when including task parameters in the models indicates that this encoding was not

merely due to the correlation between RT (or action-triggering time) and task parameters, i.e.

objective difficulty. Rather, LC neurons encoded the subjective difficulty of triggering actions

on a trial-by-trial basis, and possibly the cognitive control invested. This encoding of the time

taken to trigger actions is reminiscent of positive correlations observed between RT and either

pupil diameter or population LC activity around action onset (respectively: de Gee et al., 2020;

Gilzenrat et al., 2010, experiment 3, and: Bari et al., 2020, Figure 25).

Although we could not provide any strong evidence for the consistency of this link

between RT and the subjective difficulty of triggering the press in the one-option Force

discounting task (Study 1), probably due to strong motor effects on RT, LC neurons still reliably

encoded RT prior to press onset. Conversely, we could not find any encoding of RT prior to

press onset in the target-detection task, in which RT was clearly affected by similar motor

Discussion

250

effects, but very probably not by subjective difficulty effects. Obviously, we cannot exclude

that the absence of reported effect of RT around press onset in the target-detection task did

not merely result from a lack of statistical power due to a low number of neurons. Still, we are

not the first to report an absence of relationship between LC activity and RT in conditions in

which RT does not reflect the cognitive control required to trigger the action. Indeed, Bari and

colleagues (2020) reported a positive correlation between RT and LC activity between target

onset and response onset only in the presence of distractors in a modified Posner task, but

not in the presence of neutral cues (Figure 25). This suggests that, in their task too, LC neurons

probably encoded RT only when it was related to decision difficulty, i.e. when distractor

presence called for cognitive control investment to produce the correct response, and not to

sensory-motor demands, i.e. in the absence of a distractor.

In the two tasks mobilizing physical effort through differential force exertion on a grip,

LC neurons additionally encoded the required force and the force exerted on the grip during

the presses, suggesting that they encoded the difficulty of the action and probably the physical

effort associated. Interestingly, this effect was not present for the small force exertion

required in the Target-detection task or the middle press of the two-alternative Force-

discounting task. One possible explanation for these results would be that the required force

was so small that the monkey did not perceive variations of exerted force as variations in effort

costs. Note that in the two Force-discounting tasks, around the time of the press requiring

differential force exertion, we could not find any encoding of the exerted force on the grip by

LC neurons if we restricted our analysis to trials in which the small Force was executed.

Similarly, in a Force-discounting task with four Force requirement levels, Zénon and colleagues

(2014) did not report any effect of the increase in exerted force between the two conditions

with the smallest Force requirements on pupil dilation during the press (Figure 3A). These

findings argue for an encoding of the subjective difficulty of the press rather than of the force

actually exerted on the grip.

Overall, the findings reported throughout the two studies indicate that LC neurons

encoded the subjective difficulty, and possibly the invested cognitive and physical effort of

triggering and executing actions. In other words, LC neurons probably encoded the effort

produced at the time of its production, and this encoding was domain-general: both physical

Discussion

251

and mental effort were encoded. The question of the source of these modulations cannot be

addressed with certainty by the results presented, but a hypothesis can be formulated. A vast

body of literature reports a global sympathetic activation, evidenced by pupil diameter

increases, in response to both physical (Varazzani et al., 2015; Zénon et al., 2014) and mental

effort exertion (Alnæs et al., 2014; Hess and Polt, 1964; Kahneman and Beatty, 1966; Simpson

and Hale, 1969; Wahn et al., 2016; Wright et al., 2013). The encoding of subjective difficulty,

and possibly exerted effort, by LC neurons could result from inputs from the autonomic

system, in particular the nucleus Paragigantocellularis (PGi). The PGi is a nucleus from the

rostral medulla responsible for the respiratory and cardiovascular adaptations triggered by

sympathetic activation, and one of the two major inputs to the LC (Aston-Jones et al., 1991,

1986; Ennis and Aston-Jones, 1988, 1986; Van Bockstaele and Aston-Jones, 1995). However,

Ennis and Aston Jones (1988) report very short latencies between PGi activation and LC

activation, as inputs from the PGi to the LC produced LC responses around 11ms after their

onset. Autonomic activation increases during effort production. Under the assumption that

the PGi does encode an approximation of the mental or physical effort produced, via an

encoding of the degree of autonomic activation, PGi activity would be higher throughout

periods of effort production, such as decision periods. Given the fast effects of PGi activity on

LC activity, one would expect effort production-associated LC responses to appear from the

beginning of effort production. In our tasks, decisions provide an example of processes

associated to effort production that last for a few hundred milliseconds. Contrary to what

would be expected if effort-related modulations of LC activity were driven by PGi inputs, we

showed that there is converging evidence that the LC activates and encodes the effort

produced for the decisions at their end.

The modulations of LC activity with subjective difficulty (and possibly effort exertion)

could also result from inputs from cortical regions known to encode effort: the dorsal anterior

cingulate cortex (dACC), the anterior insula and the posterior parietal cortex (e.g. Burke et al.,

2013; Bush et al., 2002; Croxson et al., 2009; Hare et al., 2011; Kurniawan et al., 2013; Prévost

et al., 2010; Samanez-Larkin et al., 2008; Skvortsova et al., 2014). The ACC is of particular

interest as fMRI studies (Croxson et al., 2009; Forstmann et al., 2006; Mars et al., 2005; Morris

et al., 2019; Shenhav et al., 2014; Walton et al., 2004; Yoshida and Ishii, 2006) as well as single-

unit recordings (Hillman and Bilkey, 2010; Kennerley et al., 2009, Lespart and Bouret,

unpublished data) have reported increased activity during decisions as well as difficulty and

Discussion

252

effort encodings, in both physical effort tasks and cognitive control tasks (at least decision-

making tasks). Considering the very variable definitions of brain areas across studies, refining

the location of those encodings and dissecting the projections of the cortical areas encoding

effort onto the LC could be critical to better understand the circuitry underlying effort-related

LC activity. Additionally, considering the large number of inputs received by each LC unit

(Schwarz et al., 2015), LC units could in fact multiplex signals from various brain areas and

combine them into a subjective difficulty/effort signal.

D. LC activity and value-based decision-making

In three tasks involving cost-benefit tradeoffs, Force- or Delay-discounting tasks, LC

neurons displayed increased activity after cue(s) onset. Consistently across tasks, and

coherently with previous single-unit recording studies of LC activity (Bouret et al., 2012;

Bouret and Richmond, 2015; Jahn et al., 2020; Varazzani et al., 2015), LC neurons were found

to encode Reward(s) in both Force- and Delay-discounting tasks. In addition to that, only in

the Delay-discounting task were LC neurons found to encode the cost of the trial, namely

Delay, after cues onset. In contrast, in Force-discounting tasks (Jahn et al., 2020; Varazzani et

al., 2015) or reward schedule tasks (Bouret et al., 2012) the cost (Force or sequence length,

respectively) was not encoded. This pattern of strong encoding of Reward-related information

associated to faint or absent encoding of effort-related parameters during decisions is

reminiscent of the pattern of activity observed in the vmPFC (Kable and Glimcher, 2007; Peters

and Büchel, 2009; Plassmann et al., 2010; Prévost et al., 2010; San-Galli et al., 2018; Skvortsova

et al., 2014; Talmi et al., 2009), suggesting that the responses of LC neurons to cues onset may

result from an interplay with the vmPFC. It is unclear, however, whether the vmPFC would

feed the LC with reward-related information or whether it is the other way round. San-Galli

and colleagues (2018), report very short latencies to Reward encoding in the vmPFC (less than

200ms, San-Galli et al., 2018, Fig. 4A&D), quite similar to those reported in the same task by

Varazzani and colleagues (2015, Fig. 5C). Considering the slow transmission of LC inputs to the

cortex (above 100ms, Aston-Jones et al., 1985), it would seem more probable that LC neurons

receive inputs from the vmPFC or from the same area that feeds reward information to the

vmPFC, rather than the opposite.

Discussion

253

Engaging in actions leading to delayed Rewards is costly, as it requires engaging

cognitive control (Aranovich et al., 2016; Hirsh et al., 2008; Kurth-Nelson et al., 2012; Sasse et

al., 2017; Shamosh et al., 2008), and in our Delay-discounting task, monkeys did discount

Rewards with Delay, as they exhibited lower engagement rates for high-Delay offers.

Additionally, the marginal encoding of condition-by-condition engagement rate (over and

above Reward and Delay), and the coherent strength of encoding of Reward and Delay across

neurons constitute two arguments towards a representation of the discounted value of the

offered reward by LC neurons around cue onset, in the particular case of delay-discounting.

These results are in contrast to previous LC single-unit recordings in Reward-schedule tasks.

In such tasks, sequence length, which produces a delay to the reward, has been shown to

produce, in a majority of subjects, similar effects on engagement rates as delay to the reward

(Minamimoto et al., 2012), yet it is not encoded by LC neurons (Bouret et al., 2012). Such

contrast with our Delay-discounting task could come from differences in task structure.

Indeed, in our Delay-discounting task, engaging in the action (releasing the bar) predicted

Reward delivery with certainty. Conversely, in Reward-schedule tasks, repeated engagement

for each press of the sequence was required to ensure reward delivery, and each cue could in

theory lead to a decision whether to engage.

Additionally, pharmacological decreases of LC activity using α2 agonist administration

(clonidine) in monkeys performing a Reward-schedule task have failed to provoke any change

in the modulation of engagement probability by offer parameters, or any change in

engagement rates in sequences or individual presses (Jahn et al., 2018). This result argues for

an absence of involvement of LC activity in engagement in planned sequences of actions based

on the discounted value of the reward, and in individual decisions to engage within sequences

of actions. In other words, the activity of LC neurons after cue onset reflects the reward and

its characteristics: size, delay to delivery, and possibly uncertainty (Jepma et al., 2018; Jepma

and Nieuwenhuis, 2011; Nassar et al., 2012; Payzan-LeNestour et al., 2013). However, it is not

likely to directly affect decisions to engage, and in particular could not mediate the effect of

Reward or Delay (and possibly uncertainty) on engagement, at least in case of well-known

cues, i.e. outside of learning conditions.

In Borderies et al. (2020), in a two-alternative Force-discounting task, we reported that

clonidine administration, i.e. decreases in LC activity, selectively affected the probability of

Discussion

254

choosing the high-Effort options in two ways: a global decrease in the overall tendency to

choose the high-effort offers and an increase in the sensitivity of choices to the Effort

dimension. In a similar task (Study 2), I report no encoding of the effort dimension of the

offers, only of the Reward dimensions, after cues onset. Although we cannot exclude that this

result comes from a lack of statistical power, or from the lower sensitivity of the monkey’s

choices to Forces than to Rewards, these results tend to indicate that LC neurons cannot

participate in the effort-based valuation of offers, even in the context of goal-directed action-

selection. Conversely, I proposed an indirect mechanism for the relation between LC activity

and choices of high-Effort courses of action via modulations of Effort sensitivity. Within trials,

I showed that increased LC activity around early steps led to decreased subjective difficulty in

the next step of the trial. Indeed, increased activity around the first press led to increased

probability to engage in the second press, and increased activity around cues onset led to

decreased time taken to trigger the second press. In other words, increased activity early in

trials was associated to decreased subjective difficulty to engage in costly actions. Evidence

for a causal link between LC activity and the subjective difficulty to engage was provided by

Jahn and colleagues (2018), who showed that clonidine administration led to increased RTs,

which probably reflected the subjective difficulty to engage in actions.

Additionally, I showed that increased activity around cues onset was associated to

increased probability of choosing the high-Effort offer, and to an increase in the exerted force

for the second press, especially in the most difficult conditions. This indicates that increased

early LC activity probably led to decreased subjective difficulty of executing the actions.

Decreases in LC activity with clonidine have been shown to cause a decrease in exerted force

(Borderies et al., 2020; Jahn et al., 2018). We excluded that this effect was a mere muscular

effect, i.e. a consequence of a decreased motor capacity to exert force (Borderies et al., 2020).

Consequently, this result provides evidence that the relation between LC activity and the

subjective difficulty to execute effortful actions was probably causal. Additionally, in Borderies

et al. (2020) we report that the decrease in exerted force and increased force sensitivity of

choices could be accounted for by a single effort sensitivity variable, increased in case of

clonidine administration. The second study proposed a mechanism for this impact of LC

activity on value-based decision-making, and in particular effort-based decision-making.

In Study 2, I showed that increased LC activity around early steps of the trials requiring

effort production facilitated effort production later on. Importantly, this effect was domain-

Discussion

255

general, meaning that it LC activity facilitated physical and cognitive effort production later

on. Indeed, increased LC activity around the middle press was associated to increased

probability to engage in choices and faster chosen action-triggering, and increased activity

around cues onset led to decreased decision biases towards choosing easy offers. This

suggests that increased LC activity early in trials facilitated cognitive control exertion later on,

in order to override default courses of actions (refusing trials and choosing low-force options).

Moreover, increased cues-onset related activity led to increased force exertion during chosen

action execution, indicating that LC activity a little less than one second prior to action onset

may have facilitated physical effort production. In other words, our results were compatible

with domain-general delayed effects of LC activity on effort production, i.e. facilitation of

effort production regardless of the type of challenge faced (physical or cognitive). In Borderies

et al. (2020), long-term decreases in LC activity would have led to a decrease in the capacity

to produce effort, and therefore to an increase in the perception of effort. Consequently, this

would have led to an adjustment of choices to avoid high-effort offers more often, especially

if a much less effortful alternative was available. This adjustment of choices would have had

to be within very few trials, as we reported no tendency for a drift in high-Effort choice

probability throughout sessions.

The existence of a monitoring unit has been theorized for control allocation (Shenhav

et al., 2013), and could be extended to physical effort exertion, especially if cognitive and

physical effort draw on similar resources (see part I.A.3). The monitoring unit collects

information on previous resource expenses in order to adjust the parameters of the future

decisions to allocate effort made by the selection unit (see part I.A.2.c). My data as well as

previously obtained results are compatible with a role of the LC as a provider of feedback to

the monitoring unit. Consistently across three effort-discounting tasks, LC neurons displayed

an encoding of the subjective difficulty and probably the exerted effort. I additionally showed

in Study 2 that higher activity around second press execution was associated to decreased

probability of choosing the high-Effort offer in the next trial. If LC neurons do encode the

exerted effort for the press, my results indicate that this signal could participate in the

adjustment of future decisions to allocate effort. In addition to feedback regarding effort

production itself, LC neurons seem to provide feedback on some of its consequences

(outcomes). Jahn and colleagues (2020) reported that LC neurons respond to commitment

Discussion

256

errors (failures to engage in trials), and I complemented this finding by showing that LC

neurons respond to execution errors. Both these events can be used as signals of insufficient

control allocation. Botvinick and colleagues (Botvinick, 2007; Botvinick et al., 2004, 2001)

additionally theorized that response conflict detection could constitute another feedback

signal regarding the adequacy of control allocation with respect to task demands. Increased

LC activity (by tVNS) has been shown to facilitate behavioral adjustments to conflict (slowing

of RT), suggesting that LC activity could indeed provide a conflict detection signal provoking

an adjustment of control allocation (Fischer et al., 2018). NA has been repeatedly shown to

have a causal role in adaptations to changes in task rules (Lapiz et al., 2007; Lapiz and Morilak,

2006), especially via LC projections to the cortex (McGaughy et al., 2008; Tait et al., 2007).

Importantly, higher NA levels (higher atomoxetine doses) are required to facilitate set shifting

than to enhance working memory and inhibitory control (Newman et al., 2008). Error-related

LC activity could participate in set-shifting, by facilitating adjustments to new contingencies.

The targets of LC activity that account for its delayed effects on effort production

cannot be determined by the studies presented in this manuscript, but suggestions, opening

the path for further research, can be formulated. The ACC has strong reciprocal connections

to the LC (Schwarz et al., 2015). It has been shown to multiplex Reward- and Effort-related

information (Amiez et al., 2006; Hayden and Platt, 2010; Kennerley et al., 2011) and to have a

causal role in effort-based decision-making (Rudebeck et al., 2006; Walton et al., 2003, 2002).

Specifically, the ACC has been implicated in learning contingencies and consequences of effort

exertion (Rudebeck et al., 2006; Rushworth et al., 2004; Walton et al., 2009, 2004, 2003), such

as rewards and errors (Fiehler et al., 2004; Jahn et al., 2014; Quilodran et al., 2008; Ullsperger

and Cramon, 2003). Interestingly, error-related responses of the ACC are reminiscent of those

observed in the LC by Jahn and colleagues (2020) and myself in Study 2, since ACC neurons

respond to task disengagement, more strongly for bigger offered rewards (Amiez et al., 2005).

More generally, the ACC is believed to be critical for updating internal models of the

environment (Kolling et al., 2016; O’Reilly et al., 2013), planning sequences of actions (Abe

and Lee, 2011; Barraclough et al., 2004; Tervo et al., 2014; Zhu et al., 2012), and triggering

behavioral change (Ridderinkhof et al., 2004; Rushworth et al., 2004). These elements have

led several teams to propose a role of the ACC as the monitoring unit of control, collecting

feedback on effort production and its consequences to adjust and plan future courses of

Discussion

257

actions, and possibly as the specification unit as well (Botvinick, 2007; Botvinick et al., 2004,

2001; Shenhav et al., 2013).

Moreover, the role of the ACC on high-effort choice promotion has been shown to

depend critically on catecholamine neurotransmission (Schweimer et al., 2005), and more

precisely DA fixation on D2 receptors (Walton et al., 2009), but not on nucleus accumbens

integrity (NAc, Walton et al., 2009, 2005). Several studies have demonstrated that

modifications of LC activity lead to modifications of DA levels in cerebral cortices, including

the ACC (Devoto et al., 2003, 2001; Kawahara et al., 2001), probably due to a co-release of DA

with NA (Devoto et al., 2005). These results indirectly suggest that LC activity could participate

in the putative role of the ACC in monitoring effort exertion, possibly by providing it with

feedback on effort production and errors. Thus, it would be interesting for further studies to

explore this hypothesis. Additionally, certain ACC neurons exhibit plasticity (pyramidal

neurons, Lee et al., 2007; Liu et al., 2018; Shyu and Vogt, 2009; Toyoda et al., 2011), and NA

has plasticity-promoting effects in some brain regions (Bing et al., 1992, 1991; Gubits et al.,

1989; Maity et al., 2020; Sonneborn and Greene, 2021; Stone et al., 1997, 1991; Walling and

Harley, 2004). If NA does promote plasticity in the ACC, it could participate in the adjustments

of models of the environment and of effort allocation strategies, in response to past effort

production.

E. LC and facilitation of effort exertion: mechanistic

hypotheses

Previous studies exploring the link between LC activity and resource mobilization for

cognitive and physical processes had shown a causal role of LC activity in physical effort

production (Borderies et al., 2020; Jahn et al., 2018). In Study 1, we showed that LC neurons

encoded the subjective and objective cost of triggering and executing actions, at the time of

action triggering and execution, respectively. Considering the slow conduction velocity of LC

axons (Aston-Jones et al., 1985), the timing of such encodings was only compatible with effects

on ulterior processes, which could hardly be tested in single-step one-option tasks. In the

second study, using a two-step two-alternative force-discounting task, I showed that, within

trials, LC activity had a negative effect on the subjective difficulty of triggering and executing

actions, suggesting that it may have energized behavior, such that triggering and executing

Discussion

258

the actions was consequently perceived as less difficult. Importantly, the time elapsed

between the LC activity modulating a certain behavior and the behavior itself was rather long,

around one second, suggesting either very delayed or quite long-lasting effects, or both. Such

delayed effect of LC activity on behavior had previously been noted by Bari and colleagues

(2020), who reported that increased baseline activity of LC neurons, i.e. before distractor

onset, in a modified Posner task, was associated to lesser effects of distractors on behavior

and better performance. In other words, increased LC activity facilitated inhibitory control, a

subcategory of cognitive control. In a way, such effects of pre-trial onset activity on

subsequent behavior are reminiscent of the relation between LC activity and subjective

difficulty in an epoch starting before cues onset, i.e. anticipating the cues.

Effort exertion is believed to be aversive due to the consumption of some resources,

although the exact nature of such resources remains debated (Shenhav et al., 2017).

Consequently, increasing the amount or biasing the perception of the resources available

would logically make subjective difficulty decrease. Astrocytic glycogen as well as brain

glucose have been proposed as the limited quantity that the effort allocation system aims at

sparing when allocating cognitive control with caution (Baumeister et al., 1998; Baumeister

and Heatherton, 1996; Christie and Schrater, 2015; Muraven et al., 1998). As for physical

effort, the brain energy demand increases during physical exercise (Secher et al., 2008),

causing an increase in lactate and glucose consumption (Larsen et al., 2008; Vissing et al.,

1996). This increase is met by higher glycogen degradation in multiple brain areas, in particular

but not restricted to brain areas involved in motor control, an effect that could not be solely

accounted for by the muscle-activity induced decrease in blood glucose (Matsui et al., 2012,

2011). Additionally, this depletion in glycogen has been linked to endurance capacity and

fatigue following prolonged physical effort (Matsui et al., 2019, 2017), suggesting a role of

glycogen reserves in physical effort perception i.e. subjective difficulty.

LC activity and NA release are associated to modifications of astrocytic metabolism

(Cohen et al., 1997; Gibbs, 2015; Gibbs et al., 2010; Hertz et al., 2010; Müller, 2014; O’Dowd

et al., 1995; Quach et al., 1978). First, NA induces short term glycogen degradation into lactate,

an immediately available substrate for neuronal metabolism (Magistretti, 1988; Quach et al.,

1978; Sorg and Magistretti, 1991). In the longer term, NA promotes glycogen synthesis via

transcriptional effects (Allaman et al., 2000; Sorg and Magistretti, 1992). Chemical reactions

Discussion

259

(including those happening in living cells) follow an equilibrium (Lodish et al., 2000), meaning

that the final concentrations of the product depend on the initial concentration of the

substrate. Consequently, if glycogen levels decrease, the amount of lactate that can be

released decreases. In the short term, NA appears to bias the reaction transforming glycogen

into lactate, possibly by modifying enzymatic activity within astrocytes (Uhr, 1979) via

increases in cAMP (Fink et al., 2021), and could therefore be making lactate more available for

brain areas involved in the task at hand regardless of glycogen supplies. The increase in lactate

availability following NA release has been shown to be beneficial for some cognitive processes

such as memory formation and consolidation (Gibbs, 2015; Gibbs et al., 2010). Although, to

my knowledge, the direct link between NA release in the brain and cognitive control or

physical effort performance via increases in available metabolic substrate has not been

directly demonstrated, it can he hypothesized that the effects observed for memory could

generalize to more (if not all) brain areas, and therefore other processes. Additionally, the

dynamics of NA-dependent increases in astrocytic and neuronal lactate have been examined,

in the context of arousal-induced NA release. Lactate increases have been shown to have a

relatively delayed onset (a few seconds, Zuend et al., 2020), but additional studies with

increased temporal precision and, importantly, in the context of task execution and effort

production, would be required to further demonstrate the possible relation between

metabolic adjustments and effort facilitation by LC activity.

The aversiveness of control exertion has also been proposed to be caused by the

existence of bottlenecks in processing systems generating a risk of interferences between

cognitive operations (Allport, 1954; Logan, 1985). NA release is known to affect brain

functional connectivity (Coull et al., 1999; Guedj et al., 2017b; Hernaus et al., 2017; Zerbi et

al., 2019). Pharmacological modifications of brain NA levels during task execution have been

shown to coherently modulate performance and the connectivity between task-related areas

(atomoxetine: Brink et al., 2016; Hernaus et al., 2017; clonidine: Coull et al., 1999), suggesting

a causal role for NA-mediated modifications of brain connectivity in mental effort exertion.

Mather and colleagues (2016) have suggested that NA release in the presence of glutamate

could both enhance local activity and lateral inhibition, thus potentially decreasing the risk for

competition between neuronal networks. Glutamate, a major excitatory neurotransmitter, is

critical for cognitive function and especially cognitive control exertion (Dalton et al., 2011; Jett

Discussion

260

et al., 2017; Naaijen et al., 2018; Nardecchia et al., 2018; Stefani et al., 2003; Stefani and

Moghaddam, 2005; Yuen et al., 2012, 2011). Moreover, during cognitive control exertion,

intensity of neuronal activity (measured by fMRI) in critical regions such as the dorsal ACC, the

orbitofrontal cortex and the basal ganglia correlates positively with glutamate levels in these

areas (Falkenberg et al., 2012). In other words, glutamate is released in highly active areas

during control-greedy tasks, and this release is critical for cognitive control exertion. The

supposed synergy between NA and glutamate could therefore constitute a mechanism for the

facilitation of control exertion by LC activation.

F. Relationship between noradrenergic facilitation of

effort and promotion of behavioral variability

I proposed that the LC could be facilitating effort production in the short term while

providing feedback regarding current effort production to adjust future decisions. Multiple

studies have shown a causal role for the LC/NA system in behavioral variability and especially

decision noise (Dubois et al., 2021; Jahn et al., 2020; Kane et al., 2017; Tervo et al., 2014). Even

though this role may seem disconnected from effort facilitation at first glance, it may in fact

be a collateral effect of it. If the LC does provide feedback on effort production, the intrinsic

variability in LC activity across trials may cause trial-by-trial adjustments in effort allocation,

and therefore introduce decision noise. Additionally, since we showed that LC activity

influences trial-by-trial probability to exert control to engage in high-effort offers, trial-by-trial

variability in LC responses could account for some of the decision noise. Decreases in LC

activity (Borderies et al., 2020; Jahn et al., 2018) would lead to decreased feedback strength

about effort production, causing smaller trial-by-trial adjustments of effort allocation across

trials, and decreased variations in the enhancement of high-effort decisions. On average, this

would increase choice consistency. Noise in the neuronal activity has been proposed to cause

the noise in decisions (Drugowitsch et al., 2016; Faisal et al., 2008; Renart and Machens, 2014;

Wang, 2002), and I showed that the intrinsic noise in the LC neuronal activity could account

for (some of) decision variability, by injecting noise into decisions, as suggested by Doya

(2002). The proposal from Aston-Jones and Cohen (2005) that LC activity influences the noise

in the neuronal activity of target areas is not incompatible, but complementary.

Discussion

261

Tervo and colleagues (2014) have reported that stimulating LC inputs to the ACC

increased behavioral variability (noise), suggesting that the effects of LC activity on decision

noise could rely on LC-ACC projections. Interestingly, I proposed that effort-related effects of

LC activity would critically depend on LC-ACC projections as well. Still, it is unclear whether

artificial electrical LC stimulation properly mimics the few spikes evoked by task events,

justifying the need for additional studies of the effects of modifying the strength of LC inputs

to the ACC.

A second aspect of behavioral flexibility is the balance between exploratory and

exploitative behavior. Pupillometry studies have implicated LC activity in switches from

exploitation to exploration, suggesting that brief increases in LC activity (associated to

transient pupil dilations) could lead to task disengagement (Jepma et al., 2011), and that

periods of exploration could be associated to higher LC activity (associated to higher baseline

pupil diameter, Gilzenrat et al., 2010). The causal role for LC activity in task disengagement

was later demonstrated using DREADDs (Kane et al., 2017). The results described in this

manuscript, associated to previous studies, indicated that LC neurons probably signal the

subjective difficulty of cognitive and physical processes, and possibly the invested effort, and

that this signal serves to adjust future effort allocation. Task disengagement could constitute

an extreme adjustment in case of detection of very high subjective difficulty, i.e. following

strong LC activation.

G. Modularity of the LC: evidence for differences in

electrophysiological properties and consequences for

effort processing

Recent anatomo-functional studies of the LC have emphasized the heterogeneity of LC

organization, both regarding cell morphology (Loughlin et al., 1986; Swanson, 1976),

projection patterns (Schwarz et al., 2015), neurotransmitter synthesis (Sutin and Jacobowitz,

1991), electrophysiological properties (Totah et al., 2018; Williams et al., 1984) and effects on

cognitive processes (Uematsu et al., 2017). In the first study, neuron-by-neuron baseline

remained stable across the duration of the experiments (Bouret and Richmond, 2015, 2009;

Bouret and Sara, 2004; Clayton et al., 2004; Kalwani et al., 2014). However, I found that LC

Discussion

262

neurons exhibited variable baseline activity, correlating positively with the amplitude of

evoked responses to task events and their signal-to-noise ratio, and with the strength of

encoding of task and behavioral parameters, suggesting overall differences in excitability

across neurons. These findings reinforce the idea of the diversity in electrophysiological

properties across LC neurons, although the spatial precision of the recording method used

here did not allow identifying whether neurons with different properties were spatially

dissociable. Additionally, we had no way of assessing the projection pattern of the recorded

neurons, although rodent studies have suggested that excitability differences may be

associated to projection pattern heterogeneities (Chandler et al., 2014).

In Study 2, I reported that increased LC activity was associated to later decreases in

subjective difficulty perception both for cognitive and physical effort, in coherence with

pharmacological studies (Borderies et al., 2020; Jahn et al., 2018). Additionally, I provided

evidence for a potential role of LC activity in adjusting future control allocation. However, the

method of analysis considered LC neurons as a population, and it cannot be excluded that

different effects of LC activity resulted from different modules, with different projection

patterns. The detailed inspection of effects of pharmacological manipulations of LC activity by

clonidine administration (Borderies et al., 2020; Jahn et al., 2018) provides some insight into

this possibility. Indeed, α2 receptors are not evenly distributed throughout the LC, and are

more numerous in the posterior part (Chamba et al., 1991), a portion of the LC that

preferentially projects onto the thalamus (Schwarz et al., 2015), an area critically involved in

motor control (Sommer, 2003). Consequently, clonidine at lower doses would have

preferentially decreased posterior LC activity and therefore NA release into the thalamus.

Jahn and colleagues (2018) used several doses of clonidine (2.5 or 5 or 7.5μg/kg), all

lower than the dose we used by Borderies et al. (2020, 10 μg/kg). Motor effects (decreased

exerted force) were observed in both studies, and appeared even for the lowest dose of

clonidine (Figure 28B&C). These effects may in fact be dependent on projections of the

posterior part of the LC onto the thalamus. As detailed earlier, they may account for the

increase in sensitivity of choices to the effort dimension, although in Borderies et al. (2020)

we only used a high dose of clonidine, which probably had effects on the activity of other parts

of the LC. Conversely, Jahn and colleagues only reported effects on RT, and therefore possibly

cognitive effort, for the highest dose of clonidine used, suggesting that it may rely on separate

Discussion

263

LC neurons, in a more anterior part of the LC with lower α2 receptor concentration and

probably projecting onto separate brain regions (possibly frontal cortices).

H. Conclusions and perspectives

Altogether, my work provides some mechanistic insight into the involvement of LC

activity in motivational processes, and more specifically effort perception, production and

regulation of allocation. Throughout three effort-discounting tasks, LC neurons displayed

evoked responses with timings compatible with signaling the end of decision processes, and

encoded the subjective difficulty of these decisions and the cognitive control exerted. During

actions, LC neurons encoded their subjective difficulty and the physical effort involved. LC

activation was associated to modifications of subjective difficulty, and these modifications

appeared to be delayed (around 1s after the activation of neurons) and/or quite long-lasting.

Within trials, LC activity biased choices towards high-effort offers and facilitated action

execution, and I suggested that these effects may be due to metabolic and/or connectivity

effects of NA on targets. Across trials, LC activation reflecting the exerted effort and signaling

errors probably served as a learning signal to adjust future effort allocation. This effect would

be compatible with neuronal interplay between the LC and the ACC, the putative structure

monitoring effort production, with feedback on effort-related processes. Such meta-learning

role of the interplay between the LC and the ACC, in complement with an interplay between

the ACC and DA nuclei, had already been proposed in the context of (but not restricted to)

effort-discounting tasks (Silvetti et al., 2018). Consequently, my work clarifies the dynamics of

the implication of the LC in decision-making and supports a key role in effort perception,

production, and allocation adjustments.

Considering the anatomic and functional heterogeneity of the LC, a critical next step in

understanding the implication of LC activity in effort processing would consist in dissecting the

projection patterns of the LC in order to identify their relative contribution. DREADDs

associated with retrograde viruses now allow selectively inhibiting certain subpopulations of

neurons based on their projection patterns. Selectively inhibiting LC-ACC projections could

provide confirmation of the scenario proposed above regarding the role of the LC as an effort

feedback provider for the ACC, while modifications LC-thalamus projections may confirm the

contribution of these projections in physical effort facilitation.

References

264

References

Aarts, E., Roelofs, A., Franke, B., Rijpkema, M., Fernández, G., Helmich, R.C., Cools, R., 2010. Striatal Dopamine Mediates the Interface between Motivational and Cognitive Control in Humans: Evidence from Genetic Imaging. Neuropsychopharmacology 35, 1943–1951. https://doi.org/10.1038/npp.2010.68

Aasman, J., Mulder, G., Mulder, L.J.M., 1987. Operator Effort and the Measurement of Heart-Rate Variability. Hum. Factors J. Hum. Factors Ergon. Soc. 29, 161–170. https://doi.org/10.1177/001872088702900204

Abe, H., Lee, D., 2011. Distributed Coding of Actual and Hypothetical Outcomes in the Orbital and Dorsolateral Prefrontal Cortex. Neuron 70, 731–741. https://doi.org/10.1016/j.neuron.2011.03.026

Adams, L.M., Foote, S.L., 1988. Effects of locally infused pharmacological agents on spontaneous and sensory-evoked activity of locus coeruleus neurons. Brain Res. Bull. 21, 395–400. https://doi.org/10.1016/0361-9230(88)90151-7

Agster, K.L., Mejias-Aponte, C.A., Clark, B.D., Waterhouse, B.D., 2013. Evidence for a regional specificity in the density and distribution of noradrenergic varicosities in rat cortex. J. Comp. Neurol. 521, 2195–2207. https://doi.org/10.1002/cne.23270

Allaman, I., Pellerin, L., Magistretti, P.J., 2000. Protein targeting to glycogen mRNA expression is stimulated by noradrenaline in mouse cortical astrocytes. Glia 30, 382–391.

Allport, G.W., 1954. The nature of prejudice, The nature of prejudice. Addison-Wesley, Oxford, England.

Alnæs, D., Sneve, M.H., Espeseth, T., Endestad, T., van de Pavert, S.H.P., Laeng, B., 2014. Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. J. Vis. 14. https://doi.org/10.1167/14.4.1

Alvarez, V.A., Chow, C.C., Bockstaele, E.J.V., Williams, J.T., 2002. Frequency-dependent synchrony in locus ceruleus: Role of electrotonic

coupling. Proc. Natl. Acad. Sci. 99, 4032–4036. https://doi.org/10.1073/pnas.062716299

Alvarez-Maubecin, V., García-Hernández, F., Williams, J.T., Bockstaele, E.J.V., 2000. Functional Coupling between Neurons and Glia. J. Neurosci. 20, 4091–4098. https://doi.org/10.1523/JNEUROSCI.20-11-04091.2000

American Psychiatric Association, American Psychiatric Association (Eds.), 2013. Diagnostic and statistical manual of mental disorders: DSM-5, 5th ed. ed. American Psychiatric Association, Washington, D.C.

Amiez, C., Joseph, J.P., Procyk, E., 2006. Reward Encoding in the Monkey Anterior Cingulate Cortex. Cereb. Cortex 16, 1040–1055. https://doi.org/10.1093/cercor/bhj046

Amiez, C., Joseph, J.-P., Procyk, E., 2005. Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci. 21, 3447–3452. https://doi.org/10.1111/j.1460-9568.2005.04170.x

Anderson, J.R., 1990. The adaptive character of thought, The adaptive character of thought. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US.

Anderson, J.R., 1983. The architecture of cognition, Cognitive science series. Harvard University Press, Cambridge, Mass.

Andrade, L.S., Kanitz, A.C., Häfele, M.S., Schaun, G.Z., Pinto, S.S., Alberton, C.L., 2020. Relationship between Oxygen Uptake, Heart Rate, and Perceived Effort in an Aquatic Incremental Test in Older Women. Int. J. Environ. Res. Public. Health 17. https://doi.org/10.3390/ijerph17228324

Aranovich, G.J., McClure, S.M., Fryer, S., Mathalon, D.H., 2016. The effect of cognitive challenge on delay discounting. NeuroImage 124, 733–739. https://doi.org/10.1016/j.neuroimage.2015.09.027

Ariely, D., Gneezy, U., Loewenstein, G., Mazar, N., 2009. Large Stakes and Big Mistakes. Rev. Econ. Stud. 76, 451–469.

References

265

Arnsten, A.F., 1997. Catecholamine regulation of the prefrontal cortex. J. Psychopharmacol. Oxf. Engl. 11, 151–162. https://doi.org/10.1177/026988119701100208

Arnsten, A.F., Cai, J.X., Goldman-Rakic, P.S., 1988. The alpha-2 adrenergic agonist guanfacine improves memory in aged monkeys without sedative or hypotensive side effects: evidence for alpha-2 receptor subtypes. J. Neurosci. Off. J. Soc. Neurosci. 8, 4287–4298.

Arnsten, A.F., Goldman-Rakic, P.S., 1985. Alpha 2-adrenergic mechanisms in prefrontal cortex associated with cognitive decline in aged nonhuman primates. Science 230, 1273–1276. https://doi.org/10.1126/science.2999977

Arnsten, A.F.T., 2006. Stimulants: Therapeutic actions in ADHD. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 31, 2376–2383. https://doi.org/10.1038/sj.npp.1301164

Arnsten, A.F.T., 2000. Through the Looking Glass: Differential Noradenergic Modulation of Prefrontal Cortical Function. Neural Plast. 7, 133–146. https://doi.org/10.1155/NP.2000.133

Arnsten, A.F.T., Pliszka, S.R., 2011. Catecholamine influences on prefrontal cortical function: relevance to treatment of attention deficit/hyperactivity disorder and related disorders. Pharmacol. Biochem. Behav. 99, 211–216. https://doi.org/10.1016/j.pbb.2011.01.020

Arnsten, A.F.T., Rubia, K., 2012. Neurobiological Circuits Regulating Attention, Cognitive Control, Motivation, and Emotion: Disruptions in Neurodevelopmental Psychiatric Disorders. J. Am. Acad. Child Adolesc. Psychiatry 51, 356–367. https://doi.org/10.1016/j.jaac.2012.01.008

Arthur, D.L., Starr, A., 1984. Task-relevant late positive component of the auditory event-related potential in monkeys resembles P300 in humans. Science 223, 186–188. https://doi.org/10.1126/science.6691145

Ascending projections of the locus coeruleus in the rat. II. Autoradiographic study, 1977. . Brain Res. 127, 23–53. https://doi.org/10.1016/0006-8993(77)90378-X

Aston-Jones, G., 2005. Brain structures and receptors involved in alertness. Sleep Med. 6, S3–

S7. https://doi.org/10.1016/S1389-9457(05)80002-4

Aston-Jones, G., Bloom, F.E., 1981. Activity of norepinephrine-containing locus coeruleus neurons in behaving rats anticipates fluctuations in the sleep-waking cycle. J. Neurosci. Off. J. Soc. Neurosci. 1, 876–886.

Aston-Jones, G., Cohen, J.D., 2005. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. https://doi.org/10.1146/annurev.neuro.28.061604.135709

Aston-Jones, G., Ennis, M., Pieribone, V.A., Nickell, W.T., Shipley, M.T., 1986. The brain nucleus locus coeruleus: restricted afferent control of a broad efferent network. Science 234, 734–737. https://doi.org/10.1126/science.3775363

Aston-Jones, G., Foote, S.L., Segal, M., 1985. Impulse conduction properties of noradrenergic locus coeruleus axons projecting to monkey cerebrocortex. Neuroscience 15, 765–777. https://doi.org/10.1016/0306-4522(85)90077-6

Aston-Jones, G., Rajkowski, J., Cohen, J., 1999. Role of locus coeruleus in attention and behavioral flexibility. Biol. Psychiatry 46, 1309–1320. https://doi.org/10.1016/S0006-3223(99)00140-7

Aston-Jones, G., Rajkowski, J., Kubiak, P., Alexinsky, T., 1994. Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. J. Neurosci. 14, 4467–4480. https://doi.org/10.1523/JNEUROSCI.14-07-04467.1994

Aston-Jones, G., Segal, M., Bloom, F.E., 1980. Brain aminergic axons exhibit marked variability in conduction velocity. Brain Res. 195, 215–222. https://doi.org/10.1016/0006-8993(80)90880-x

Aston-Jones, G., Shipley, M.T., Chouvet, G., Ennis, M., van Bockstaele, E., Pieribone, V., Shiekhattar, R., Akaoka, H., Drolet, G., Astier, B., Charléty, P., Valentino, R.J., Williams, J.T., 1991. Afferent regulation of locus coeruleus neurons: anatomy, physiology and pharmacology, in: Progress in Brain Research. Elsevier, pp. 47–75. https://doi.org/10.1016/S0079-6123(08)63799-1

References

266

Awasthi, V., Pratt, J., 1990. The Effects of Monetary Incentives on Effort and Decision Performance: The Role of Cognitive Characteristics. Account. Rev. 65, 797–811.

Backs, R.W., Ryan, A.M., 1992. Multimodal Measures of Mental Workload during Dual-Task Performance: Energetic Demands of Cognitive Processes. Proc. Hum. Factors Soc. Annu. Meet. 36, 1413–1417. https://doi.org/10.1177/154193129203601811

Backs, R.W., Seljos, K.A., 1994. Metabolic and cardiorespiratory measures of mental effort: the effects of level of difficulty in a working memory task. Int. J. Psychophysiol. 16, 57–68. https://doi.org/10.1016/0167-8760(94)90042-6

Balleine, B.W., Dickinson, A., 1998. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407–419. https://doi.org/10.1016/S0028-3908(98)00033-1

Bari, A., Eagle, D.M., Mar, A.C., Robinson, E.S.J., Robbins, T.W., 2009. Dissociable effects of noradrenaline, dopamine, and serotonin uptake blockade on stop task performance in rats. Psychopharmacology (Berl.) 205, 273–283. https://doi.org/10.1007/s00213-009-1537-0

Bari, A., Xu, S., Pignatelli, M., Takeuchi, D., Feng, J., Li, Y., Tonegawa, S., 2020. Differential attentional control mechanisms by two distinct noradrenergic coeruleo-frontal cortical pathways. Proc. Natl. Acad. Sci. U. S. A. 117, 29080–29089. https://doi.org/10.1073/pnas.2015635117

Barraclough, D.J., Conroy, M.L., Lee, D., 2004. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410. https://doi.org/10.1038/nn1209

Baumeister, R.F., 1984. Choking under pressure: self-consciousness and paradoxical effects of incentives on skillful performance. J. Pers. Soc. Psychol. 46, 610–620. https://doi.org/10.1037//0022-3514.46.3.610

Baumeister, R.F., Bratslavsky, E., Muraven, M., Tice, D.M., 1998. Ego depletion: is the active self a limited resource? J. Pers. Soc. Psychol. 74, 1252–1265. https://doi.org/10.1037//0022-3514.74.5.1252

Baumeister, R.F., Heatherton, T.F., 1996. Self-Regulation Failure: An Overview. Psychol. Inq. 7, 1–15. https://doi.org/10.1207/s15327965pli0701_1

Beach, L.R., Mitchell, T.R., 1978. A Contingency Model for the Selection of Decision Strategies. Acad. Manage. Rev. 3, 439–449. https://doi.org/10.2307/257535

Beatty, J., 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol. Bull. 91, 276–292. https://doi.org/10.1037/0033-2909.91.2.276

Beck, S.M., Locke, H.S., Savine, A.C., Jimura, K., Braver, T.S., 2010. Primary and secondary rewards differentially modulate neural activity dynamics during working memory. PloS One 5, e9251. https://doi.org/10.1371/journal.pone.0009251

Beilock, S., 2010. Choke: what the secrets of the brain reveal about getting it right when you have to, 1st Free Press hardcover ed. ed. Free Press, New York.

Benton, D., Brett, V., Brain, P.F., 1987. Glucose improves attention and reaction to frustration in children. Biol. Psychol. 24, 95–100. https://doi.org/10.1016/0301-0511(87)90016-0

Benton, D., Owens, D.S., Parker, P.Y., 1994. Blood glucose influences memory and attention in young adults. Neuropsychologia 32, 595–607. https://doi.org/10.1016/0028-3932(94)90147-3

Benton, D., Parker, P.Y., Donohoe, R.T., 1996. The supply of glucose to the brain and cognitive functioning. J. Biosoc. Sci. 28, 463–479. https://doi.org/10.1017/S0021932000022537

Berridge, C.W., España, R.A., 2000. Synergistic sedative effects of noradrenergic alpha(1)- and beta-receptor blockade on forebrain electroencephalographic and behavioral indices. Neuroscience 99, 495–505. https://doi.org/10.1016/s0306-4522(00)00215-3

Berridge, C.W., Foote, S.L., 1991. Effects of locus coeruleus activation on electroencephalographic activity in neocortex and hippocampus. J. Neurosci. Off. J. Soc. Neurosci. 11, 3135–3145.

Berridge, C.W., Waterhouse, B.D., 2003. The locus coeruleus-noradrenergic system: modulation of behavioral state and state-

References

267

dependent cognitive processes. Brain Res. Brain Res. Rev. 42, 33–84. https://doi.org/10.1016/s0165-0173(03)00143-7

Bettman, J.R., Johnson, E.J., Payne, J.W., 1990. A componential analysis of cognitive effort in choice. Organ. Behav. Hum. Decis. Process. 45, 111–139. https://doi.org/10.1016/0749-5978(90)90007-V

Bettman, J.R., Zins, M.A., 1979. Information Format and Choice Task Effects in Decision Making. J. Consum. Res. 6, 141–53.

Bing, G., Filer, D., Miller, J.C., Stone, E.A., 1991. Noradrenergic activation of immediate early genes in rat cerebral cortex. Mol. Brain Res. 11, 43–46. https://doi.org/10.1016/0169-328X(91)90019-T

Bing, G., Stone, E.A., Zhang, Y., Filer, D., 1992. Immunohistochemical studies of noradrenergic-induced expression of c-fos in the rat CNS. Brain Res. 592, 57–62. https://doi.org/10.1016/0006-8993(92)91658-2

Bingham, B., McFadden, K., Zhang, X., Bhatnagar, S., Beck, S., Valentino, R., 2011. Early adolescence as a critical window during which social stress distinctly alters behavior and brain norepinephrine activity. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 36, 896–909. https://doi.org/10.1038/npp.2010.229

Birnbaum, S.G., Yuan, P.X., Wang, M., Vijayraghavan, S., Bloom, A.K., Davis, D.J., Gobeske, K.T., Sweatt, J.D., Manji, H.K., Arnsten, A.F.T., 2004. Protein kinase C overactivity impairs prefrontal cortical regulation of working memory. Science 306, 882–884. https://doi.org/10.1126/science.1100021

Boehler, C.N., Schevernels, H., Hopf, J.-M., Stoppel, C.M., Krebs, R.M., 2014. Reward prospect rapidly speeds up response inhibition via reactive control. Cogn. Affect. Behav. Neurosci. 14, 593–609. https://doi.org/10.3758/s13415-014-0251-5

Bogacz, R., Brown, E., Moehlis, J., Holmes, P., Cohen, J.D., 2006. The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks. Psychol. Rev. 113, 700–765. https://doi.org/10.1037/0033-295X.113.4.700

Boksem, M.A.S., Meijman, T.F., Lorist, M.M., 2006. Mental fatigue, motivation and action monitoring. Biol. Psychol. 72, 123–132. https://doi.org/10.1016/j.biopsycho.2005.08.007

Boksem, M.A.S., Tops, M., 2008. Mental fatigue: Costs and benefits. Brain Res. Rev. 59, 125–139. https://doi.org/10.1016/j.brainresrev.2008.07.001

Bonnelle, V., Manohar, S., Behrens, T., Husain, M., 2016. Individual Differences in Premotor Brain Systems Underlie Behavioral Apathy. Cereb. Cortex 26, 807–819. https://doi.org/10.1093/cercor/bhv247

Bonner, S., Hastie, R., Sprinkle, G., Young, S., 2000. A Review of the Effects of Financial Incentives on Performance in Laboratory Tasks: Implications for Management Accounting. J. Manag. Account. Res. - J Manag Acc. Res 12, 19–64. https://doi.org/10.2308/jmar.2000.12.1.19

Borchert, R.J., Rittman, T., Passamonti, L., Ye, Z., Sami, S., Jones, S.P., Nombela, C., Vázquez Rodríguez, P., Vatansever, D., Rae, C.L., Hughes, L.E., Robbins, T.W., Rowe, J.B., 2016. Atomoxetine Enhances Connectivity of Prefrontal Networks in Parkinson’s Disease. Neuropsychopharmacology 41, 2171–2177. https://doi.org/10.1038/npp.2016.18

Borderies, N., Bornert, P., Gilardeau, S., Bouret, S., 2020. Pharmacological evidence for the implication of noradrenaline in effort. PLOS Biol. 18, e3000793. https://doi.org/10.1371/journal.pbio.3000793

Borg, G.A., 1982. Psychophysical bases of perceived exertion. Med. Sci. Sports Exerc. 14, 377–381.

Botvinick, M.M., 2007. Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function. Cogn. Affect. Behav. Neurosci. 7, 356–366. https://doi.org/10.3758/CABN.7.4.356

Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S., Cohen, J.D., 2001. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652. https://doi.org/10.1037/0033-295X.108.3.624

Botvinick, M.M., Cohen, J.D., 2014. The computational and neural basis of cognitive control: charted territory and new frontiers.

References

268

Cogn. Sci. 38, 1249–1285. https://doi.org/10.1111/cogs.12126

Botvinick, M.M., Cohen, J.D., Carter, C.S., 2004. Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn. Sci. 8, 539–546. https://doi.org/10.1016/j.tics.2004.10.003

Boureau, Y.-L., Sokol-Hessner, P., Daw, N.D., 2015. Deciding How To Decide: Self-Control and Meta-Decision Making. Trends Cogn. Sci. 19, 700–710. https://doi.org/10.1016/j.tics.2015.08.013

Bouret, S., Duvel, A., Onat, S., Sara, S.J., 2003. Phasic activation of locus ceruleus neurons by the central nucleus of the amygdala. J. Neurosci. Off. J. Soc. Neurosci. 23, 3491–3497.

Bouret, S., Ravel, S., Richmond, B.J., 2012. Complementary neural correlates of motivation in dopaminergic and noradrenergic neurons of monkeys. Front. Behav. Neurosci. 6. https://doi.org/10.3389/fnbeh.2012.00040

Bouret, S., Richmond, B.J., 2015. Sensitivity of Locus Ceruleus Neurons to Reward Value for Goal-Directed Actions. J. Neurosci. 35, 4005–4014. https://doi.org/10.1523/JNEUROSCI.4553-14.2015

Bouret, S., Richmond, B.J., 2009. Relation of Locus Coeruleus Neurons in Monkeys to Pavlovian and Operant Behaviors. J. Neurophysiol. 101, 898–911. https://doi.org/10.1152/jn.91048.2008

Bouret, S., Sara, S.J., 2005. Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582. https://doi.org/10.1016/j.tins.2005.09.002

Bouret, S., Sara, S.J., 2004. Reward expectation, orientation of attention and locus coeruleus-medial frontal cortex interplay during learning. Eur. J. Neurosci. 20, 791–802. https://doi.org/10.1111/j.1460-9568.2004.03526.x

Bouret, S., Sara, S.J., 2002. Locus coeruleus activation modulates firing rate and temporal organization of odour-induced single-cell responses in rat piriform cortex. Eur. J. Neurosci. 16, 2371–2382. https://doi.org/10.1046/j.1460-9568.2002.02413.x

Bowman, E.M., Aigner, T.G., Richmond, B.J., 1996. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J. Neurophysiol. 75, 1061–1073. https://doi.org/10.1152/jn.1996.75.3.1061

Braith R W, Wood C E, Limacher M C, Pollock M L, Lowenthal D T, Phillips M I, Staples E D, 1992. Abnormal neuroendocrine responses during exercise in heart transplant recipients. Circulation 86, 1453–1463. https://doi.org/10.1161/01.CIR.86.5.1453

Brink, R.L. van den, Pfeffer, T., Warren, C.M., Murphy, P.R., Tona, K.-D., Wee, N.J.A. van der, Giltay, E., Noorden, M.S. van, Rombouts, S.A.R.B., Donner, T.H., Nieuwenhuis, S., 2016. Catecholaminergic Neuromodulation Shapes Intrinsic MRI Functional Connectivity in the Human Brain. J. Neurosci. 36, 7865–7876. https://doi.org/10.1523/JNEUROSCI.0744-16.2016

Buffalari, D.M., Grace, A.A., 2007. Noradrenergic modulation of basolateral amygdala neuronal activity: opposing influences of alpha-2 and beta receptor activation. J. Neurosci. Off. J. Soc. Neurosci. 27, 12358–12366. https://doi.org/10.1523/JNEUROSCI.2007-07.2007

Burke, C.J., Brünger, C., Kahnt, T., Park, S.Q., Tobler, P.N., 2013. Neural Integration of Risk and Effort Costs by the Frontal Pole: Only upon Request. J. Neurosci. 33, 1706–1713. https://doi.org/10.1523/JNEUROSCI.3662-12.2013

Bush, G., Vogt, B.A., Holmes, J., Dale, A.M., Greve, D., Jenike, M.A., Rosen, B.R., 2002. Dorsal anterior cingulate cortex: A role in reward-based decision making. Proc. Natl. Acad. Sci. 99, 523–528. https://doi.org/10.1073/pnas.012470999

Bymaster, F.P., Katner, J.S., Nelson, D.L., Hemrick-Luecke, S.K., Threlkeld, P.G., Heiligenstein, J.H., Morin, S.M., Gehlert, D.R., Perry, K.W., 2002. Atomoxetine increases extracellular levels of norepinephrine and dopamine in prefrontal cortex of rat: a potential mechanism for efficacy in attention deficit/hyperactivity disorder. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 27, 699–711. https://doi.org/10.1016/S0893-133X(02)00346-9

References

269

Cadegiani, F.A., Kater, C.E., 2019. Novel insights of overtraining syndrome discovered from the EROS study. BMJ Open Sport — Exerc. Med. 5. https://doi.org/10.1136/bmjsem-2019-000542

Cahill, L., McGaugh, J.L., 1996. Modulation of memory storage. Curr. Opin. Neurobiol. 6, 237–242. https://doi.org/10.1016/s0959-4388(96)80078-x

Camerer, C.F., Hogarth, R.M., 1999. The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework. J. Risk Uncertain. 19, 7–42. https://doi.org/10.1023/A:1007850605129

Carey, M.R., Regehr, W.G., 2009. Noradrenergic Control of Associative Synaptic Plasticity by Selective Modulation of Instructive Signals. Neuron 62, 112–122. https://doi.org/10.1016/j.neuron.2009.02.022

Carpenter, P.A., Just, M.A., 1975. Sentence comprehension: A psycholinguistic processing model of verification. Psychol. Rev. 82, 45–73. https://doi.org/10.1037/h0076248

Carroll, D., Turner, J.R., Hellawell, J.C., 1986a. Heart Rate and Oxygen Consumption during Active Psychological Challenge: The Effects of Level of Difficulty. Psychophysiology 23, 174–181. https://doi.org/10.1111/j.1469-8986.1986.tb00613.x

Carroll, D., Turner, J.R., Prasad, R., 1986b. The effects of level of difficulty of mental arithmetic challenge on heart rate and oxygen consumption. Int. J. Psychophysiol. 4, 167–173. https://doi.org/10.1016/0167-8760(86)90012-7

Carter, M.E., Yizhar, O., Chikahisa, S., Nguyen, H., Adamantidis, A., Nishino, S., Deisseroth, K., de Lecea, L., 2010. Tuning arousal with optogenetic modulation of locus coeruleus neurons. Nat. Neurosci. 13, 1526–1533. https://doi.org/10.1038/nn.2682

Cassens, G., Kuruc, A., Roffman, M., Orsulak, P.J., Schildkraut, J.J., 1981. Alterations in brain norepinephrine metabolism and behavior induced by environmental stimuli previously paired with inescapable shock. Behav. Brain Res. 2, 387–407. https://doi.org/10.1016/0166-4328(81)90020-6

Chamba, G., Weissmann, D., Rousset, C., Renaud, B., Pujol, J.F., 1991. Distribution of alpha-1 and

alpha-2 binding sites in the rat locus coeruleus. Brain Res. Bull. 26, 185–193. https://doi.org/10.1016/0361-9230(91)90225-9

Chamberlain, S.R., Del Campo, N., Dowson, J., Müller, U., Clark, L., Robbins, T.W., Sahakian, B.J., 2007. Atomoxetine improved response inhibition in adults with attention deficit/hyperactivity disorder. Biol. Psychiatry 62, 977–984. https://doi.org/10.1016/j.biopsych.2007.03.003

Chamberlain, S.R., Hampshire, A., Müller, U., Rubia, K., Del Campo, N., Craig, K., Regenthal, R., Suckling, J., Roiser, J.P., Grant, J.E., Bullmore, E.T., Robbins, T.W., Sahakian, B.J., 2009. Atomoxetine modulates right inferior frontal activation during inhibitory control: a pharmacological functional magnetic resonance imaging study. Biol. Psychiatry 65, 550–555. https://doi.org/10.1016/j.biopsych.2008.10.014

Chamberlain, S.R., Müller, U., Blackwell, A.D., Clark, L., Robbins, T.W., Sahakian, B.J., 2006a. Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311, 861–863. https://doi.org/10.1126/science.1121218

Chamberlain, S.R., Müller, U., Blackwell, A.D., Robbins, T.W., Sahakian, B.J., 2006b. Noradrenergic modulation of working memory and emotional memory in humans. Psychopharmacology (Berl.) 188, 397–407. https://doi.org/10.1007/s00213-006-0391-6

Chandler, D.J., Gao, W.-J., Waterhouse, B.D., 2014. Heterogeneous organization of the locus coeruleus projections to prefrontal and motor cortices. Proc. Natl. Acad. Sci. 111, 6816–6821. https://doi.org/10.1073/pnas.1320827111

Chaulk, P.C., Harley, C.W., 1998. Intracerebroventricular norepinephrine potentiation of the perforant path-evoked potential in dentate gyrus of anesthetized and awake rats: A role for both alpha- and beta-adrenoceptor activation. Brain Res. 787, 59–70. https://doi.org/10.1016/s0006-8993(97)01460-1

Chipman, P.H., Toma, J.S., Rafuse, V.F., 2012. Chapter 16 - Generation of motor neurons from pluripotent stem cells, in: Dunnett, S.B., Björklund, A. (Eds.), Progress in Brain Research, Functional Neural Transplantation III. Elsevier, pp. 313–331. https://doi.org/10.1016/B978-0-444-59544-7.00015-9

References

270

Chittka, L., Dyer, A.G., Bock, F., Dornhaus, A., 2003. Bees trade off foraging speed for accuracy. Nature 424, 388–388. https://doi.org/10.1038/424388a

Chittka, L., Skorupski, P., Raine, N.E., 2009. Speed–accuracy tradeoffs in animal decision making. Trends Ecol. Evol. 24, 400–407. https://doi.org/10.1016/j.tree.2009.02.010

Chong, T.T.-J., 2015. Disrupting the Perception of Effort with Continuous Theta Burst Stimulation. J. Neurosci. 35, 13269–13271. https://doi.org/10.1523/JNEUROSCI.2624-15.2015

Chong, T.T.-J., Bonnelle, V., Manohar, S., Veromann, K.-R., Muhammed, K., Tofaris, G.K., Hu, M., Husain, M., 2015. Dopamine enhances willingness to exert effort for reward in Parkinson’s disease. Cortex 69, 40–46. https://doi.org/10.1016/j.cortex.2015.04.003

Christi, M.J., Jelinek, H.F., 1993. Dye-coupling among neurons of the rat locus coeruleus during postnatal development. Neuroscience 56, 129–137. https://doi.org/10.1016/0306-4522(93)90568-Z

Christie, M.J., 1997. Generators of synchronous activity of the locus coeruleus during development. Semin. Cell Dev. Biol. 8, 29–34. https://doi.org/10.1006/scdb.1996.0118

Christie, M.J., Williams, J.T., North, R.A., 1989. Electrical coupling synchronizes subthreshold activity in locus coeruleus neurons in vitro from neonatal rats. J. Neurosci. 9, 3584–3589. https://doi.org/10.1523/JNEUROSCI.09-10-03584.1989

Christie, S.T., Schrater, P., 2015. Cognitive cost as dynamic allocation of energetic resources. Front. Neurosci. 9, 289. https://doi.org/10.3389/fnins.2015.00289

Ciombor, K.J., Ennis, M., Shipley, M.T., 1999. Norepinephrine increases rat mitral cell excitatory responses to weak olfactory nerve input via alpha-1 receptors in vitro. Neuroscience 90, 595–606. https://doi.org/10.1016/S0306-4522(98)00437-0

Citri, A., Malenka, R.C., 2008. Synaptic Plasticity: Multiple Forms, Functions, and Mechanisms. Neuropsychopharmacology 33, 18–41. https://doi.org/10.1038/sj.npp.1301559

Clark, C.R., Geffen, G.M., Geffen, L.B., 1987. Catecholamines and attention. I: Animal and clinical studies. Neurosci. Biobehav. Rev. 11, 341–352. https://doi.org/10.1016/s0149-7634(87)80006-4

Clark, L., Cools, R., Robbins, T.W., 2004. The neuropsychology of ventral prefrontal cortex: decision-making and reversal learning. Brain Cogn. 55, 41–53. https://doi.org/10.1016/S0278-2626(03)00284-7

Clayson, P.E., Larson, M.J., 2013. Psychometric properties of conflict monitoring and conflict adaptation indices: Response time and conflict N2 event-related potentials. Psychophysiology 50, 1209–1219. https://doi.org/10.1111/psyp.12138

Clayton, E.C., Rajkowski, J., Cohen, J.D., Aston-Jones, G., 2004. Phasic Activation of Monkey Locus Ceruleus Neurons by Simple Decisions in a Forced-Choice Task. J. Neurosci. 24, 9914–9920. https://doi.org/10.1523/JNEUROSCI.2446-04.2004

Clemow, D.B., Bushe, C.J., 2015. Atomoxetine in patients with ADHD: A clinical and pharmacological review of the onset, trajectory, duration of response and implications for patients. J. Psychopharmacol. Oxf. Engl. 29, 1221–1230. https://doi.org/10.1177/0269881115602489

Cléry-Melin, M.-L., Schmidt, L., Lafargue, G., Baup, N., Fossati, P., Pessiglione, M., 2011. Why Don’t You Try Harder? An Investigation of Effort Production in Major Depression. PLOS ONE 6, e23178. https://doi.org/10.1371/journal.pone.0023178

Cleveland, C.J., Ayres, R.U. (Eds.), 2004. Encyclopedia of energy. Elsevier Academic Press, Amsterdam ; Boston.

Coggan, J.S., Keller, D., Calì, C., Lehväslaiho, H., Markram, H., Schürmann, F., Magistretti, P.J., 2018. Norepinephrine stimulates glycogenolysis in astrocytes to fuel neurons with lactate. PLoS Comput. Biol. 14, e1006392. https://doi.org/10.1371/journal.pcbi.1006392

Cohen, J., Dunbar, K., Mcclelland, J., 1990. Cohen JD, McClelland JL, Dunbar K. On the control of automatic processes: a parallel distributed processing account of the Stroop effect. Psychol

References

271

Rev 97: 332-361. Psychol. Rev. 97, 332–61. https://doi.org/10.1037/0033-295X.97.3.332

Cohen, R., Lohr, I., Paul, R., Boland, R., 2001. Impairments of attention and effort among patients with major affective disorders. J. Neuropsychiatry Clin. Neurosci. 13, 385–395. https://doi.org/10.1176/jnp.13.3.385

Cohen, Z., Molinatti, G., Hamel, E., 1997. Astroglial and vascular interactions of noradrenaline terminals in the rat cerebral cortex. J. Cereb. Blood Flow Metab. Off. J. Int. Soc. Cereb. Blood Flow Metab. 17, 894–904. https://doi.org/10.1097/00004647-199708000-00008

Coradazzi, M., Gulino, R., Fieramosca, F., Falzacappa, L.V., Riggi, M., Leanza, G., 2016. Selective noradrenaline depletion impairs working memory and hippocampal neurogenesis. Neurobiol. Aging 48, 93–102. https://doi.org/10.1016/j.neurobiolaging.2016.08.012

Corbetta, M., Patel, G., Shulman, G.L., 2008. The Reorienting System of the Human Brain: From Environment to Theory of Mind. Neuron 58, 306–324. https://doi.org/10.1016/j.neuron.2008.04.017

Coricelli, G., Critchley, H.D., Joffily, M., O’Doherty, J.P., Sirigu, A., Dolan, R.J., 2005. Regret and its avoidance: a neuroimaging study of choice behavior. Nat. Neurosci. 8, 1255–1262. https://doi.org/10.1038/nn1514

Coull, J.T., Büchel, C., Friston, K.J., Frith, C.D., 1999. Noradrenergically mediated plasticity in a human attentional neuronal network. NeuroImage 10, 705–715. https://doi.org/10.1006/nimg.1999.0513

Coull, J.T., Nobre, A.C., Frith, C.D., 2001. The Noradrenergic α2 Agonist Clonidine Modulates Behavioural and Neuroanatomical Correlates of Human Attentional Orienting and Alerting. Cereb. Cortex 11, 73–84. https://doi.org/10.1093/cercor/11.1.73

Cousins, M.S., Salamone, J.D., 1994. Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit procedure. Pharmacol. Biochem. Behav. 49, 85–91. https://doi.org/10.1016/0091-3057(94)90460-x

Cowan, N., 2005. Working memory capacity, Essays in cognitive psychology. Psychology Press, New York.

Croxson, P.L., Walton, M.E., O’Reilly, J.X., Behrens, T.E.J., Rushworth, M.F.S., 2009. Effort-Based Cost–Benefit Valuation and the Human Brain. J. Neurosci. 29, 4531–4541. https://doi.org/10.1523/JNEUROSCI.4515-08.2009

Culbreth, A., Westbrook, A., Barch, D., 2016. Negative symptoms are associated with an increased subjective cost of cognitive effort. J. Abnorm. Psychol. 125, 528–536. https://doi.org/10.1037/abn0000153

Curtis, A.L., Lechner, S.M., Pavcovich, L.A., Valentino, R.J., 1997. Activation of the locus coeruleus noradrenergic system by intracoerulear microinfusion of corticotropin-releasing factor: effects on discharge rate, cortical norepinephrine levels and cortical electroencephalographic activity. J. Pharmacol. Exp. Ther. 281, 163–172.

Curtis, A.L., Leiser, S.C., Snyder, K., Valentino, R.J., 2012. Predator stress engages corticotropin-releasing factor and opioid systems to alter the operating mode of locus coeruleus norepinephrine neurons. Neuropharmacology 62, 1737–1745. https://doi.org/10.1016/j.neuropharm.2011.11.020

Da Silva, D.F., Mohammad, S., Hutchinson, K.A., Adamo, K.B., 2020. Cross-Validation of Ratings of Perceived Exertion Derived from Heart Rate Target Ranges Recommended for Pregnant Women. Int. J. Exerc. Sci. 13, 1340–1351.

Dahlstroem, A., Fuxe, K., 1964. EVIDENCE FOR THE EXISTENCE OF MONOAMINE-CONTAINING NEURONS IN THE CENTRAL NERVOUS SYSTEM. I. DEMONSTRATION OF MONOAMINES IN THE CELL BODIES OF BRAIN STEM NEURONS. Acta Physiol. Scand. Suppl. SUPPL 232:1-55.

Dajani, D.R., Uddin, L.Q., 2015. Demystifying cognitive flexibility: Implications for clinical and developmental neuroscience. Trends Neurosci. 38, 571–578. https://doi.org/10.1016/j.tins.2015.07.003

Dalton, G.L., Ma, L.M., Phillips, A.G., Floresco, S.B., 2011. Blockade of NMDA GluN2B receptors selectively impairs behavioral flexibility but not

References

272

initial discrimination learning. Psychopharmacology (Berl.) 216, 525–535. https://doi.org/10.1007/s00213-011-2246-z

Dan, Y., Poo, M.-M., 2006. Spike timing-dependent plasticity: from synapse to perception. Physiol. Rev. 86, 1033–1048. https://doi.org/10.1152/physrev.00030.2005

Dayan, P., 2012. How to set the switches on this thing. Curr. Opin. Neurobiol., Decision making 22, 1068–1074. https://doi.org/10.1016/j.conb.2012.05.011

de Gee, J.W., Colizoli, O., Kloosterman, N.A., Knapen, T., Nieuwenhuis, S., Donner, T.H., 2017. Dynamic modulation of decision biases by brainstem arousal systems. eLife 6, e23232. https://doi.org/10.7554/eLife.23232

de Gee, J.W., Knapen, T., Donner, T.H., 2014. Decision-related pupil dilation reflects upcoming choice and individual bias. Proc. Natl. Acad. Sci. 111, E618–E625. https://doi.org/10.1073/pnas.1317557111

de Gee, J.W., Tsetsos, K., Schwabe, L., Urai, A.E., McCormick, D., McGinley, M.J., Donner, T.H., 2020. Pupil-linked phasic arousal predicts a reduction of choice bias across species and decision domains. eLife 9, e54014. https://doi.org/10.7554/eLife.54014

De Martino, B., Kumaran, D., Seymour, B., Dolan, R.J., 2006. Frames, biases, and rational decision-making in the human brain. Science 313, 684–687. https://doi.org/10.1126/science.1128356

de Morree, H.M., Klein, C., Marcora, S.M., 2012. Perception of effort reflects central motor command during movement execution. Psychophysiology 49, 1242–1253. https://doi.org/10.1111/j.1469-8986.2012.01399.x

De Neys, W., Glumicic, T., 2008. Conflict monitoring in dual process theories of thinking. Cognition 106, 1248–1299. https://doi.org/10.1016/j.cognition.2007.06.002

De Neys, W., Vartanian, O., Goel, V., 2008. Smarter than we think: when our brains detect that we are biased. Psychol. Sci. 19, 483–489. https://doi.org/10.1111/j.1467-9280.2008.02113.x

de Rover, M., Brown, S.B.R.E., Band, G.P., Giltay, E.J., van Noorden, M.S., van der Wee, N.J.A., Nieuwenhuis, S., 2015. Beta receptor-mediated modulation of the oddball P3 but not error-related ERP components in humans. Psychopharmacology (Berl.) 232, 3161–3172. https://doi.org/10.1007/s00213-015-3966-2

De Sarro, G.B., Ascioti, C., Froio, F., Libri, V., Nisticò, G., 1987. Evidence that locus coeruleus is the site where clonidine and drugs acting at alpha 1- and alpha 2-adrenoceptors affect sleep and arousal mechanisms. Br. J. Pharmacol. 90, 675–685. https://doi.org/10.1111/j.1476-5381.1987.tb11220.x

Dehaene, S., Posner, M.I., Tucker, D.M., 1994. Localization of a Neural System for Error Detection and Compensation. Psychol. Sci. 5, 303–305. https://doi.org/10.1111/j.1467-9280.1994.tb00630.x

Demonstration of extensive brainstem projections to medial and lateral thalamus and hypothalamus in the rat, 1990. . Neuroscience 35, 609–626. https://doi.org/10.1016/0306-4522(90)90333-Y

Denes-Raj, V., Epstein, S., 1994. Conflict between intuitive and rational processing: When people behave against their better judgment. J. Pers. Soc. Psychol. 66, 819–829. https://doi.org/10.1037/0022-3514.66.5.819

Denson, T.F., von Hippel, W., Kemp, R.I., Teo, L.S., 2010. Glucose consumption decreases impulsive aggression in response to provocation in aggressive individuals. J. Exp. Soc. Psychol. 46, 1023–1028. https://doi.org/10.1016/j.jesp.2010.05.023

Devauges, V., Sara, S.J., 1990. Activation of the noradrenergic system facilitates an attentional shift in the rat. Behav. Brain Res. 39, 19–28. https://doi.org/10.1016/0166-4328(90)90118-x

Devilbiss, D.M., Page, M.E., Waterhouse, B.D., 2006. Locus ceruleus regulates sensory encoding by neurons and networks in waking animals. J. Neurosci. Off. J. Soc. Neurosci. 26, 9860–9872. https://doi.org/10.1523/JNEUROSCI.1776-06.2006

Devoto, P., Flore, G., 2006. On the origin of cortical dopamine: is it a co-transmitter in noradrenergic neurons? Curr. Neuropharmacol.

References

273

4, 115–125. https://doi.org/10.2174/157015906776359559

Devoto, P., Flore, G., Longu, G., Pira, L., Gessa, G.L., 2003. Origin of extracellular dopamine from dopamine and noradrenaline neurons in the medial prefrontal and occipital cortex. Synap. N. Y. N 50, 200–205. https://doi.org/10.1002/syn.10264

Devoto, P., Flore, G., Pani, L., Gessa, G.L., 2001. Evidence for co-release of noradrenaline and dopamine from noradrenergic neurons in the cerebral cortex. Mol. Psychiatry 6, 657–664. https://doi.org/10.1038/sj.mp.4000904

Devoto, P., Flore, G., Saba, P., Fà, M., Gessa, G.L., 2005. Co-release of noradrenaline and dopamine in the cerebral cortex elicited by single train and repeated train stimulation of the locus coeruleus. BMC Neurosci. 6, 31. https://doi.org/10.1186/1471-2202-6-31

Dewey, J., 1897. The Psychology of Effort. Philos. Rev. 6, 43–56. https://doi.org/10.2307/2175586

Dickinson, A., 1980. Contemporary Animal Learning Theory. CUP Archive.

Dickinson, A., Balleine, B., 1994. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18. https://doi.org/10.3758/BF03199951

Dienel, G.A., 2017. The metabolic trinity, glucose-glycogen-lactate, links astrocytes and neurons in brain energetics, signaling, memory, and gene expression. Neurosci. Lett. 637, 18–25. https://doi.org/10.1016/j.neulet.2015.02.052

Dixon, M.L., Christoff, K., 2012. The Decision to Engage Cognitive Control Is Driven by Expected Reward-Value: Neural and Behavioral Evidence. PLOS ONE 7, e51637. https://doi.org/10.1371/journal.pone.0051637

Dmochowski, J.P., Norcia, A.M., 2015. Cortical Components of Reaction-Time during Perceptual Decisions in Humans. PLOS ONE 10, e0143339. https://doi.org/10.1371/journal.pone.0143339

Donohoe, R.T., Benton, D., 1999. Blood glucose control and aggressiveness in females. Personal. Individ. Differ. 26, 905–911. https://doi.org/10.1016/S0191-8869(98)00191-3

Dorr, A.E., Debonnel, G., 2006. Effect of vagus nerve stimulation on serotonergic and noradrenergic transmission. J. Pharmacol. Exp. Ther. 318, 890–898. https://doi.org/10.1124/jpet.106.104166

Dow, J., Werlang, S.R. da C., 1992. Uncertainty Aversion, Risk Aversion, and the Optimal Choice of Portfolio. Econometrica 60, 197–204. https://doi.org/10.2307/2951685

Doya, K., 2002. Metalearning and neuromodulation. Neural Netw. 15, 495–506. https://doi.org/10.1016/S0893-6080(02)00044-8

Dretsch, M.N., Tipples, J., 2008. Working memory involved in predicting future outcomes based on past experiences. Brain Cogn. 66, 83–90. https://doi.org/10.1016/j.bandc.2007.05.006

Drugowitsch, J., Wyart, V., Devauchelle, A.-D., Koechlin, E., 2016. Computational Precision of Mental Inference as Critical Source of Human Choice Suboptimality. Neuron 92, 1398–1411. https://doi.org/10.1016/j.neuron.2016.11.005

Dubois, M., Habicht, J., Michely, J., Moran, R., Dolan, R.J., Hauser, T.U., 2021. Human complex exploration strategies are enriched by noradrenaline-modulated heuristics. eLife 10, e59907. https://doi.org/10.7554/eLife.59907

Dunn, T.L., Lutes, D.J.C., Risko, E.F., 2016. Metacognitive evaluation in the avoidance of demand. J. Exp. Psychol. Hum. Percept. Perform. 42, 1372–1387. https://doi.org/10.1037/xhp0000236

Dvorak, R.D., Simons, J.S., 2009. Moderation of Resource Depletion in the Self-Control Strength Model: Differing Effects of Two Modes of Self-Control. Pers. Soc. Psychol. Bull. 35, 572–583. https://doi.org/10.1177/0146167208330855

Einhäuser, W., Koch, C., Carter, O.L., 2010. Pupil Dilation Betrays the Timing of Decisions. Front. Hum. Neurosci. 4, 18. https://doi.org/10.3389/fnhum.2010.00018

Einhauser, W., Stout, J., Koch, C., Carter, O., 2008. Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci. 105, 1704–1709. https://doi.org/10.1073/pnas.0707727105

References

274

Ellis, K.A., Nathan, P.J., 2001. The pharmacology of human working memory. Int. J. Neuropsychopharmacol. 4, 299–313. https://doi.org/10.1017/S1461145701002541

Engelmann, J.B., Damaraju, E., Padmala, S., Pessoa, L., 2009. Combined Effects of Attention and Motivation on Visual Task Performance: Transient and Sustained Motivational Effects. Front. Hum. Neurosci. 3. https://doi.org/10.3389/neuro.09.004.2009

Ennis, M., Aston-Jones, G., 1988. Activation of locus coeruleus from nucleus paragigantocellularis: a new excitatory amino acid pathway in brain. J. Neurosci. 8, 3644–3657. https://doi.org/10.1523/JNEUROSCI.08-10-03644.1988

Ennis, M., Aston-Jones, G., 1986. Evidence for self- and neighbor-mediated postactivation inhibition of locus coeruleus neurons. Brain Res. 374, 299–305. https://doi.org/10.1016/0006-8993(86)90424-5

Epstein, S., 1994. Integration of the cognitive and the psychodynamic unconscious. Am. Psychol. 49, 709–724. https://doi.org/10.1037/0003-066X.49.8.709

Epstein, S., Pacini, R., 1999. Some basic issues regarding dual-process theories from the perspective of cognitive–experiential self-theory, in: Dual-Process Theories in Social Psychology. The Guilford Press, New York, NY, US, pp. 462–482.

Ericsson, K.A., Simon, H.A., 1980. Verbal reports as data. Psychol. Rev. 87, 215–251. https://doi.org/10.1037/0033-295X.87.3.215

Eslinger, P.J., Damasio, A.R., 1985. Severe disturbance of higher cognition after bilateral frontal lobe ablation: patient EVR. Neurology 35, 1731–1741. https://doi.org/10.1212/wnl.35.12.1731

Evans, J.S.B.T., 1984. Heuristic and analytic processes in reasoning*. Br. J. Psychol. 75, 451–468. https://doi.org/10.1111/j.2044-8295.1984.tb01915.x

Evans, J.S.B.T., Over, D.E., 2013. Rationality and Reasoning. Psychology Press.

Evans, M., Shaw, A., Thompson, E.A., Falk, S., Turton, P., Thompson, T., Sharp, D., 2007.

Decisions to use complementary and alternative medicine (CAM) by male cancer patients: information-seeking roles and types of evidence used. BMC Complement. Altern. Med. 7, 25. https://doi.org/10.1186/1472-6882-7-25

Fairclough, S.H., Houston, K., 2004. A metabolic measure of mental effort. Biol. Psychol. 66, 177–190. https://doi.org/10.1016/j.biopsycho.2003.10.001

Faisal, A.A., Selen, L.P.J., Wolpert, D.M., 2008. Noise in the nervous system. Nat. Rev. Neurosci. 9, 292–303. https://doi.org/10.1038/nrn2258

Falcone, R., Weintraub, D.B., Setogawa, T., Wittig, J.H., Chen, G., Richmond, B.J., 2019. Temporal Coding of Reward Value in Monkey Ventral Striatal Tonically Active Neurons. J. Neurosci. Off. J. Soc. Neurosci. 39, 7539–7550. https://doi.org/10.1523/JNEUROSCI.0869-19.2019

Falkenberg, L.E., Westerhausen, R., Specht, K., Hugdahl, K., 2012. Resting-state glutamate level in the anterior cingulate predicts blood-oxygen level-dependent response to cognitive control. Proc. Natl. Acad. Sci. 109, 5069–5073.

Falkenstein, M., Hoormann, J., Hohnsbein, J., 1999. ERP components in Go/Nogo tasks and their relation to inhibition. Acta Psychol. (Amst.) 101, 267–291. https://doi.org/10.1016/S0001-6918(99)00008-6

Faraone, S.V., Biederman, J., Spencer, T., Michelson, D., Adler, L., Reimherr, F., Seidman, L., 2005. Atomoxetine and Stroop Task Performance in Adult Attention-Deficit/Hyperactivity Disorder. J. Child Adolesc. Psychopharmacol. 15, 664–670. https://doi.org/10.1089/cap.2005.15.664

Feldman, J., Barshi, I., 2007. The Effects of Blood Glucose Levels on Cognitive Performance: A Review of the Literature [WWW Document]. Tech. Rep. NASA Moffett Field CA. URL https://ntrs.nasa.gov/citations/20070031714 (accessed 4.23.21).

Fernando, A.B.P., Economidou, D., Theobald, D.E., Zou, M.-F., Newman, A.H., Spoelder, M., Caprioli, D., Moreno, M., Hipólito, L., Aspinall, A.T., Robbins, T.W., Dalley, J.W., 2012. Modulation of high impulsivity and attentional performance in rats by selective direct and indirect dopaminergic and noradrenergic

References

275

receptor agonists. Psychopharmacology (Berl.) 219, 341–352. https://doi.org/10.1007/s00213-011-2408-z

Ferry, B., Roozendaal, B., McGaugh, J.L., 1999. Basolateral Amygdala Noradrenergic Influences on Memory Storage Are Mediated by an Interaction between β- and α1-Adrenoceptors. J. Neurosci. 19, 5119–5123. https://doi.org/10.1523/JNEUROSCI.19-12-05119.1999

Fiedler, S., Glöckner, A., 2012. The Dynamics of Decision Making in Risky Choice: An Eye-Tracking Analysis. Front. Psychol. 3. https://doi.org/10.3389/fpsyg.2012.00335

Fiehler, K., Ullsperger, M., Cramon, D.Y.V., 2004. Neural correlates of error detection and error correction: is there a common neuroanatomical substrate? Eur. J. Neurosci. 19, 3081–3087. https://doi.org/10.1111/j.0953-816X.2004.03414.x

Filevich, E., Kühn, S., Haggard, P., 2013. There Is No Free Won’t: Antecedent Brain Activity Predicts Decisions to Inhibit. PLOS ONE 8, e53053. https://doi.org/10.1371/journal.pone.0053053

Filevich, E., Kühn, S., Haggard, P., 2012. Intentional inhibition in human action: the power of “no.” Neurosci. Biobehav. Rev. 36, 1107–1118. https://doi.org/10.1016/j.neubiorev.2012.01.006

Fink, K., Velebit, J., Vardjan, N., Zorec, R., Kreft, M., 2021. Noradrenaline-induced l-lactate production requires d-glucose entry and transit through the glycogen shunt in single-cultured rat astrocytes. J. Neurosci. Res. 99, 1084–1098. https://doi.org/10.1002/jnr.24783

Fischer, R., Ventura-Bort, C., Hamm, A., Weymar, M., 2018. Transcutaneous vagus nerve stimulation (tVNS) enhances conflict-triggered adjustment of cognitive control. Cogn. Affect. Behav. Neurosci. 18, 680–693. https://doi.org/10.3758/s13415-018-0596-2

Flicker, C., Geyer, M.A., 1982. The hippocampus as a possible site of action for increased locomotion during intracerebral infusions of norepinephrine. Behav. Neural Biol. 34, 421–426. https://doi.org/10.1016/s0163-1047(82)91843-x

Floresco, S.B., Tse, M.T.L., Ghods-Sharifi, S., 2008. Dopaminergic and Glutamatergic Regulation of Effort- and Delay-Based Decision Making. Neuropsychopharmacology 33, 1966–1979. https://doi.org/10.1038/sj.npp.1301565

Folstein, J.R., Van Petten, C., 2008. Influence of cognitive control and mismatch on the N2 component of the ERP: a review. Psychophysiology 45, 152–170. https://doi.org/10.1111/j.1469-8986.2007.00602.x

Foote, S.L., Aston-Jones, G., Bloom, F.E., 1980. Impulse activity of locus coeruleus neurons in awake rats and monkeys is a function of sensory stimulation and arousal. Proc. Natl. Acad. Sci. U. S. A. 77, 3033–3037. https://doi.org/10.1073/pnas.77.5.3033

Foote, S.L., Bloom, F.E., Aston-Jones, G., 1983. Nucleus locus ceruleus: new evidence of anatomical and physiological specificity. Physiol. Rev. 63, 844–914. https://doi.org/10.1152/physrev.1983.63.3.844

Foote, S.L., Freedman, R., Oliver, A.P., 1975. Effects of putative neurotransmitters on neuronal activity in monkey auditory cortex. Brain Res. 86, 229–242. https://doi.org/10.1016/0006-8993(75)90699-x

Förster, J., Higgins, E.T., Bianco, A.T., 2003. Speed/accuracy decisions in task performance: Built-in trade-off or separate strategic concerns? Organ. Behav. Hum. Decis. Process. 90, 148–164. https://doi.org/10.1016/S0749-5978(02)00509-5

Forstmann, B.U., Brass, M., Koch, I., von Cramon, D.Y., 2006. Voluntary selection of task sets revealed by functional magnetic resonance imaging. J. Cogn. Neurosci. 18, 388–398. https://doi.org/10.1162/089892906775990589

Fox, M.D., Raichle, M.E., 2007. Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat. Rev. Neurosci. 8, 700–711. https://doi.org/10.1038/nrn2201

Frangos, E., Ellrich, J., Komisaruk, B.R., 2015. Non-invasive Access to the Vagus Nerve Central Projections via Electrical Stimulation of the External Ear: fMRI Evidence in Humans. Brain Stimulat. 8, 624–636. https://doi.org/10.1016/j.brs.2014.11.018

References

276

Frank, M.J., Badre, D., 2012. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb. Cortex N. Y. N 1991 22, 509–526. https://doi.org/10.1093/cercor/bhr114

Franks, N.R., Dornhaus, A., Fitzsimmons, J.P., Stevens, M., 2003. Speed versus accuracy in collective decision making. Proc. R. Soc. Lond. B Biol. Sci. 270, 2457–2463. https://doi.org/10.1098/rspb.2003.2527

Franowicz, J.S., Arnsten, A.F., 1998. The alpha-2a noradrenergic agonist, guanfacine, improves delayed response performance in young adult rhesus monkeys. Psychopharmacology (Berl.) 136, 8–14. https://doi.org/10.1007/s002130050533

Freedman, R., Hoffer, B.J., Woodward, D.J., Puro, D., 1977. Interaction of norepinephrine with cerebellar activity evoked by mossy and climbing fibers. Exp. Neurol. 55, 269–288. https://doi.org/10.1016/0014-4886(77)90175-3

Fries, P., 2005. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn. Sci. 9, 474–480. https://doi.org/10.1016/j.tics.2005.08.011

Fudenberg, D., Strack, P., Strzalecki, T., 2018. Speed, Accuracy, and the Optimal Timing of Choices. Am. Econ. Rev. 108, 3651–3684. https://doi.org/10.1257/aer.20150742

Fuxe, K., Borroto-Escuela, D.O., Romero-Fernandez, W., Diaz-Cabiale, Z., Rivera, A., Ferraro, L., Tanganelli, S., Tarakanov, A.O., Garriga, P., Narváez, J.A., Ciruela, F., Guescini, M., Agnati, L.F., 2012. Extrasynaptic neurotransmission in the modulation of brain function. Focus on the striatal neuronal-glial networks. Front. Physiol. 3, 136. https://doi.org/10.3389/fphys.2012.00136

Gaillard, A.W.K., 2001. Stress, workload, and fatigue as three biobehavioral states: A general overview, in: Stress, Workload, and Fatigue, Human Factors in Transportation. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, pp. 623–639.

Gaillard, A.W.K., 1993. Comparing the concepts of mental load and stress. Ergonomics 36, 991–1005. https://doi.org/10.1080/00140139308967972

Gailliot, M.T., 2008. Unlocking the Energy Dynamics of Executive Functioning: Linking Executive Functioning to Brain Glycogen. Perspect. Psychol. Sci. J. Assoc. Psychol. Sci. 3, 245–263. https://doi.org/10.1111/j.1745-6924.2008.00077.x

Gailliot, M.T., Baumeister, R.F., 2007. The Physiology of Willpower: Linking Blood Glucose to Self-Control. Personal. Soc. Psychol. Rev. 11, 303–327. https://doi.org/10.1177/1088868307303030

Gailliot, M.T., Baumeister, R.F., DeWall, C.N., Maner, J.K., Plant, E.A., Tice, D.M., Brewer, L.E., Schmeichel, B.J., 2007. Self-control relies on glucose as a limited energy source: willpower is more than a metaphor. J. Pers. Soc. Psychol. 92, 325–336. https://doi.org/10.1037/0022-3514.92.2.325

Gajewski, P.D., Falkenstein, M., 2011. Diversity of the P3 in the task-switching paradigm. Brain Res. 1411, 87–97. https://doi.org/10.1016/j.brainres.2011.07.010

Gallagher, K.M., Fadel, P.J., Strømstad, M., Ide, K., Smith, S.A., Querry, R.G., Raven, P.B., Secher, N.H., 2001. Effects of partial neuromuscular blockade on carotid baroreflex function during exercise in humans. J. Physiol. 533, 861–870. https://doi.org/10.1111/j.1469-7793.2001.t01-1-00861.x

Gamo, N.J., Wang, M., Arnsten, A.F.T., 2010. Methylphenidate and Atomoxetine Enhance Prefrontal Function Through α2-Adrenergic and Dopamine D1 Receptors. J. Am. Acad. Child Adolesc. Psychiatry 49, 1011–1023. https://doi.org/10.1016/j.jaac.2010.06.015

Garavan, H., Ross, T.J., Stein, E.A., 1999. Right hemispheric dominance of inhibitory control: An event-related functional MRI study. Proc. Natl. Acad. Sci. 96, 8301–8306. https://doi.org/10.1073/pnas.96.14.8301

Garrison, K.E., Finley, A.J., Schmeichel, B.J., 2019. Ego Depletion Reduces Attention Control: Evidence From Two High-Powered Preregistered Experiments. Pers. Soc. Psychol. Bull. 45, 728–739. https://doi.org/10.1177/0146167218796473

Gatti, P.J., Hill, K.J., Da Silva, A.M., Norman, W.P., Gillis, R.A., 1988. Central nervous system site of

References

277

action for the hypotensive effect of clonidine in the cat. J. Pharmacol. Exp. Ther. 245, 373–380.

Gehring, W.J., Knight, R.T., 2000. Prefrontal–cingulate interactions in action monitoring. Nat. Neurosci. 3, 516–520. https://doi.org/10.1038/74899

Gelbard-Sagiv, H., Magidov, E., Sharon, H., Hendler, T., Nir, Y., 2018. Noradrenaline Modulates Visual Perception and Late Visually Evoked Activity. Curr. Biol. 28, 2239-2249.e6. https://doi.org/10.1016/j.cub.2018.05.051

Genkova-Papazova, M., Petkova, B.P., Lazarova-Bakarova, M., Boyanova, E., Staneva-Stoytcheva, D., 1997. Effects of F lunarizine and Nitrendipine on Electroconvulsive Shock- and Clonidine-Induced Amnesia. Pharmacol. Biochem. Behav. 56, 583–587. https://doi.org/10.1016/S0091-3057(96)00406-6

Geurts, H.M., Verté, S., Oosterlaan, J., Roeyers, H., Sergeant, J.A., 2004. How specific are executive functioning deficits in attention deficit hyperactivity disorder and autism? J. Child Psychol. Psychiatry 45, 836–854. https://doi.org/10.1111/j.1469-7610.2004.00276.x

Gibbs, M.E., 2015. Role of Glycogenolysis in Memory and Learning: Regulation by Noradrenaline, Serotonin and ATP. Front. Integr. Neurosci. 9, 70. https://doi.org/10.3389/fnint.2015.00070

Gibbs, M.E., Hutchinson, D.S., Summers, R.J., 2010. Noradrenaline release in the locus coeruleus modulates memory formation and consolidation; roles for α- and β-adrenergic receptors. Neuroscience 170, 1209–1222. https://doi.org/10.1016/j.neuroscience.2010.07.052

Gigerenzer, G., 2010. Rationality for mortals: how people cope with uncertainty, Evolution and cognition. Oxford University Press, New York ; Oxford.

Gigerenzer, G., 2008. Why Heuristics Work. Perspect. Psychol. Sci. 3, 20–29. https://doi.org/10.1111/j.1745-6916.2008.00058.x

Gigerenzer, G., Goldstein, D.G., 1996. Reasoning the fast and frugal way: models of bounded

rationality. Psychol. Rev. 103, 650–669. https://doi.org/10.1037/0033-295x.103.4.650

Gilden, D.L., 1997. Fluctuations in the Time Required for Elementary Decisions. Psychol. Sci. 8, 296–301. https://doi.org/10.1111/j.1467-9280.1997.tb00441.x

Gilzenrat, M.S., Holmes, B.D., Rajkowski, J., Aston-Jones, G., Cohen, J.D., 2002. Simplified dynamics in a model of noradrenergic modulation of cognitive performance. Neural Netw. 15, 647–663. https://doi.org/10.1016/S0893-6080(02)00055-2

Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M., Cohen, J.D., 2010. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affect. Behav. Neurosci. 10, 252–269. https://doi.org/10.3758/CABN.10.2.252

Gloeckner, A., Witteman, C., 2010. Beyond dual-process models: A categorisation of processes underlying intuitive judgement and decision making. Think. Reason. 16, 1–25. https://doi.org/10.1080/13546780903395748

Gneezy, U., Rustichini, A., 2000. Pay Enough or Don’t Pay at All. Q. J. Econ. 115, 791–810.

Goel, V., 1995. Sketches of Thought. MIT Press.

Goldman, G., Coleman, P.D., 1981. Neuron numbers in locus coeruleus do not change with age in Fisher 344 rat. Neurobiol. Aging 2, 33–36. https://doi.org/10.1016/0197-4580(81)90056-7

Grant, S.J., Aston-Jones, G., Redmond, D.E., 1988. Responses of primate locus coeruleus neurons to simple and complex sensory stimuli. Brain Res. Bull. 21, 401–410. https://doi.org/10.1016/0361-9230(88)90152-9

Green, L., Myerson, J., Oliveira, L., Chang, S.E., 2013. Delay discounting of monetary rewards over a wide range of amounts. J. Exp. Anal. Behav. 100, 269–281. https://doi.org/10.1002/jeab.45

Griffiths, T.L., Lieder, F., Goodman, N.D., 2015. Rational use of cognitive resources: levels of analysis between the computational and the algorithmic. Top. Cogn. Sci. 7, 217–229. https://doi.org/10.1111/tops.12142

References

278

Gross, J.J., 2011. Handbook of Emotion Regulation, First Edition. Guilford Press.

Gruetter, R., 2003. Glycogen: The forgotten cerebral energy store. J. Neurosci. Res. 74, 179–183. https://doi.org/10.1002/jnr.10785

Gubits, R.M., Smith, T.M., Fairhurst, J.L., Yu, H., 1989. Adrenergic receptors mediate changes in c-fos mRNA levels in brain. Mol. Brain Res. 6, 39–45. https://doi.org/10.1016/0169-328X(89)90026-0

Guedj, C., Meunier, D., Meunier, M., Hadj-Bouziane, F., 2017a. Could LC-NE-Dependent Adjustment of Neural Gain Drive Functional Brain Network Reorganization? Neural Plast. 2017, 4328015. https://doi.org/10.1155/2017/4328015

Guedj, C., Monfardini, E., Reynaud, A.J., Farnè, A., Meunier, M., Hadj-Bouziane, F., 2017b. Boosting Norepinephrine Transmission Triggers Flexible Reconfiguration of Brain Networks at Rest. Cereb. Cortex N. Y. N 1991 27, 4691–4700. https://doi.org/10.1093/cercor/bhw262

Habay, J., Van Cutsem, J., Verschueren, J., De Bock, S., Proost, M., De Wachter, J., Tassignon, B., Meeusen, R., Roelands, B., 2021. Mental Fatigue and Sport-Specific Psychomotor Performance: A Systematic Review. Sports Med. https://doi.org/10.1007/s40279-021-01429-6

Hagena, H., Hansen, N., Manahan-Vaughan, D., 2016. β-Adrenergic Control of Hippocampal Function: Subserving the Choreography of Synaptic Information Storage and Memory. Cereb. Cortex N. Y. NY 26, 1349–1364. https://doi.org/10.1093/cercor/bhv330

Hagger, M.S., Wood, C., Stiff, C., Chatzisarantis, N.L.D., 2010. Ego depletion and the strength model of self-control: A meta-analysis. Psychol. Bull. 136, 495–525. https://doi.org/10.1037/a0019486

Halliday, R., Naylor, H., Brandeis, D., Callaway, E., Yano, L., Herzig, K., 1994. The effect of D-amphetamine, clonidine, and yohimbine on human information processing. Psychophysiology 31, 331–337. https://doi.org/10.1111/j.1469-8986.1994.tb02441.x

Hare, T.A., Schultz, W., Camerer, C.F., O’Doherty, J.P., Rangel, A., 2011. Transformation of stimulus

value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. 108, 18120–18125. https://doi.org/10.1073/pnas.1109322108

Harley, C.W., Sara, S.J., 1992. Locus coeruleus bursts induced by glutamate trigger delayed perforant path spike amplitude potentiation in the dentate gyrus. Exp. Brain Res. 89, 581–587. https://doi.org/10.1007/BF00229883

Hay, N., Russell, S., Tolpin, D., Shimony, S.E., 2014. Selecting Computations: Theory and Applications. ArXiv14082048 Cs.

Hayden, B.Y., Platt, M.L., 2010. Neurons in Anterior Cingulate Cortex Multiplex Information about Reward and Action. J. Neurosci. 30, 3339–3346.

Hazy, T.E., Frank, M.J., O’Reilly, R.C., 2007. Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philos. Trans. R. Soc. B Biol. Sci. 362, 1601–1613. https://doi.org/10.1098/rstb.2007.2055

Heath, T.P., Melichar, J.K., Nutt, D.J., Donaldson, L.F., 2006. Human Taste Thresholds Are Modulated by Serotonin and Noradrenaline. J. Neurosci. 26, 12664–12671. https://doi.org/10.1523/JNEUROSCI.3459-06.2006

Heginbotham, L.R., Dunwiddie, T.V., 1991. Long-term increases in the evoked population spike in the CA1 region of rat hippocampus induced by beta-adrenergic receptor activation. J. Neurosci. 11, 2519–2527. https://doi.org/10.1523/JNEUROSCI.11-08-02519.1991

Heitz, R.P., 2014. The speed-accuracy tradeoff: history, physiology, methodology, and behavior. Front. Neurosci. 8. https://doi.org/10.3389/fnins.2014.00150

Heitz, R.P., Schall, J.D., 2012. Neural Mechanisms of Speed-Accuracy Tradeoff. Neuron 76, 616–628. https://doi.org/10.1016/j.neuron.2012.08.030

Heitz, R.P., Schrock, J.C., Payne, T.W., Engle, R.W., 2007. Effects of incentive on working memory capacity: Behavioral and pupillometric data. Psychophysiology 0, 071003012229005-??? https://doi.org/10.1111/j.1469-8986.2007.00605.x

References

279

Hermans, E.J., Henckens, M.J.A.G., Joëls, M., Fernández, G., 2014. Dynamic adaptation of large-scale brain networks in response to acute stressors. Trends Neurosci. 37, 304–314. https://doi.org/10.1016/j.tins.2014.03.006

Hermans, E.J., Marle, H.J.F. van, Ossewaarde, L., Henckens, M.J.A.G., Qin, S., Kesteren, M.T.R. van, Schoots, V.C., Cousijn, H., Rijpkema, M., Oostenveld, R., Fernández, G., 2011. Stress-Related Noradrenergic Activity Prompts Large-Scale Neural Network Reconfiguration. Science 334, 1151–1153. https://doi.org/10.1126/science.1209603

Hernaus, D., Casales Santa, M.M., Offermann, J.S., Van Amelsvoort, T., 2017. Noradrenaline transporter blockade increases fronto-parietal functional connectivity relevant for working memory. Eur. Neuropsychopharmacol. 27, 399–410. https://doi.org/10.1016/j.euroneuro.2017.02.004

Hertz, L., Lovatt, D., Goldman, S.A., Nedergaard, M., 2010. Adrenoceptors in brain: cellular gene expression and effects on astrocytic metabolism and [Ca(2+)]i. Neurochem. Int. 57, 411–420. https://doi.org/10.1016/j.neuint.2010.03.019

Hertz, L., Peng, L., Dienel, G.A., 2007. Energy metabolism in astrocytes: high rate of oxidative metabolism and spatiotemporal dependence on glycolysis/glycogenolysis. J. Cereb. Blood Flow Metab. Off. J. Int. Soc. Cereb. Blood Flow Metab. 27, 219–249. https://doi.org/10.1038/sj.jcbfm.9600343

Hervé-Minvielle, A., Sara, S., 1995. Rapid habituation of auditory responses of locus coeruleus cells in anaesthetized and awake rats. Neuroreport 6, 1363–1368. https://doi.org/10.1097/00001756-199507100-00001

Hess, E.H., Polt, J.M., 1964. Pupil Size in Relation to Mental Activity during Simple Problem-Solving. Science 143, 1190–1192. https://doi.org/10.1126/science.143.3611.1190

Hillman, K.L., Bilkey, D.K., 2010. Neurons in the Rat Anterior Cingulate Cortex Dynamically Encode Cost–Benefit in a Spatial Decision-Making Task. J. Neurosci. 30, 7705–7713. https://doi.org/10.1523/JNEUROSCI.1273-10.2010

Hinson, J.M., Jameson, T.L., Whitney, P., 2003. Impulsive decision making and working memory. J. Exp. Psychol. Learn. Mem. Cogn. 29, 298–306. https://doi.org/10.1037/0278-7393.29.2.298

Hirschberg, S., Li, Y., Randall, A., Kremer, E.J., Pickering, A.E., 2017. Functional dichotomy in spinal- vs prefrontal-projecting locus coeruleus modules splits descending noradrenergic analgesia from ascending aversion and anxiety in rats. eLife 6. https://doi.org/10.7554/eLife.29808

Hirsh, J.B., Morisano, D., Peterson, J.B., 2008. Delay discounting: Interactions between personality and cognitive ability. J. Res. Personal. 42, 1646–1650. https://doi.org/10.1016/j.jrp.2008.07.005

Hobson, J.A., McCarley, R.W., Wyzinski, P.W., 1975. Sleep cycle oscillation: reciprocal discharge by two brainstem neuronal groups. Science 189, 55–58. https://doi.org/10.1126/science.1094539

Hoffer, B.J., Siggins, G.R., Oliver, A.P., Bloom, F.E., 1973. Activation of the Pathway from Locus Coeruleus to Rat Cerebellar Purkinje Neurons: Pharmacological Evidence of Noradrenrtgic Central Inhibition. J. Pharmacol. Exp. Ther. 184, 553–569.

Hogarth, R.M., 2005. Deciding analytically or trusting your intuition? The advantages and disadvantages of analytic and intuitive thought, in: The Routines of Decision Making. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, pp. 67–82.

Holroyd, C., 2016. The waste disposal problem of effortful control, in: Motivation and Cognitive Control. Routledge, pp. 235–260.

Holroyd, C.B., McClure, S.M., 2015. Hierarchical control over effortful behavior by rodent medial frontal cortex: A computational model. Psychol. Rev. 122, 54–83. https://doi.org/10.1037/a0038339

Hopkins, W.F., Johnston, D., 1988. Noradrenergic enhancement of long-term potentiation at mossy fiber synapses in the hippocampus. J. Neurophysiol. 59, 667–687. https://doi.org/10.1152/jn.1988.59.2.667

Hopkins, W.F., Johnston, D., 1984. Frequency-dependent noradrenergic modulation of long-term potentiation in the hippocampus. Science

References

280

226, 350–352. https://doi.org/10.1126/science.6091272

Hori, Y., Minamimoto, T., Kimura, M., 2009. Neuronal Encoding of Reward Value and Direction of Actions in the Primate Putamen. J. Neurophysiol. 102, 3530–3543. https://doi.org/10.1152/jn.00104.2009

Hosokawa, T., Kennerley, S.W., Sloan, J., Wallis, J.D., 2013. Single-Neuron Mechanisms Underlying Cost-Benefit Analysis in Frontal Cortex. J. Neurosci. 33, 17385–17397. https://doi.org/10.1523/JNEUROSCI.2221-13.2013

Huber, O., 1980. The influence of some task variables on cognitive operations in an information-processing decision model. Acta Psychol. (Amst.) 45, 187–196. https://doi.org/10.1016/0001-6918(80)90031-1

Hui, S., Ghergurovich, J.M., Morscher, R.J., Jang, C., Teng, X., Lu, W., Esparza, L.A., Reya, T., Le Zhan, null, Yanxiang Guo, J., White, E., Rabinowitz, J.D., 2017. Glucose feeds the TCA cycle via circulating lactate. Nature 551, 115–118. https://doi.org/10.1038/nature24057

Hull, C.L., 1943. Principles of behavior: an introduction to behavior theory, Principles of behavior: an introduction to behavior theory. Appleton-Century, Oxford, England.

Hunt, L.T., Kolling, N., Soltani, A., Woolrich, M.W., Rushworth, M.F.S., Behrens, T.E.J., 2012. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 15, 470–476. https://doi.org/10.1038/nn.3017

Hupé, J.-M., Lamirel, C., Lorenceau, J., 2009. Pupil dynamics during bistable motion perception. J. Vis. 9, 10–10. https://doi.org/10.1167/9.7.10

Hurley, L., Devilbiss, D., Waterhouse, B., 2004. A matter of focus: monoaminergic modulation of stimulus coding in mammalian sensory networks. Curr. Opin. Neurobiol. 14, 488–495. https://doi.org/10.1016/j.conb.2004.06.007

Ikemoto, S., Yang, C., Tan, A., 2015. Basal ganglia circuit loops, dopamine and motivation: A review and enquiry. Behav. Brain Res. 290, 17–31. https://doi.org/10.1016/j.bbr.2015.04.018

Inzlicht, M., Schmeichel, B.J., 2012. What Is Ego Depletion? Toward a Mechanistic Revision of the

Resource Model of Self-Control. Perspect. Psychol. Sci. 7, 450–463. https://doi.org/10.1177/1745691612454134

Inzlicht, M., Schmeichel, B.J., Macrae, C.N., 2014. Why self-control seems (but may not be) limited. Trends Cogn. Sci. 18, 127–133. https://doi.org/10.1016/j.tics.2013.12.009

Ishimatsu, M., Williams, J.T., 1996. Synchronous Activity in Locus Coeruleus Results from Dendritic Interactions in Pericoerulear Regions. J. Neurosci. 16, 5196–5204. https://doi.org/10.1523/JNEUROSCI.16-16-05196.1996

Jahn, A., Nee, D.E., Alexander, W.H., Brown, J.W., 2014. Distinct regions of anterior cingulate cortex signal prediction and outcome evaluation. NeuroImage 95, 80–89. https://doi.org/10.1016/j.neuroimage.2014.03.050

Jahn, C.I., Gilardeau, S., Varazzani, C., Blain, B., Sallet, J., Walton, M.E., Bouret, S., 2018. Dual contributions of noradrenaline to behavioural flexibility and motivation. Psychopharmacology (Berl.) 235, 2687–2702. https://doi.org/10.1007/s00213-018-4963-z

Jahn, C.I., Varazzani, C., Sallet, J., Walton, M.E., Bouret, S., 2020. Noradrenergic But Not Dopaminergic Neurons Signal Task State Changes and Predict Reengagement After a Failure. Cereb. Cortex N. Y. N 1991 30, 4979–4994. https://doi.org/10.1093/cercor/bhaa089

Jamadar, S., Thienel, R., Karayanidis, F., 2015. Task switching processes. Brain Mapp. Encycl. Ref. 327–335. https://doi.org/10.1016/B978-0-12-397025-1.00250-5

Jepma, M., Brown, S.B.R.E., Murphy, P.R., Koelewijn, S.C., de Vries, B., van den Maagdenberg, A.M., Nieuwenhuis, S., 2018. Noradrenergic and Cholinergic Modulation of Belief Updating. J. Cogn. Neurosci. 30, 1803–1820. https://doi.org/10.1162/jocn_a_01317

Jepma, M., Deinum, J., Asplund, C.L., Rombouts, S.A., Tamsma, J.T., Tjeerdema, N., Spapé, M.M., Garland, E.M., Robertson, D., Lenders, J.W., Nieuwenhuis, S., 2011. Neurocognitive function in dopamine-β-hydroxylase deficiency. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 36, 1608–1619. https://doi.org/10.1038/npp.2011.42

References

281

Jepma, M., Nieuwenhuis, S., 2011. Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory. J. Cogn. Neurosci. 23, 1587–1596. https://doi.org/10.1162/jocn.2010.21548

Jett, J.D., Bulin, S.E., Hatherall, L.C., McCartney, C.M., Morilak, D.A., 2017. Deficits in cognitive flexibility induced by chronic unpredictable stress are associated with impaired glutamate neurotransmission in the rat medial prefrontal cortex. Neuroscience 346, 284–297. https://doi.org/10.1016/j.neuroscience.2017.01.017

Jiang, M., Griff, E.R., Ennis, M., Zimmer, L.A., Shipley, M.T., 1996. Activation of Locus Coeruleus Enhances the Responses of Olfactory Bulb Mitral Cells to Weak Olfactory Nerve Input. J. Neurosci. 16, 6319–6329. https://doi.org/10.1523/JNEUROSCI.16-19-06319.1996

Jimura, K., Locke, H.S., Braver, T.S., 2010. Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts. Proc. Natl. Acad. Sci. 107, 8871–8876. https://doi.org/10.1073/pnas.1002007107

Job, V., Dweck, C.S., Walton, G.M., 2010. Ego Depletion—Is It All in Your Head?: Implicit Theories About Willpower Affect Self-Regulation. Psychol. Sci. 21, 1686–1693. https://doi.org/10.1177/0956797610384745

Jodo, E., Takeuchi, S., Kayama, Y., 1995. P3b-like potential of rats recorded in an active discrimination task. Electroencephalogr. Clin. Neurophysiol. 96, 555–560. https://doi.org/10.1016/0013-4694(95)00067-9

Johns, M., Inzlicht, M., Schmader, T., 2008. Stereotype threat and executive resource depletion: Examining the influence of emotion regulation. J. Exp. Psychol. Gen. 137, 691–705. https://doi.org/10.1037/a0013834

Johnson, E.J., Payne, J.W., 1985. Effort and Accuracy in Choice. Manag. Sci. 31, 395–414. https://doi.org/10.1287/mnsc.31.4.395

Jorna, P.G.A.M., 1992. Spectral analysis of heart rate and psychological state: A review of its validity as a workload index. Biol. Psychol. 34, 237–257. https://doi.org/10.1016/0301-0511(92)90017-O

Joshi, S., Gold, J.I., 2020. Context-Dependent Relationships between Locus Coeruleus Firing Patterns and Coordinated Neural Activity in the Anterior Cingulate Cortex. bioRxiv 2020.09.26.314831. https://doi.org/10.1101/2020.09.26.314831

Joshi, S., Li, Y., Kalwani, R.M., Gold, J.I., 2016. Relationships between Pupil Diameter and Neuronal Activity in the Locus Coeruleus, Colliculi, and Cingulate Cortex. Neuron 89, 221–234. https://doi.org/10.1016/j.neuron.2015.11.028

Jouvet, M., 1969. Biogenic amines and the states of sleep. Science 163, 32–41. https://doi.org/10.1126/science.163.3862.32

Kable, J.W., Glimcher, P.W., 2007. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10, 1625–1633. https://doi.org/10.1038/nn2007

Kahneman, D., 1973. Attention and effort. Prentice-Hall, Englewood Cliffs, N.J.

Kahneman, D., Beatty, J., 1966. Pupil diameter and load on memory. Science 154, 1583–1585. https://doi.org/10.1126/science.154.3756.1583

Kahneman, D., Frederick, S., 2005. A Model of Heuristic Judgment, in: Holyoak, K., Morrison, B. (Eds.), The Cambridge Handbook of Thinking and Reasoning. Cambridge University Press, pp. 267–293.

Kahneman, D., Tversky, A., 1979. Prospect Theory: An Analysis of Decision under Risk. Econometrica 47, 263–291. https://doi.org/10.2307/1914185

Kalwani, R.M., Joshi, S., Gold, J.I., 2014. Phasic Activation of Individual Neurons in the Locus Ceruleus/Subceruleus Complex of Monkeys Reflects Rewarded Decisions to Go But Not Stop. J. Neurosci. 34, 13656–13669. https://doi.org/10.1523/JNEUROSCI.2566-14.2014

Kamp, S.-M., Donchin, E., 2015. ERP and pupil responses to deviance in an oddball paradigm. Psychophysiology 52, 460–471. https://doi.org/10.1111/psyp.12378

Kane, G.A., Vazey, E.M., Wilson, R.C., Shenhav, A., Daw, N.D., Aston-Jones, G., Cohen, J.D., 2017. Increased locus coeruleus tonic activity causes

References

282

disengagement from a patch-foraging task. Cogn. Affect. Behav. Neurosci. 17, 1073–1083. https://doi.org/10.3758/s13415-017-0531-y

Kawahara, H., Kawahara, Y., Westerink, B.H., 2001. The noradrenaline-dopamine interaction in the rat medial prefrontal cortex studied by multi-probe microdialysis. Eur. J. Pharmacol. 418, 177–186. https://doi.org/10.1016/s0014-2999(01)00863-9

Kebschull, J.M., da Silva, P.G., Reid, A.P., Peikon, I.D., Albeanu, D.F., Zador, A.M., 2016. High-throughput mapping of single neuron projections by sequencing of barcoded RNA. Neuron 91, 975–987. https://doi.org/10.1016/j.neuron.2016.07.036

Kempadoo, K.A., Mosharov, E.V., Choi, S.J., Sulzer, D., Kandel, E.R., 2016. Dopamine release from the locus coeruleus to the dorsal hippocampus promotes spatial learning and memory. Proc. Natl. Acad. Sci. 113, 14835–14840. https://doi.org/10.1073/pnas.1616515114

Kennedy, D.O., Scholey, A.B., 2000. Glucose administration, heart rate and cognitive performance: effects of increasing mental effort. Psychopharmacology (Berl.) 149, 63–71. https://doi.org/10.1007/s002139900335

Kennerley, S.W., Behrens, T.E.J., Wallis, J.D., 2011. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581–1589. https://doi.org/10.1038/nn.2961

Kennerley, S.W., Dahmubed, A.F., Lara, A.H., Wallis, J.D., 2009. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178. https://doi.org/10.1162/jocn.2009.21100

Kennerley, S.W., Wallis, J.D., 2009a. Reward-dependent modulation of working memory in lateral prefrontal cortex. J. Neurosci. Off. J. Soc. Neurosci. 29, 3259–3270. https://doi.org/10.1523/JNEUROSCI.5353-08.2009

Kennerley, S.W., Wallis, J.D., 2009b. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29, 2061–2073. https://doi.org/10.1111/j.1460-9568.2009.06743.x

Kirby, K.N., MarakoviĆ, N.N., 1996. Delay-discounting probabilistic rewards: Rates decrease as amounts increase. Psychon. Bull. Rev. 3, 100–104. https://doi.org/10.3758/BF03210748

Kirchner, W.K., 1958. Age differences in short-term retention of rapidly changing information. J. Exp. Psychol. 55, 352–358. https://doi.org/10.1037/h0043688

Kirino, E., Belger, A., Goldman-Rakic, P., McCarthy, G., 2000. Prefrontal activation evoked by infrequent target and novel stimuli in a visual target detection task: an event-related functional magnetic resonance imaging study. J. Neurosci. Off. J. Soc. Neurosci. 20, 6612–6618.

Kleinsorge, T., Rinkenauer, G., 2012. Effects of monetary incentives on task switching. Exp. Psychol. 59, 216–226. https://doi.org/10.1027/1618-3169/a000146

Klimesch, W., 2011. Evoked alpha and early access to the knowledge system: The P1 inhibition timing hypothesis. Brain Res. 1408, 52–71. https://doi.org/10.1016/j.brainres.2011.06.003

Koga, K., Yamada, A., Song, Q., Li, X.-H., Chen, Q.-Y., Liu, R.-H., Ge, J., Zhan, C., Furue, H., Zhuo, M., Chen, T., 2020. Ascending noradrenergic excitation from the locus coeruleus to the anterior cingulate cortex. Mol. Brain 13, 49. https://doi.org/10.1186/s13041-020-00586-5

Kolling, N., Behrens, T., Wittmann, M., Rushworth, M., 2016. Multiple signals in anterior cingulate cortex. Curr. Opin. Neurobiol. 37, 36–43. https://doi.org/10.1016/j.conb.2015.12.007

Koob, G.F., 1999. Corticotropin-releasing factor, norepinephrine, and stress. Biol. Psychiatry 46, 1167–1180. https://doi.org/10.1016/s0006-3223(99)00164-x

Kool, W., Botvinick, M., 2018. Mental labour. Nat. Hum. Behav. 2, 899–908. https://doi.org/10.1038/s41562-018-0401-9

Kool, W., Botvinick, M., 2014. A labor/leisure tradeoff in cognitive control. J. Exp. Psychol. Gen. 143, 131–141. https://doi.org/10.1037/a0031048

Kool, W., McGuire, J.T., Rosen, Z.B., Botvinick, M.M., 2010. Decision making and the avoidance

References

283

of cognitive demand. J. Exp. Psychol. Gen. 139, 665–682. https://doi.org/10.1037/a0020198

Kopp, B., Steinke, A., Visalli, A., 2020. Cognitive flexibility and N2/P3 event-related brain potentials. Sci. Rep. 10, 9859. https://doi.org/10.1038/s41598-020-66781-5

Kossl, M., Vater, M., 1989. Noradrenaline enhances temporal auditory contrast and neuronal timing precision in the cochlear nucleus of the mustached bat. J. Neurosci. 9, 4169–4178. https://doi.org/10.1523/JNEUROSCI.09-12-04169.1989

Krajbich, I., Armel, C., Rangel, A., 2010. Visual fixations and the computation and comparison of value in simple choice. Nat. Neurosci. 13, 1292–1298. https://doi.org/10.1038/nn.2635

Krajbich, I., Rangel, A., 2011. Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc. Natl. Acad. Sci. 108, 13852–13857. https://doi.org/10.1073/pnas.1101328108

Krebs, R.M., Boehler, C.N., Woldorff, M.G., 2010. The influence of reward associations on conflict processing in the Stroop task. Cognition 117, 341–347. https://doi.org/10.1016/j.cognition.2010.08.018

Kurniawan, I.T., Guitart-Masip, M., Dayan, P., Dolan, R.J., 2013. Effort and Valuation in the Brain: The Effects of Anticipation and Execution. J. Neurosci. 33, 6160–6169. https://doi.org/10.1523/JNEUROSCI.4777-12.2013

Kurniawan, I.T., Seymour, B., Talmi, D., Yoshida, W., Chater, N., Dolan, R.J., 2010. Choosing to Make an Effort: The Role of Striatum in Signaling Physical Effort of a Chosen Action. J. Neurophysiol. 104, 313–321. https://doi.org/10.1152/jn.00027.2010

Kurth-Nelson, Z., Bickel, W., Redish, A.D., 2012. A theoretical account of cognitive effects in delay discounting. Eur. J. Neurosci. 35, 1052–1064. https://doi.org/10.1111/j.1460-9568.2012.08058.x

Kurzban, R., 2010. Does the Brain Consume Additional Glucose during Self-Control Tasks? Evol. Psychol. 8, 147470491000800200. https://doi.org/10.1177/147470491000800208

Kurzban, R., Duckworth, A., Kable, J.W., Myers, J., 2013. An opportunity cost model of subjective effort and task performance. Behav. Brain Sci. 36. https://doi.org/10.1017/S0140525X12003196

La Camera, G., Bouret, S., Richmond, B.J., 2018. Contributions of Lateral and Orbital Frontal Regions to Abstract Rule Acquisition and Reversal in Monkeys. Front. Neurosci. 12, 165. https://doi.org/10.3389/fnins.2018.00165

Lacey, J., 1967. Somatic response patterning and stress : some revisions of activation theory. Psychol. Stress Issues Res. 14–37.

Lapiz, M.D.S., Bondi, C.O., Morilak, D.A., 2007. Chronic treatment with desipramine improves cognitive performance of rats in an attentional set-shifting test. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 32, 1000–1010. https://doi.org/10.1038/sj.npp.1301235

Lapiz, M.D.S., Morilak, D.A., 2006. Noradrenergic modulation of cognitive function in rat medial prefrontal cortex as measured by attentional set shifting capability. Neuroscience 137, 1039–1049. https://doi.org/10.1016/j.neuroscience.2005.09.031

Larsen, T.S., Rasmussen, P., Overgaard, M., Secher, N.H., Nielsen, H.B., 2008. Non-selective β-adrenergic blockade prevents reduction of the cerebral metabolic ratio during exhaustive exercise in humans. J. Physiol. 586, 2807–2815. https://doi.org/10.1113/jphysiol.2008.151449

Le Bouc, R., Rigoux, L., Schmidt, L., Degos, B., Welter, M.-L., Vidailhet, M., Daunizeau, J., Pessiglione, M., 2016. Computational Dissection of Dopamine Motor and Motivational Functions in Humans. J. Neurosci. 36, 6623–6633. https://doi.org/10.1523/JNEUROSCI.3078-15.2016

Lecas, J.-C., 2004. Locus coeruleus activation shortens synaptic drive while decreasing spike latency and jitter in sensorimotor cortex. Implications for neuronal integration. Eur. J. Neurosci. 19, 2519–2530. https://doi.org/10.1111/j.0953-816X.2004.03341.x

LeDoux, J.E., 2000. Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184.

References

284

https://doi.org/10.1146/annurev.neuro.23.1.155

Lee, C.-M., Chang, W.-C., Chang, K.-B., Shyu, B.-C., 2007. Synaptic organization and input-specific short-term plasticity in anterior cingulate cortical neurons with intact thalamic inputs. Eur. J. Neurosci. 25, 2847–2861. https://doi.org/10.1111/j.1460-9568.2007.05485.x

Lemon, N., Aydin-Abidin, S., Funke, K., Manahan-Vaughan, D., 2009. Locus Coeruleus Activation Facilitates Memory Encoding and Induces Hippocampal LTD that Depends on β-Adrenergic Receptor Activation. Cereb. Cortex N. Y. NY 19, 2827–2837. https://doi.org/10.1093/cercor/bhp065

Lempert, K.M., Chen, Y.L., Fleming, S.M., 2015. Relating Pupil Dilation and Metacognitive Confidence during Auditory Decision-Making. PloS One 10, e0126588. https://doi.org/10.1371/journal.pone.0126588

Léna, C., de Kerchove D’Exaerde, A., Cordero-Erausquin, M., Le Novère, N., del Mar Arroyo-Jimenez, M., Changeux, J.P., 1999. Diversity and distribution of nicotinic acetylcholine receptors in the locus ceruleus neurons. Proc. Natl. Acad. Sci. U. S. A. 96, 12126–12131. https://doi.org/10.1073/pnas.96.21.12126

Leotti, L.A., Wager, T.D., 2010. Motivational influences on response inhibition measures. J. Exp. Psychol. Hum. Percept. Perform. 36, 430–447. https://doi.org/10.1037/a0016802

Lestienne, R., 2001. Spike timing, synchronization and information processing on the sensory side of the central nervous system. Prog. Neurobiol. 65, 545–591. https://doi.org/10.1016/s0301-0082(01)00019-3

Levitt, P., Moore, R.Y., 1978. Noradrenaline neuron innervation of the neocortex in the rat. Brain Res. 139, 219–231. https://doi.org/10.1016/0006-8993(78)90925-3

Lewis, D.A., Morrison, J.H., 1989. Noradrenergic innervation of monkey prefrontal cortex: a dopamine-beta-hydroxylase immunohistochemical study. J. Comp. Neurol. 282, 317–330. https://doi.org/10.1002/cne.902820302

Li, B.-M., Mao, Z.-M., Wang, M., Mei, Z.-T., 1999. Alpha-2 Adrenergic Modulation of Prefrontal Cortical Neuronal Activity Related to Spatial Working Memory in Monkeys. Neuropsychopharmacology 21, 601–610. https://doi.org/10.1016/S0893-133X(99)00070-6

Li, B.M., Mei, Z.T., 1994. Delayed-response deficit induced by local injection of the alpha 2-adrenergic antagonist yohimbine into the dorsolateral prefrontal cortex in young adult monkeys. Behav. Neural Biol. 62, 134–139. https://doi.org/10.1016/s0163-1047(05)80034-2

Libby, R., Lipe, M.G., 1992. Incentives, Effort, and the Cognitive Processes Involved in Accounting-Related Judgments. J. Account. Res. 30, 249–273. https://doi.org/10.2307/2491126

Lieder, F., Griffiths, T.L., 2017. Strategy selection as rational metareasoning. Psychol. Rev. 124, 762–794. https://doi.org/10.1037/rev0000075

Lieder, F., Griffiths, T.L., Goodman, N.D., 2012. Burn-in, bias, and the rationality of anchoring. Presented at the NIPS.

Lieder, F., Plunkett, D., Hamrick, J.B., Russell, S.J., Hay, N.J., Griffiths, T.L., 2014. Algorithm selection by rational metareasoning as a model of human strategy selection, in: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14. MIT Press, Cambridge, MA, USA, pp. 2870–2878.

Likhtik, E., Johansen, J.P., 2019. Neuromodulation in circuits of aversive emotional learning. Nat. Neurosci. 22, 1586–1597. https://doi.org/10.1038/s41593-019-0503-3

Liu, G., Feng, D., Wang, J., Zhang, H., Peng, Z., Cai, M., Yang, J., Zhang, R., Wang, H., Wu, S., Tan, Q., 2018. rTMS Ameliorates PTSD Symptoms in Rats by Enhancing Glutamate Transmission and Synaptic Plasticity in the ACC via the PTEN/Akt Signalling Pathway. Mol. Neurobiol. 55, 3946–3958. https://doi.org/10.1007/s12035-017-0602-7

Liu, K.Y., Marijatta, F., Hämmerer, D., Acosta-Cabronero, J., Düzel, E., Howard, R.J., 2017. Magnetic resonance imaging of the human locus coeruleus: A systematic review. Neurosci. Biobehav. Rev. 83, 325–355.

References

285

https://doi.org/10.1016/j.neubiorev.2017.10.023

Lodish, H., Berk, A., Zipursky, S.L., Matsudaira, P., Baltimore, D., Darnell, J., 2000. Chemical Equilibrium. Mol. Cell Biol. 4th Ed.

Logan, G.D., 1985. Skill and automaticity: Relations, implications, and future directions. Can. J. Psychol. Can. Psychol. 39, 367–386. https://doi.org/10.1037/h0080066

LoTemplio, S., Silcox, J., Federmeier, K.D., Payne, B.R., 2021. Inter- and intra-individual coupling between pupillary, electrophysiological, and behavioral responses in a visual oddball task. Psychophysiology 58, e13758. https://doi.org/10.1111/psyp.13758

Loughlin, S.E., Foote, S.L., Grzanna, R., 1986. Efferent projections of nucleus locus coeruleus: Morphologic subpopulations have different efferent targets. Neuroscience 18, 307–319. https://doi.org/10.1016/0306-4522(86)90156-9

Luck, S.J., Vogel, E.K., 1997. The capacity of visual working memory for features and conjunctions. Nature 390, 279–281. https://doi.org/10.1038/36846

Luppi, P.-H., Aston-Jones, G., Akaoka, H., Chouvet, G., Jouvet, M., 1995. Afferent projections to the rat locus coeruleus demonstrated by retrograde and anterograde tracing with cholera-toxin B subunit and Phaseolus vulgaris leucoagglutinin. Neuroscience 65, 119–160. https://doi.org/10.1016/0306-4522(94)00481-J

Luu, P., Tucker, D.M., 2001. Regulating action: alternating activation of midline frontal and motor cortical networks. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 112, 1295–1306. https://doi.org/10.1016/s1388-2457(01)00559-4

Ma, C.-L., Qi, X.-L., Peng, J.-Y., Li, B.-M., 2003. Selective deficit in no-go performance induced by blockade of prefrontal cortical alpha 2-adrenoceptors in monkeys. Neuroreport 14, 1013–1016. https://doi.org/10.1097/01.wnr.0000070831.57864.7b

Madden, G.J., Begotka, A.M., Raiff, B.R., Kastern, L.L., 2003. Delay discounting of real and hypothetical rewards. Exp. Clin.

Psychopharmacol. 11, 139–145. https://doi.org/10.1037/1064-1297.11.2.139

Magistretti, P.J., 1988. Regulation of glycogenolysis by neurotransmitters in the central nervous system. Diabete Metab. 14, 237–246.

Magistretti, P.J., Morrison, J.H., 1988. Noradrenaline- and vasoactive intestinal peptide-containing neuronal systems in neocortex: functional convergence with contrasting morphology. Neuroscience 24, 367–378. https://doi.org/10.1016/0306-4522(88)90338-7

Magistretti, P.J., Morrison, J.H., Shoemaker, W.J., Sapin, V., Bloom, F.E., 1981. Vasoactive intestinal polypeptide induces glycogenolysis in mouse cortical slices: a possible regulatory mechanism for the local control of energy metabolism. Proc. Natl. Acad. Sci. U. S. A. 78, 6535–6539. https://doi.org/10.1073/pnas.78.10.6535

Maity, S., Chandanathil, M., Millis, R.M., Connor, S.A., 2020. Norepinephrine stabilizes translation-dependent, homosynaptic long-term potentiation through mechanisms requiring the cAMP sensor Epac, mTOR and MAPK. Eur. J. Neurosci. 52, 3679–3688. https://doi.org/10.1111/ejn.14735

Manalis, R.S., 1977. Voltage-dependent effect of curare at the frog neuromuscular junction. Nature 267, 366–368. https://doi.org/10.1038/267366a0

Manes, F., Sahakian, B., Clark, L., Rogers, R., Antoun, N., Aitken, M., Robbins, T., 2002. Decision-making processes following damage to the prefrontal cortex. Brain J. Neurol. 125, 624–639. https://doi.org/10.1093/brain/awf049

Mäntylä, T., Still, J., Gullberg, S., Del Missier, F., 2012. Decision Making in Adults With ADHD. J. Atten. Disord. 16, 164–173. https://doi.org/10.1177/1087054709360494

Manunta, Y., Edeline, J.-M., 2004. Noradrenergic induction of selective plasticity in the frequency tuning of auditory cortex neurons. J. Neurophysiol. 92, 1445–1463. https://doi.org/10.1152/jn.00079.2004

Mao, Z.-M., Arnsten, A.F.T., Li, B.-M., 1999. Local infusion of an α-1 adrenergic agonist into the prefrontal cortex impairs spatial working

References

286

memory performance in monkeys. Biol. Psychiatry 46, 1259–1265. https://doi.org/10.1016/S0006-3223(99)00139-0

Marcora, S., 2009. Perception of effort during exercise is independent of afferent feedback from skeletal muscles, heart, and lungs. J. Appl. Physiol. 106, 2060–2062. https://doi.org/10.1152/japplphysiol.90378.2008

Mars, R.B., Coles, M.G.H., Grol, M.J., Holroyd, C.B., Nieuwenhuis, S., Hulstijn, W., Toni, I., 2005. Neural dynamics of error processing in medial frontal cortex. NeuroImage 28, 1007–1013. https://doi.org/10.1016/j.neuroimage.2005.06.041

Martin, K., Meeusen, R., Thompson, K.G., Keegan, R., Rattray, B., 2018. Mental Fatigue Impairs Endurance Performance: A Physiological Explanation. Sports Med. 48, 2041–2051. https://doi.org/10.1007/s40279-018-0946-9

Martínez-Horta, S., Riba, J., de Bobadilla, R.F., Pagonabarraga, J., Pascual-Sedano, B., Antonijoan, R.M., Romero, S., Mañanas, M.À., García-Sanchez, C., Kulisevsky, J., 2014. Apathy in Parkinson’s Disease: Neurophysiological Evidence of Impaired Incentive Processing. J. Neurosci. 34, 5918–5926.

Mather, M., Clewett, D., Sakaki, M., Harley, C.W., 2016. Norepinephrine ignites local hotspots of neuronal excitation: How arousal amplifies selectivity in perception and memory. Behav. Brain Sci. 39, e200. https://doi.org/10.1017/S0140525X15000667

Matsui, T., Ishikawa, T., Ito, H., Okamoto, M., Inoue, K., Lee, M., Fujikawa, T., Ichitani, Y., Kawanaka, K., Soya, H., 2012. Brain glycogen supercompensation following exhaustive exercise. J. Physiol. 590, 607–616. https://doi.org/10.1113/jphysiol.2011.217919

Matsui, T., Omuro, H., Liu, Y.-F., Soya, M., Shima, T., McEwen, B.S., Soya, H., 2017. Astrocytic glycogen-derived lactate fuels the brain during exhaustive exercise to maintain endurance capacity. Proc. Natl. Acad. Sci. U. S. A. 114, 6358–6363. https://doi.org/10.1073/pnas.1702739114

Matsui, T., Soya, M., Soya, H., 2019. Endurance and Brain Glycogen: A Clue Toward Understanding Central Fatigue. Adv. Neurobiol.

23, 331–346. https://doi.org/10.1007/978-3-030-27480-1_11

Matsui, T., Soya, S., Okamoto, M., Ichitani, Y., Kawanaka, K., Soya, H., 2011. Brain glycogen decreases during prolonged exercise. J. Physiol. 589, 3383–3393. https://doi.org/10.1113/jphysiol.2011.203570

Matsumoto, N., Minamimoto, T., Graybiel, A.M., Kimura, M., 2001. Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. J. Neurophysiol. 85, 960–976. https://doi.org/10.1152/jn.2001.85.2.960

Mayr, U., Keele, S.W., 2000. Changing internal constraints on action: The role of backward inhibition. J. Exp. Psychol. Gen. 129, 4–26. https://doi.org/10.1037/0096-3445.129.1.4

McBride, R.L., Sutin, J., 1976. Projections of the locus coeruleus and adjacent pontine tegmentum in the cat. J. Comp. Neurol. 165, 265–284. https://doi.org/10.1002/cne.901650302

McCall, J.G., Al-Hasani, R., Siuda, E.R., Hong, D.Y., Norris, A.J., Ford, C.P., Bruchas, M.R., 2015. CRH Engagement of the Locus Coeruleus Noradrenergic System Mediates Stress-Induced Anxiety. Neuron 87, 605–620. https://doi.org/10.1016/j.neuron.2015.07.002

McCarthy, G., Luby, M., Gore, J., Goldman-Rakic, P., 1997. Infrequent events transiently activate human prefrontal and parietal cortex as measured by functional MRI. J. Neurophysiol. 77, 1630–1634. https://doi.org/10.1152/jn.1997.77.3.1630

McCarthy, G., Wood, C.C., Williamson, P.D., Spencer, D.D., 1989. Task-dependent field potentials in human hippocampal formation. J. Neurosci. 9, 4253–4268. https://doi.org/10.1523/JNEUROSCI.09-12-04253.1989

McCloskey, D.I., 1981. Centrally-generated commands and cardiovascular control in man. Clin. Exp. Hypertens. 3, 369–378. https://doi.org/10.3109/10641968109033671

McCormick, D.A., 1989. Cholinergic and noradrenergic modulation of thalamocortical processing. Trends Neurosci. 12, 215–221. https://doi.org/10.1016/0166-2236(89)90125-2

References

287

McGaugh, J.L., 2004. The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annu. Rev. Neurosci. 27, 1–28. https://doi.org/10.1146/annurev.neuro.27.070203.144157

McGAUGHY, J., ROSS, R.S., EICHENBAUM, H., 2008. NORADRENERGIC, BUT NOT CHOLINERGIC, DEAFFERENTATION OF PREFRONTAL CORTEX IMPAIRS ATTENTIONAL SET-SHIFTING. Neuroscience 153, 63–71. https://doi.org/10.1016/j.neuroscience.2008.01.064

McGuire, J.T., Botvinick, M.M., 2010. Prefrontal cortex, cognitive control, and the registration of decision costs. Proc. Natl. Acad. Sci. 107, 7922–7926. https://doi.org/10.1073/pnas.0910662107

McKerchar, T.L., Green, L., Myerson, J., Pickford, T.S., Hill, J.C., Stout, S.C., 2009. A comparison of four models of delay discounting in humans. Behav. Processes, Proceedings of the meeting of the Society for the Quantitative Analyses of Behavior (SQAB 2008), which took place in Chicago IL, May 22-24 of 2008 81, 256–259. https://doi.org/10.1016/j.beproc.2008.12.017

McMorris, T., 2020. Cognitive Fatigue Effects on Physical Performance: The Role of Interoception. Sports Med. Auckl. NZ 50, 1703–1708. https://doi.org/10.1007/s40279-020-01320-w

McNamara, C.G., Dupret, D., 2017. Two sources of dopamine for the hippocampus. Trends Neurosci. 40, 383–384. https://doi.org/10.1016/j.tins.2017.05.005

McNay, E.C., Fries, T.M., Gold, P.E., 2000. Decreases in rat extracellular hippocampal glucose concentration associated with cognitive demand during a spatial task. Proc. Natl. Acad. Sci. U. S. A. 97, 2881–2885. https://doi.org/10.1073/pnas.050583697

McNay, E.C., McCarty, R.C., Gold, P.E., 2001. Fluctuations in Brain Glucose Concentration during Behavioral Testing: Dissociations between Brain Areas and between Brain and Blood. Neurobiol. Learn. Mem. 75, 325–337. https://doi.org/10.1006/nlme.2000.3976

Meer, E.V.D., Beyer, R., Horn, J., Foth, M., Bornemann, B., Ries, J., Kramer, J., Warmuth, E., Heekeren, H.R., Wartenburger, I., 2010. Resource allocation and fluid intelligence: Insights from

pupillometry. Psychophysiology 47, 158–169. https://doi.org/10.1111/j.1469-8986.2009.00884.x

Meeusen, R., Duclos, M., Foster, C., Fry, A., Gleeson, M., Nieman, D., Raglin, J., Rietjens, G., Steinacker, J., Urhausen, A., European College of Sport Science, American College of Sports Medicine, 2013. Prevention, diagnosis, and treatment of the overtraining syndrome: joint consensus statement of the European College of Sport Science and the American College of Sports Medicine. Med. Sci. Sports Exerc. 45, 186–205. https://doi.org/10.1249/MSS.0b013e318279a10a

Miller, G.A., 1956. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63, 81–97. https://doi.org/10.1037/h0043158

Miller, H.C., Pattison, K.F., DeWall, C.N., Rayburn-Reeves, R., Zentall, T.R., 2010. Self-Control Without a “Self”?: Common Self-Control Processes in Humans and Dogs. Psychol. Sci. 21, 534–538. https://doi.org/10.1177/0956797610364968

Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., Rangel, A., 2010. The Drift Diffusion Model Can Account for the Accuracy and Reaction Time of Value-Based Choices Under High and Low Time Pressure. SSRN Electron. J. https://doi.org/10.2139/ssrn.1901533

Minamimoto, T., Hori, Y., Kimura, M., 2009a. Roles of the thalamic CM–PF complex—Basal ganglia circuit in externally driven rebias of action. Brain Res. Bull. 78, 75–79. https://doi.org/10.1016/j.brainresbull.2008.08.013

Minamimoto, T., Hori, Y., Kimura, M., 2005. Complementary process to response bias in the centromedian nucleus of the thalamus. Science 308, 1798–1801. https://doi.org/10.1126/science.1109154

Minamimoto, T., Hori, Y., Richmond, B.J., 2012. Is Working More Costly than Waiting in Monkeys? PLOS ONE 7, e48434. https://doi.org/10.1371/journal.pone.0048434

Minamimoto, T., Kimura, M., 2002. Participation of the thalamic CM-Pf complex in attentional orienting. J. Neurophysiol. 87, 3090–3101. https://doi.org/10.1152/jn.2002.87.6.3090

References

288

Minamimoto, T., La Camera, G., Richmond, B.J., 2009b. Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys. J. Neurophysiol. 101, 437–447. https://doi.org/10.1152/jn.90959.2008

Missier, F.D., Mäntylä, T., Bruin, W.B. de, 2012. Decision-making Competence, Executive Functioning, and General Cognitive Abilities. J. Behav. Decis. Mak. 25, 331–351. https://doi.org/10.1002/bdm.731

Missier, F.D., Mäntylä, T., Bruin, W.B. de, 2010. Executive functions in decision making: An individual differences approach. Think. Reason. 16, 69–97. https://doi.org/10.1080/13546781003630117

Miyake, A., Friedman, N.P., Emerson, M.J., Witzki, A.H., Howerter, A., Wager, T.D., 2000. The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: a latent variable analysis. Cognit. Psychol. 41, 49–100. https://doi.org/10.1006/cogp.1999.0734

Mobbs, D., Hassabis, D., Seymour, B., Marchant, J.L., Weiskopf, N., Dolan, R.J., Frith, C.D., 2009. Choking on the money: reward-based performance decrements are associated with midbrain activity. Psychol. Sci. 20, 955–962. https://doi.org/10.1111/j.1467-9280.2009.02399.x

Molden, D.C., Hui, C.M., Scholer, A.A., Meier, B.P., Noreen, E.E., D’Agostino, P.R., Martin, V., 2012. Motivational Versus Metabolic Effects of Carbohydrates on Self-Control. Psychol. Sci. 23, 1137–1144. https://doi.org/10.1177/0956797612439069

Monsell, S., 1996. Control of mental processes, in: Unsolved Mysteries of the Mind: Tutorial Essays in Cognition. Erlbaum (Uk) Taylor & Francis, Publ, Oxford, England, pp. 93–148.

Monsell, S., Sumner, P., Waters, H., 2003. Task-set reconfiguration with predictable and unpredictable task switches. Mem. Cognit. 31, 327–342. https://doi.org/10.3758/bf03194391

Morris, R., Bornstein, A., Shenhav, A. (Eds.), 2019. Goal-directed decision making: computations and neural circuits. Academic Press, Elsevier, London San Diego, CA Cambridge.

Morrison, J.H., Foote, S.L., 1986. Noradrenergic and serotoninergic innervation of cortical, thalamic, and tectal visual structures in Old and New World monkeys. J. Comp. Neurol. 243, 117–138. https://doi.org/10.1002/cne.902430110

Morrison, J.H., Grzanna, R., Molliver, M.E., Coyle, J.T., 1978. The distribution and orientation of noradrenergic fibers in neocortex of the rat: An immunofluorescence study. J. Comp. Neurol. 181, 17–39. https://doi.org/10.1002/cne.901810103

Mulder, G., 1986. The Concept and Measurement of Mental Effort, in: Hockey, G.R.J., Gaillard, A.W.K., Coles, M.G.H. (Eds.), Energetics and Human Information Processing, NATO ASI Series. Springer Netherlands, Dordrecht, pp. 175–198. https://doi.org/10.1007/978-94-009-4448-0_12

Mulder, G., 1979. Sinusarrhythmia and Mental Work Load, in: Moray, N. (Ed.), Mental Workload: Its Theory and Measurement, NATO Conference Series. Springer US, Boston, MA, pp. 327–343. https://doi.org/10.1007/978-1-4757-0884-4_19

Mulder, G., Mulder, L.J.M., 1981. Information Processing and Cardiovascular Control. Psychophysiology 18, 392–402. https://doi.org/10.1111/j.1469-8986.1981.tb02470.x

Müller, M.S., 2014. Functional impact of glycogen degradation on astrocytic signalling. Biochem. Soc. Trans. 42, 1311–1315. https://doi.org/10.1042/BST20140157

Muraven, M., Baumeister, R.F., 2000. Self-regulation and depletion of limited resources: Does self-control resemble a muscle? Psychol. Bull. 126, 247–259. https://doi.org/10.1037/0033-2909.126.2.247

Muraven, M., Slessareva, E., 2003. Mechanisms of self-control failure: motivation and limited resources. Pers. Soc. Psychol. Bull. 29, 894–906. https://doi.org/10.1177/0146167203029007008

Muraven, M., Tice, D.M., Baumeister, R.F., 1998. Self-control as a limited resource: Regulatory depletion patterns. J. Pers. Soc. Psychol. 74, 774–789. https://doi.org/10.1037/0022-3514.74.3.774

Murphy, P.R., Moort, M.L. van, Nieuwenhuis, S., 2016. The Pupillary Orienting Response Predicts Adaptive Behavioral Adjustment after Errors.

References

289

PLOS ONE 11, e0151763. https://doi.org/10.1371/journal.pone.0151763

Murphy, P.R., O’Connell, R.G., O’Sullivan, M., Robertson, I.H., Balsters, J.H., 2014. Pupil diameter covaries with BOLD activity in human locus coeruleus: Pupil Diameter and Locus Coeruleus Activity. Hum. Brain Mapp. 35, 4140–4154. https://doi.org/10.1002/hbm.22466

Murphy, P.R., Robertson, I.H., Balsters, J.H., O’connell, R.G., 2011. Pupillometry and P3 index the locus coeruleus–noradrenergic arousal function in humans. Psychophysiology 48, 1532–1543. https://doi.org/10.1111/j.1469-8986.2011.01226.x

Naaijen, J., Lythgoe, D.J., Zwiers, M.P., Hartman, C.A., Hoekstra, P.J., Buitelaar, J.K., Aarts, E., 2018. Anterior cingulate cortex glutamate and its association with striatal functioning during cognitive control. Eur. Neuropsychopharmacol. 28, 381–391. https://doi.org/10.1016/j.euroneuro.2018.01.002

Nagai, T., Satoh, K., Imamoto, K., Maeda, T., 1981. Divergent projections of catecholamine neurons of the locus coeruleus as revealed by fluorescent retrograde double labeling technique. Neurosci. Lett. 23, 117–123. https://doi.org/10.1016/0304-3940(81)90027-6

Nardecchia, F., Orlando, R., Iacovelli, L., Colamartino, M., Fiori, E., Leuzzi, V., Piccinin, S., Nistico, R., Puglisi-Allegra, S., Di Menna, L., Battaglia, G., Nicoletti, F., Pascucci, T., 2018. Targeting mGlu5 Metabotropic Glutamate Receptors in the Treatment of Cognitive Dysfunction in a Mouse Model of Phenylketonuria. Front. Neurosci. 12. https://doi.org/10.3389/fnins.2018.00154

Nassar, M.R., Rumsey, K.M., Wilson, R.C., Parikh, K., Heasly, B., Gold, J.I., 2012. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15, 1040–1046. https://doi.org/10.1038/nn.3130

Navarra, R., Graf, R., Huang, Y., Logue, S., Comery, T., Hughes, Z., Day, M., 2008. Effects of atomoxetine and methylphenidate on attention and impulsivity in the 5-choice serial reaction time test. Prog. Neuropsychopharmacol. Biol. Psychiatry 32, 34–41. https://doi.org/10.1016/j.pnpbp.2007.06.017

Navon, D., 1984. Resources--a theoretical soup stone? Psychol. Rev. 91, 216–234. https://doi.org/10.1037/0033-295X.91.2.216

Nelson, S.R., Schulz, D.W., Passonneau, J.V., Lowry, O.H., 1968. Control of glycogen levels in brain. J. Neurochem. 15, 1271–1279. https://doi.org/10.1111/j.1471-4159.1968.tb05904.x

Neuman, R.S., Harley, C.W., 1983. Long-lasting potentiation of the dentate gyrus population spike by norepinephrine. Brain Res. 273, 162–165. https://doi.org/10.1016/0006-8993(83)91106-x

Newell, A., Simon, H.A., 1972. Human problem solving. Prentice-Hall, Englewood Cliffs, N.J.

Newman, L.A., Darling, J., McGaughy, J., 2008. Atomoxetine reverses attentional deficits produced by noradrenergic deafferentation of medial prefrontal cortex. Psychopharmacology (Berl.) 200, 39–50. https://doi.org/10.1007/s00213-008-1097-8

Nicholas, A.P., Pieribone, V., Hökfelt, T., 1993. Distributions of mRNAs for alpha-2 adrenergic receptor subtypes in rat brain: an in situ hybridization study. J. Comp. Neurol. 328, 575–594. https://doi.org/10.1002/cne.903280409

Nicholson, W., Snyder, C., 2012. Microeconomic theory: basic principles and extensions, 11th ed. Cengage Learning.

Nickel, P., Nachreiner, F., 2003. Sensitivity and Diagnosticity of the 0.1-Hz Component of Heart Rate Variability as an Indicator of Mental Workload. Hum. Factors J. Hum. Factors Ergon. Soc. 45, 575–590. https://doi.org/10.1518/hfes.45.4.575.27094

Nidal, K., Malik, A.S., 2014. EEG/ERP Analysis: Methods and Applications. CRC Press.

Nieuwenhuis, S., 2011. Learning, the P3, and the Locus Coeruleus-Norepinephrine System, Neural Basis of Motivational and Cognitive Control. The MIT Press.

Nieuwenhuis, S., Aston-Jones, G., Cohen, J.D., 2005. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychol. Bull. 131, 510–532. https://doi.org/10.1037/0033-2909.131.4.510

References

290

Nieuwenhuis, S., de Geus, E.J., Aston-Jones, G., 2011. The anatomical and functional relationship between the P3 and autonomic components of the orienting response. Psychophysiology 48, 162–175. https://doi.org/10.1111/j.1469-8986.2010.01057.x

Nieuwenhuis, S., Monsell, S., 2002. Residual costs in task switching: testing the failure-to-engage hypothesis. Psychon. Bull. Rev. 9, 86–92. https://doi.org/10.3758/bf03196259

Nieuwenhuys, R., Voogd, J., Huijzen, C. van, 2008. The Human Central Nervous System: A Synopsis and Atlas, 4th ed. Steinkopff-Verlag Heidelberg. https://doi.org/10.1007/978-3-540-34686-9

Nir, Y., Mukamel, R., Dinstein, I., Privman, E., Harel, M., Fisch, L., Gelbard-Sagiv, H., Kipervasser, S., Andelman, F., Neufeld, M.Y., Kramer, U., Arieli, A., Fried, I., Malach, R., 2008. Interhemispheric correlations of slow spontaneous neuronal fluctuations revealed in human sensory cortex. Nat. Neurosci. 11, 1100–1108. https://doi.org/10.1038/nn.2177

Norman, D.A., Bobrow, D.G., 1975. On data-limited and resource-limited processes. Cognit. Psychol. 7, 44–64. https://doi.org/10.1016/0010-0285(75)90004-3

Obel, L.F., Müller, M.S., Walls, A.B., Sickmann, H.M., Bak, L.K., Waagepetersen, H.S., Schousboe, A., 2012. Brain glycogen-new perspectives on its metabolic function and regulation at the subcellular level. Front. Neuroenergetics 4, 3. https://doi.org/10.3389/fnene.2012.00003

O’Connor, T.A., Starr, A., 1985. Intracranial potentials correlated with an event-related potential, P300, in the cat. Brain Res. 339, 27–38. https://doi.org/10.1016/0006-8993(85)90618-3

O’Dowd, B.S., Barrington, J., Ng, K.T., Hertz, E., Hertz, L., 1995. Glycogenolytic response of primary chick and mouse cultures of astrocytes to noradrenaline across development. Brain Res. Dev. Brain Res. 88, 220–223. https://doi.org/10.1016/0165-3806(95)00084-q

Olivers, C.N.L., Peters, J., Houtkamp, R., Roelfsema, P.R., 2011. Different states in visual working memory: when it guides attention and when it does not. Trends Cogn. Sci. 15, 327–334. https://doi.org/10.1016/j.tics.2011.05.004

Olson, V.G., Rockett, H.R., Reh, R.K., Redila, V.A., Tran, P.M., Venkov, H.A., Defino, M.C., Hague, C., Peskind, E.R., Szot, P., Raskind, M.A., 2011. The role of norepinephrine in differential response to stress in an animal model of posttraumatic stress disorder. Biol. Psychiatry 70, 441–448. https://doi.org/10.1016/j.biopsych.2010.11.029

Olszewski, J., Baxter, D., 1954. Cytoarchitecture of the human brainstem. By Jerzy Olszewski and Donald Baxter. Published and distributed in North America for S. Karger by J. B. Lippincott Company, Philadelphia and Montreal. 1954. 199 pages. Price $16.00 (Reviewed by Gerhardt von Bonin). J. Comp. Neurol. 101, 825–825. https://doi.org/10.1002/cne.901010308

O’Reilly, J.X., Schüffelgen, U., Cuell, S.F., Behrens, T.E.J., Mars, R.B., Rushworth, M.F.S., 2013. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl. Acad. Sci. U. S. A. 110, E3660–E3669. https://doi.org/10.1073/pnas.1305373110

O’Reilly, R.C., Frank, M.J., 2006. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283–328. https://doi.org/10.1162/089976606775093909

Overtoom, C.C.E., Verbaten, M.N., Kemner, C., Kenemans, J.L., van Engeland, H., Buitelaar, J.K., van der Molen, M.W., van der Gugten, J., Westenberg, H., Maes, R. a. A., Koelega, H.S., 2003. Effects of methylphenidate, desipramine, and L-dopa on attention and inhibition in children with Attention Deficit Hyperactivity Disorder. Behav. Brain Res. 145, 7–15. https://doi.org/10.1016/s0166-4328(03)00097-4

Owen, L., Scholey, A.B., Finnegan, Y., Hu, H., Sünram-Lea, S.I., 2012. The effect of glucose dose and fasting interval on cognitive function: a double-blind, placebo-controlled, six-way crossover study. Psychopharmacology (Berl.) 220, 577–589. https://doi.org/10.1007/s00213-011-2510-2

Padmala, S., Pessoa, L., 2011. Reward reduces conflict by enhancing attentional control and biasing visual cortical processing. J. Cogn. Neurosci. 23, 3419–3432. https://doi.org/10.1162/jocn_a_00011

Padmala, S., Pessoa, L., 2010. Interactions between cognition and motivation during response inhibition. Neuropsychologia 48, 558–

References

291

565. https://doi.org/10.1016/j.neuropsychologia.2009.10.017

Page, M.E., Abercrombie, E.D., 1999. Discrete local application of corticotropin-releasing factor increases locus coeruleus discharge and extracellular norepinephrine in rat hippocampus. Synap. N. Y. N 33, 304–313. https://doi.org/10.1002/(SICI)1098-2396(19990915)33:4<304::AID-SYN7>3.0.CO;2-Z

Pageaux, B., Lepers, R., Dietz, K.C., Marcora, S.M., 2014. Response inhibition impairs subsequent self-paced endurance performance. Eur. J. Appl. Physiol. 114, 1095–1105. https://doi.org/10.1007/s00421-014-2838-5

Pattij, T., Vanderschuren, L.J.M.J., 2008. The neuropharmacology of impulsive behaviour. Trends Pharmacol. Sci. 29, 192–199. https://doi.org/10.1016/j.tips.2008.01.002

Payne, J.W., Bettman, J.R., Johnson, E.J., 1988. Adaptive strategy selection in decision making. J. Exp. Psychol. Learn. Mem. Cogn. 14, 534–552. https://doi.org/10.1037/0278-7393.14.3.534

Payzan-LeNestour, E., Dunne, S., Bossaerts, P., O’Doherty, J.P., 2013. The Neural Representation of Unexpected Uncertainty during Value-Based Decision Making. Neuron 79, 191–201. https://doi.org/10.1016/j.neuron.2013.04.037

Pellerin, L., Magistretti, P.J., 1994. Glutamate uptake into astrocytes stimulates aerobic glycolysis: a mechanism coupling neuronal activity to glucose utilization. Proc. Natl. Acad. Sci. U. S. A. 91, 10625–10629. https://doi.org/10.1073/pnas.91.22.10625

Pennington, B.F., Ozonoff, S., 1996. Executive functions and developmental psychopathology. Child Psychol. Psychiatry Allied Discip. 37, 51–87. https://doi.org/10.1111/j.1469-7610.1996.tb01380.x

Perez, D.M., 2020. α1-Adrenergic Receptors in Neurotransmission, Synaptic Plasticity, and Cognition. Front. Pharmacol. 11, 581098. https://doi.org/10.3389/fphar.2020.581098

Pessiglione, M., Vinckier, F., Bouret, S., Daunizeau, J., Le Bouc, R., 2018. Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases. Brain 141, 629–650. https://doi.org/10.1093/brain/awx278

Peters, J., Büchel, C., 2009. Overlapping and Distinct Neural Systems Code for Subjective Value during Intertemporal and Risky Decision Making. J. Neurosci. 29, 15727–15734.

Petersen, S.E., Posner, M.I., 2012. The Attention System of the Human Brain: 20 Years After. Annu. Rev. Neurosci. 35, 73–89. https://doi.org/10.1146/annurev-neuro-062111-150525

Pettigrew, J.D., 1982. Pharmacologic control of cortical plasticity. Retina Phila. Pa 2, 360–372.

Pettigrew, J.D., Kasamatsu, T., 1978. Local perfusion of noradrenaline maintains visual cortical plasticity. Nature 271, 761–763. https://doi.org/10.1038/271761a0

Phelps, C.H., 1972. Barbiturate-induced glycogen accumulation in brain. An electron microscopic study. Brain Res. 39, 225–234. https://doi.org/10.1016/0006-8993(72)90797-4

Philiastides, M.G., Ratcliff, R., 2013. Influence of Branding on Preference-Based Decision Making. Psychol. Sci. 24, 1208–1215. https://doi.org/10.1177/0956797612470701

Phillips, M.A., Szabadi, E., Bradshaw, C.M., 2000. Comparison of the effects of clonidine and yohimbine on spontaneous pupillary fluctuations in healthy human volunteers. Psychopharmacology (Berl.) 150, 85–89. https://doi.org/10.1007/s002130000398

Phillips, P.E.M., Walton, M.E., Jhou, T.C., 2007. Calculating utility: preclinical evidence for cost-benefit analysis by mesolimbic dopamine. Psychopharmacology (Berl.) 191, 483–495. https://doi.org/10.1007/s00213-006-0626-6

Pineda, J.A., Foote, S.L., Neville, H.J., 1989. Effects of locus coeruleus lesions on auditory, long-latency, event-related potentials in monkey. J. Neurosci. Off. J. Soc. Neurosci. 9, 81–93.

Plassmann, H., O’Doherty, J.P., Rangel, A., 2010. Appetitive and Aversive Goal Values Are Encoded in the Medial Orbitofrontal Cortex at the Time of Decision Making. J. Neurosci. 30, 10799–10808.

Plewnia, C., Hoppe, J., Cohen, L.G., Gerloff, C., 2004. Improved motor skill acquisition after selective stimulation of central norepinephrine. Neurology 62, 2124–2126.

References

292

https://doi.org/10.1212/01.WNL.0000128041.92710.17

Polich, J., 2007. Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 118, 2128–2148. https://doi.org/10.1016/j.clinph.2007.04.019

Polyn, S.M., Norman, K.A., Kahana, M.J., 2009. A context maintenance and retrieval model of organizational processes in free recall. Psychol. Rev. 116, 129–156. https://doi.org/10.1037/a0014420

Porat, O., Hassin-Baer, S., Cohen, O.S., Markus, A., Tomer, R., 2014. Asymmetric dopamine loss differentially affects effort to maximize gain or minimize loss. Cortex 51, 82–91. https://doi.org/10.1016/j.cortex.2013.10.004

Posner, M.I., Snyder, C.R.R., 2004. Attention and Cognitive Control, Cognitive psychology: Key readings. Psychology Press, New York, NY, US.

Pratt, N., Willoughby, A., Swick, D., 2011. Effects of working memory load on visual selective attention: behavioral and electrophysiological evidence. Front. Hum. Neurosci. 5, 57. https://doi.org/10.3389/fnhum.2011.00057

Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., Dreher, J.-C., 2010. Separate Valuation Subsystems for Delay and Effort Decision Costs. J. Neurosci. 30, 14080–14090. https://doi.org/10.1523/JNEUROSCI.2752-10.2010

Proske, U., 2005. What is the role of muscle receptors in proprioception? Muscle Nerve 31, 780–787. https://doi.org/10.1002/mus.20330

Puumala, T., Greijus, S., Narinen, K., Haapalinna, A., Riekkinen, P., Sirviö, J., 1998. Stimulation of alpha-1 adrenergic receptors facilitates spatial learning in rats. Eur. Neuropsychopharmacol. J. Eur. Coll. Neuropsychopharmacol. 8, 17–26. https://doi.org/10.1016/s0924-977x(97)00040-0

Qi, S., Zeng, Q., Luo, Y., Duan, H., Ding, C., Hu, W., Li, H., 2014. Impact of working memory load on cognitive control in trait anxiety: an ERP study. PloS One 9, e111791. https://doi.org/10.1371/journal.pone.0111791

Quach, T.T., Rose, C., Schwartz, J.C., 1978. [3H]Glycogen hydrolysis in brain slices: responses to neurotransmitters and modulation of

noradrenaline receptors. J. Neurochem. 30, 1335–1341. https://doi.org/10.1111/j.1471-4159.1978.tb10464.x

Quilodran, R., Rothé, M., Procyk, E., 2008. Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex. Neuron 57, 314–325. https://doi.org/10.1016/j.neuron.2007.11.031

Rabbitt, P., 1979. How old and young subjects monitor and control responses for accuracy and speed. Br. J. Psychol. 70, 305–311. https://doi.org/10.1111/j.2044-8295.1979.tb01687.x

Rachlin, H., Jones, B.A., 2008. Social discounting and delay discounting. J. Behav. Decis. Mak. 21, 29–43. https://doi.org/10.1002/bdm.567

Radel, R., Gruet, M., Barzykowski, K., 2019. Testing the ego-depletion effect in optimized conditions. PloS One 14, e0213026. https://doi.org/10.1371/journal.pone.0213026

Rajkowski, J., Kubiak, P., Aston-Jones, G., 1994. Locus coeruleus activity in monkey: Phasic and tonic changes are associated with altered vigilance. Brain Res. Bull. 35, 607–616. https://doi.org/10.1016/0361-9230(94)90175-9

Rajkowski, J., Majczynski, H., Clayton, E., Aston-Jones, G., 2004. Activation of Monkey Locus Coeruleus Neurons Varies With Difficulty and Performance in a Target Detection Task. J. Neurophysiol. 92, 361–371. https://doi.org/10.1152/jn.00673.2003

Rämä, P., Linnankoski, I., Tanila, H., Pertovaara, A., Carlson, S., 1996. Medetomidine, atipamezole, and guanfacine in delayed response performance of aged monkeys. Pharmacol. Biochem. Behav. 55, 415–422. https://doi.org/10.1016/s0091-3057(96)00111-6

Raskind, M.A., Peterson, K., Williams, T., Hoff, D.J., Hart, K., Holmes, H., Homas, D., Hill, J., Daniels, C., Calohan, J., Millard, S.P., Rohde, K., O’Connell, J., Pritzl, D., Feiszli, K., Petrie, E.C., Gross, C., Mayer, C.L., Freed, M.C., Engel, C., Peskind, E.R., 2013. A trial of prazosin for combat trauma PTSD with nightmares in active-duty soldiers returned from Iraq and Afghanistan. Am. J. Psychiatry 170, 1003–1010. https://doi.org/10.1176/appi.ajp.2013.12081133

References

293

Ratcliff, R., McKoon, G., 2008. The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput. 20, 873–922. https://doi.org/10.1162/neco.2008.12-06-420

Rauch, S.L., Whalen, P.J., Shin, L.M., McInerney, S.C., Macklin, M.L., Lasko, N.B., Orr, S.P., Pitman, R.K., 2000. Exaggerated amygdala response to masked facial stimuli in posttraumatic stress disorder: a functional MRI study. Biol. Psychiatry 47, 769–776. https://doi.org/10.1016/s0006-3223(00)00828-3

Renart, A., Machens, C.K., 2014. Variability in neural activity and behavior. Curr. Opin. Neurobiol., Theoretical and computational neuroscience 25, 211–220. https://doi.org/10.1016/j.conb.2014.02.013

Reyes, B. a. S., Drolet, G., Van Bockstaele, E.J., 2008. Dynorphin and stress-related peptides in rat locus coeruleus: contribution of amygdalar efferents. J. Comp. Neurol. 508, 663–675. https://doi.org/10.1002/cne.21683

Reynaud, A.J., Froesel, M., Guedj, C., Ben Hadj Hassen, S., Cléry, J., Meunier, M., Ben Hamed, S., Hadj-Bouziane, F., 2019. Atomoxetine improves attentional orienting in a predictive context. Neuropharmacology 150, 59–69. https://doi.org/10.1016/j.neuropharm.2019.03.012

Ridderinkhof, K.R., Ullsperger, M., Crone, E.A., Nieuwenhuis, S., 2004. The Role of the Medial Frontal Cortex in Cognitive Control. Science 306, 443–447. https://doi.org/10.1126/science.1100301

Robbins, T.W., 1984. Cortical noradrenaline, attention and arousal. Psychol. Med. 14, 13–21. https://doi.org/10.1017/s0033291700003032

Robbins, T.W., Arnsten, A.F.T., 2009. The Neuropsychopharmacology of Fronto-Executive Function: Monoaminergic Modulation. Annu. Rev. Neurosci. 32, 267–287. https://doi.org/10.1146/annurev.neuro.051508.135535

Robinson, E.S.J., 2012. Blockade of noradrenaline re-uptake sites improves accuracy and impulse control in rats performing a five-choice serial reaction time tasks. Psychopharmacology (Berl.) 219, 303–312. https://doi.org/10.1007/s00213-011-2420-3

Robinson, E.S.J., Eagle, D.M., Mar, A.C., Bari, A., Banerjee, G., Jiang, X., Dalley, J.W., Robbins, T.W., 2008. Similar effects of the selective noradrenaline reuptake inhibitor atomoxetine on three distinct forms of impulsivity in the rat. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 33, 1028–1037. https://doi.org/10.1038/sj.npp.1301487

Rogawski, M.A., Aghajanian, G.K., 1980. Modulation of lateral geniculate neurone excitability by noradrenaline microiontophoresis or locus coeruleus stimulation. Nature 287, 731–734. https://doi.org/10.1038/287731a0

Room, P., Postema, F., Korf, J., 1981. Divergent axon collaterals of rat locus coeruleus neurons: demonstration by a fluorescent double labeling technique. Brain Res. 221, 219–230. https://doi.org/10.1016/0006-8993(81)90773-3

Roullet, P., Sara, S., 1998. Consolidation of memory after its reactivation: involvement of beta noradrenergic receptors in the late phase. Neural Plast. 6, 63–68. https://doi.org/10.1155/NP.1998.63

Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M., Rushworth, M.F.S., 2006. Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161–1168. https://doi.org/10.1038/nn1756

Rushworth, M.F.S., Walton, M.E., Kennerley, S.W., Bannerman, D.M., 2004. Action sets and decisions in the medial frontal cortex. Trends Cogn. Sci. 8, 410–417. https://doi.org/10.1016/j.tics.2004.07.009

Rydval, O., Ortmann, A., 2004. How financial incentives and cognitive abilities affect task performance in laboratory settings: An illustration. Econ. Lett. 85, 315–320. https://doi.org/10.1016/j.econlet.2004.04.020

Salamone, J.D., Correa, M., Farrar, A., Mingote, S.M., 2007. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology (Berl.) 191, 461–482. https://doi.org/10.1007/s00213-006-0668-9

Salgado, H., Köhr, G., Treviño, M., 2012. Noradrenergic ‘Tone’ Determines Dichotomous Control of Cortical Spike-Timing-Dependent Plasticity. Sci. Rep. 2. https://doi.org/10.1038/srep00417

References

294

Samanez-Larkin, G.R., Hollon, N.G., Carstensen, L.L., Knutson, B., 2008. Individual Differences in Insular Sensitivity During Loss Anticipation Predict Avoidance Learning. Psychol. Sci. 19, 320–323. https://doi.org/10.1111/j.1467-9280.2008.02087.x

Samuels, J.A., Whitecotton, S.M., 2011. An effort based analysis of the paradoxical effects of incentives on decision-aided performance. J. Behav. Decis. Mak. 24, 345–360. https://doi.org/10.1002/bdm.691

Sanders, A.F., 1979. Some Remarks on Mental Load, in: Moray, N. (Ed.), Mental Workload: Its Theory and Measurement, NATO Conference Series. Springer US, Boston, MA, pp. 41–77. https://doi.org/10.1007/978-1-4757-0884-4_5

San-Galli, A., Varazzani, C., Abitbol, R., Pessiglione, M., Bouret, S., 2018. Primate Ventromedial Prefrontal Cortex Neurons Continuously Encode the Willingness to Engage in Reward-Directed Behavior. Cereb. Cortex 28, 73–89. https://doi.org/10.1093/cercor/bhw351

Saper, C.B., Scammell, T.E., Lu, J., 2005. Hypothalamic regulation of sleep and circadian rhythms. Nature 437, 1257–1263. https://doi.org/10.1038/nature04284

Sara, S.J., 2009. The locus coeruleus and noradrenergic modulation of cognition. Nat. Rev. Neurosci. 10, 211–223. https://doi.org/10.1038/nrn2573

Sara, S.J., 1985. Noradrenergic Modulation of Selective Attention: Its Role in Memory Retrieval. Ann. N. Y. Acad. Sci. 444, 178–193. https://doi.org/10.1111/j.1749-6632.1985.tb37588.x

Sara, S.J., Bouret, S., 2012. Orienting and reorienting: the locus coeruleus mediates cognition through arousal. Neuron 76, 130–141. https://doi.org/10.1016/j.neuron.2012.09.011

Sasse, L.K., Peters, J., Brassen, S., 2017. Cognitive Control Modulates Effects of Episodic Simulation on Delay Discounting in Aging. Front. Aging Neurosci. 9. https://doi.org/10.3389/fnagi.2017.00058

Schiphof-Godart, L., Roelands, B., Hettinga, F.J., 2018. Drive in Sports: How Mental Fatigue Affects Endurance Performance. Front. Psychol. 9. https://doi.org/10.3389/fpsyg.2018.01383

Schmeichel, B.J., 2007. Attention control, memory updating, and emotion regulation temporarily reduce the capacity for executive control. J. Exp. Psychol. Gen. 136, 241–255. https://doi.org/10.1037/0096-3445.136.2.241

Schouppe, N., Demanet, J., Boehler, C.N., Ridderinkhof, K.R., Notebaert, W., 2014. The role of the striatum in effort-based decision-making in the absence of reward. J. Neurosci. Off. J. Soc. Neurosci. 34, 2148–2154. https://doi.org/10.1523/JNEUROSCI.1214-13.2014

Schwarz, L.A., Luo, L., 2015. Organization of the Locus Coeruleus-Norepinephrine System. Curr. Biol. 25, R1051–R1056. https://doi.org/10.1016/j.cub.2015.09.039

Schwarz, L.A., Miyamichi, K., Gao, X.J., Beier, K.T., Weissbourd, B., DeLoach, K.E., Ren, J., Ibanes, S., Malenka, R.C., Kremer, E.J., Luo, L., 2015. Viral-genetic tracing of the input-output organization of a central noradrenaline circuit. Nature 524, 88–92. https://doi.org/10.1038/nature14600

Schweimer, J., Saft, S., Hauber, W., 2005. Involvement of catecholamine neurotransmission in the rat anterior cingulate in effort-related decision making. Behav. Neurosci. 119, 1687–1692. https://doi.org/10.1037/0735-7044.119.6.1687

Secher, N.H., Seifert, T., Van Lieshout, J.J., 2008. Cerebral blood flow and metabolism during exercise: implications for fatigue. J. Appl. Physiol. 104, 306–314. https://doi.org/10.1152/japplphysiol.00853.2007

Seeley, W.W., Menon, V., Schatzberg, A.F., Keller, J., Glover, G.H., Kenna, H., Reiss, A.L., Greicius, M.D., 2007. Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control. J. Neurosci. 27, 2349–2356. https://doi.org/10.1523/JNEUROSCI.5587-06.2007

Segal, D.S., Mandell, A.J., 1970. Behavioral activation of rats during intraventricular infusion of norepinephrine. Proc. Natl. Acad. Sci. U. S. A. 66, 289–293. https://doi.org/10.1073/pnas.66.2.289

Segal, M., Bloom, F.E., 1976. The action of norepinephrine in the rat hippocampus. IV. The effects of locus coeruleus stimulation on evoked

References

295

hippocampal unit activity. Brain Res. 107, 513–525. https://doi.org/10.1016/0006-8993(76)90141-4

Segal, M., Bloom, F.E., 1974. The action of norepinephrine in the rat hippocampus. I. Iontophoretic studies. Brain Res. 72, 79–97. https://doi.org/10.1016/0006-8993(74)90652-0

Sellaro, R., van Leusden, J.W.R., Tona, K.-D., Verkuil, B., Nieuwenhuis, S., Colzato, L.S., 2015. Transcutaneous Vagus Nerve Stimulation Enhances Post-error Slowing. J. Cogn. Neurosci. 27, 2126–2132. https://doi.org/10.1162/jocn_a_00851

Sergeant, J.A., Geurts, H., Oosterlaan, J., 2002. How specific is a deficit of executive functioning for attention-deficit/hyperactivity disorder? Behav. Brain Res. 130, 3–28. https://doi.org/10.1016/s0166-4328(01)00430-2

Servan-Schreiber, D., Printz, H., Cohen, J.D., 1990. A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. Science 249, 892–895. https://doi.org/10.1126/science.2392679

Shamosh, N.A., DeYoung, C.G., Green, A.E., Reis, D.L., Johnson, M.R., Conway, A.R.A., Engle, R.W., Braver, T.S., Gray, J.R., 2008. Individual Differences in Delay Discounting: Relation to Intelligence, Working Memory, and Anterior Prefrontal Cortex. Psychol. Sci. 19, 904–911. https://doi.org/10.1111/j.1467-9280.2008.02175.x

Shamosh, N.A., Gray, J.R., 2008. Delay discounting and intelligence: A meta-analysis. Intelligence 36, 289–305. https://doi.org/10.1016/j.intell.2007.09.004

Sharma, Y., Xu, T., Graf, W.M., Fobbs, A., Sherwood, C.C., Hof, P.R., Allman, J.M., Manaye, K.F., 2010. Comparative Anatomy of the Locus Coeruleus in Humans and Non-Human Primates. J. Comp. Neurol. 518, 963. https://doi.org/10.1002/cne.22249

Shea-Brown, E., Gilzenrat, M.S., Cohen, J.D., 2008. Optimization of decision making in multilayer networks: the role of locus coeruleus. Neural Comput. 20, 2863–2894. https://doi.org/10.1162/neco.2008.03-07-487

Shekhar, A., Sajdyk, T.S., Keim, S.R., Yoder, K.K., Sanders, S.K., 1999. Role of the basolateral

amygdala in panic disorder. Ann. N. Y. Acad. Sci. 877, 747–750. https://doi.org/10.1111/j.1749-6632.1999.tb09315.x

Shenhav, A., Botvinick, M.M., Cohen, J.D., 2013. The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function. Neuron 79, 217–240. https://doi.org/10.1016/j.neuron.2013.07.007

Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T.L., Cohen, J.D., Botvinick, M.M., 2017. Toward a Rational and Mechanistic Account of Mental Effort. Annu. Rev. Neurosci. 40, 99–124. https://doi.org/10.1146/annurev-neuro-072116-031526

Shenhav, A., Straccia, M.A., Cohen, J.D., Botvinick, M.M., 2014. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254. https://doi.org/10.1038/nn.3771

Shidara, M., Mizuhiki, T., Richmond, B.J., 2005. Neuronal firing in anterior cingulate neurons changes modes across trials in single states of multitrial reward schedules. Exp. Brain Res. 163, 242–245. https://doi.org/10.1007/s00221-005-2232-y

Shiffrin, R.M., Schneider, W., 1977. Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. Psychol. Rev. 84, 127–190. https://doi.org/10.1037/0033-295X.84.2.127

Shiv, B., Fedorikhin, A., 1999. Heart and mind in conflict: The interplay of affect and cognition in consumer decision making. J. Consum. Res. 26, 278–292. https://doi.org/10.1086/209563

Shugan, S.M., 1980. The Cost Of Thinking. J. Consum. Res. 7, 99–111. https://doi.org/10.1086/208799

Shulman, R.G., Hyder, F., Rothman, D.L., 2001. Cerebral energetics and the glycogen shunt: neurochemical basis of functional imaging. Proc. Natl. Acad. Sci. U. S. A. 98, 6417–6422. https://doi.org/10.1073/pnas.101129298

Shyu, B.-C., Vogt, B.A., 2009. Short-term synaptic plasticity in the nociceptive thalamic-anterior cingulate pathway. Mol. Pain 5, 51. https://doi.org/10.1186/1744-8069-5-51

References

296

Sidarus, N., Palminteri, S., Chambon, V., 2019. Cost-benefit trade-offs in decision-making and learning. PLOS Comput. Biol. 15, e1007326. https://doi.org/10.1371/journal.pcbi.1007326

Silvetti, M., Vassena, E., Abrahamse, E., Verguts, T., 2018. Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner. PLOS Comput. Biol. 14, e1006370. https://doi.org/10.1371/journal.pcbi.1006370

Simon, H.A., 1997. Models of Bounded Rationality: Empirically grounded economic reason. MIT Press.

Simon, H.A., 1956. Rational choice and the structure of the environment. Psychol. Rev. 63, 129–138. https://doi.org/10.1037/h0042769

Simon, H.A., 1955. A behavioral model of rational choice. Q. J. Econ. 69, 99–118. https://doi.org/10.2307/1884852

Simpson, H.M., Hale, S.M., 1969. Pupillary changes during a decision-making task. Percept. Mot. Skills 29, 495–498. https://doi.org/10.2466/pms.1969.29.2.495

Simpson, K.L., Waterhouse, B.D., Lin, R.C., 1999. Origin, distribution, and morphology of galaninergic fibers in the rodent trigeminal system. J. Comp. Neurol. 411, 524–534. https://doi.org/10.1002/(sici)1096-9861(19990830)411:3<524::aid-cne13>3.0.co;2-x

Sims, J., Carroll, D., 1990. Cardiovascular and Metabolic Activity at Rest and During Psychological and Physical Challenge in Normotensives and Subjects With Mildly Elevated Blood Pressure. Psychophysiology 27, 149–156. https://doi.org/10.1111/j.1469-8986.1990.tb00366.x

Sims, J., Carroll, D., Turner, J.R., Hewitt, J.K., 1988. Cardiac and Metabolic Activity in Mild Hypertensive and Normotensive Subjects. Psychophysiology 25, 172–178. https://doi.org/10.1111/j.1469-8986.1988.tb00982.x

Sirviö, J., Riekkinen, P., Vajanto, I., Koivisto, E., Riekkinen, P.J., 1991. The effects of guanfacine, alpha-2 agonist, on the performance of young and aged rats in spatial navigation task. Behav. Neural Biol. 56, 101–107. https://doi.org/10.1016/0163-1047(91)90327-m

Skvortsova, V., Palminteri, S., Pessiglione, M., 2014. Learning To Minimize Efforts versus Maximizing Rewards: Computational Principles and Neural Correlates. J. Neurosci. 34, 15621–15630. https://doi.org/10.1523/JNEUROSCI.1350-14.2014

Sloman, S.A., 1996. The empirical case for two systems of reasoning. Psychol. Bull. 119, 3–22. https://doi.org/10.1037/0033-2909.119.1.3

Smit, P.J., 1978. Sports medicine and “overtraining.” South Afr. Med. J. Suid-Afr. Tydskr. Vir Geneeskd. 54, 4.

Smith, C.C., Greene, R.W., 2012. CNS Dopamine Transmission Mediated by Noradrenergic Innervation. J. Neurosci. 32, 6072–6080. https://doi.org/10.1523/JNEUROSCI.6486-11.2012

Smith, S.A., Querry, R.G., Fadel, P.J., Gallagher, K.M., Strømstad, M., Ide, K., Raven, P.B., Secher, N.H., 2003. Partial blockade of skeletal muscle somatosensory afferents attenuates baroreflex resetting during exercise in humans. J. Physiol. 551, 1013–1021. https://doi.org/10.1113/jphysiol.2003.044925

Smith, V.L., Walker, J.M., 1993. Monetary Rewards and Decision Cost in Experimental Economics. Econ. Inq. 31, 245–261. https://doi.org/10.1111/j.1465-7295.1993.tb00881.x

Sofuoglu, M., Herman, A.I., Robinson, C., Waters, A.J., 2014. Chapter Twelve - Cognitive Effects of Nicotine, in: Madras, B., Kuhar, M. (Eds.), The Effects of Drug Abuse on the Human Nervous System. Academic Press, Boston, pp. 367–385. https://doi.org/10.1016/B978-0-12-418679-8.00012-5

Soltani, M., Knight, R.T., 2000. Neural origins of the P300. Crit. Rev. Neurobiol. 14, 199–224.

Sommer, M.A., 2003. The role of the thalamus in motor control. Curr. Opin. Neurobiol. 13, 663–670. https://doi.org/10.1016/j.conb.2003.10.014

Sonneborn, A., Greene, R.W., 2021. Norepinephrine transporter antagonism prevents dopamine-dependent synaptic plasticity in the mouse dorsal hippocampus.

References

297

Neurosci. Lett. 740, 135450. https://doi.org/10.1016/j.neulet.2020.135450

Sorg, O., Magistretti, P.J., 1992. Vasoactive intestinal peptide and noradrenaline exert long-term control on glycogen levels in astrocytes: blockade by protein synthesis inhibition. J. Neurosci. Off. J. Soc. Neurosci. 12, 4923–4931.

Sorg, O., Magistretti, P.J., 1991. Characterization of the glycogenolysis elicited by vasoactive intestinal peptide, noradrenaline and adenosine in primary cultures of mouse cerebral cortical astrocytes. Brain Res. 563, 227–233. https://doi.org/10.1016/0006-8993(91)91538-c

Stanovich, K.E., West, R.F., 2000. Individual differences in reasoning: Implications for the rationality debate? Behav. Brain Sci. 23, 645–665. https://doi.org/10.1017/S0140525X00003435

Stanton, P.K., Sarvey, J.M., 1985. Depletion of norepinephrine, but not serotonin, reduces long-term potentiation in the dentate gyrus of rat hippocampal slices. J. Neurosci. 5, 2169–2176. https://doi.org/10.1523/JNEUROSCI.05-08-02169.1985

Stefani, M.R., Groth, K., Moghaddam, B., 2003. Glutamate receptors in the rat medial prefrontal cortex regulate set-shifting ability. Behav. Neurosci. 117, 728–737. https://doi.org/10.1037/0735-7044.117.4.728

Stefani, M.R., Moghaddam, B., 2005. Systemic and Prefrontal Cortical NMDA Receptor Blockade Differentially Affect Discrimination Learning and Set-Shift Ability in Rats. Behav. Neurosci. 119, 420–428. https://doi.org/10.1037/0735-7044.119.2.420

Steindler, D.A., 1981. Locus coeruleus neurons have axons that branch to the forebrain and cerebellum. Brain Res. 223, 367–373. https://doi.org/10.1016/0006-8993(81)91149-5

Stephens, D.W., Krebs, J.R., 1986. Foraging theory, Monographs in behavior and ecology. Princeton University Press, Princeton, N.J.

Stevens, J.R., Rosati, A.G., Ross, K.R., Hauser, M.D., 2005. Will travel for food: spatial discounting in two new world monkeys. Curr. Biol. CB 15, 1855–1860. https://doi.org/10.1016/j.cub.2005.09.016

Stone, E.A., Zhang, Y., Hiller, J.M., Simon, E.J., Hillman, D.E., 1997. Activation of fos in mouse amygdala by local infusion of norepinephrine or atipamezole. Brain Res. 778, 1–5. https://doi.org/10.1016/S0006-8993(97)00667-7

Stone, E.A., Zhang, Y., John, S.M., Bing, G., 1991. C-Fos response to administration of catecholamines into brain by microdialysis. Neurosci. Lett. 133, 33–35. https://doi.org/10.1016/0304-3940(91)90050-4

Straube, T., Korz, V., Balschun, D., Frey, J.U., 2003. Requirement of beta-adrenergic receptor activation and protein synthesis for LTP-reinforcement by novelty in rat dentate gyrus. J. Physiol. 552, 953–960. https://doi.org/10.1113/jphysiol.2003.049452

Stroop, J.R., 1935. Studies of interference in serial verbal reactions. J. Exp. Psychol. 18, 643–662. https://doi.org/10.1037/h0054651

Sullivan, R., Wilson, D., Leon, M., 1989. Norepinephrine and learning-induced plasticity in infant rat olfactory system. J. Neurosci. 9, 3998–4006. https://doi.org/10.1523/JNEUROSCI.09-11-03998.1989

Sutin, E.L., Jacobowitz, D.M., 1991. Neurochemicals in the dorsal pontine tegmentum. Prog. Brain Res. 88, 3–14. https://doi.org/10.1016/s0079-6123(08)63796-6

Sved, A.F., Cano, G., Passerin, A.M., Rabin, B.S., 2002. The locus coeruleus, Barrington’s nucleus, and neural circuits of stress. Physiol. Behav. 77, 737–742. https://doi.org/10.1016/s0031-9384(02)00927-7

Swanson, L.W., 1976. The locus coeruleus: a cytoarchitectonic, Golgi and immunohistochemical study in the albino rat. Brain Res. 110, 39–56. https://doi.org/10.1016/0006-8993(76)90207-9

Swanson, R.A., 1992. Physiologic coupling of glial glycogen metabolism to neuronal activity in brain. Can. J. Physiol. Pharmacol. 70 Suppl, S138-144. https://doi.org/10.1139/y92-255

Swanson, R.A., Morton, M.M., Sagar, S.M., Sharp, F.R., 1992. Sensory stimulation induces local cerebral glycogenolysis: demonstration by

References

298

autoradiography. Neuroscience 51, 451–461. https://doi.org/10.1016/0306-4522(92)90329-z

Swick, D., Pineda, J.A., Schacher, S., Foote, S.L., 1994. Locus coeruleus neuronal activity in awake monkeys: relationship to auditory P300-like potentials and spontaneous EEG. Exp. Brain Res. 101, 86–92. https://doi.org/10.1007/BF00243219

Szabadi, E., 2013. Functional neuroanatomy of the central noradrenergic system. J. Psychopharmacol. Oxf. Engl. 27, 659–693. https://doi.org/10.1177/0269881113490326

Tait, D.S., Brown, V.J., Farovik, A., Theobald, D.E., Dalley, J.W., Robbins, T.W., 2007. Lesions of the dorsal noradrenergic bundle impair attentional set-shifting in the rat. Eur. J. Neurosci. 25, 3719–3724. https://doi.org/10.1111/j.1460-9568.2007.05612.x

Takeuchi, T., Duszkiewicz, A.J., Sonneborn, A., Spooner, P.A., Yamasaki, M., Watanabe, M., Smith, C.C., Fernández, G., Deisseroth, K., Greene, R.W., Morris, R.G.M., 2016. Locus coeruleus and dopaminergic consolidation of everyday memory. Nature 537, 357–362. https://doi.org/10.1038/nature19325

Talmi, D., Dayan, P., Kiebel, S.J., Frith, C.D., Dolan, R.J., 2009. How Humans Integrate the Prospects of Pain and Reward during Choice. J. Neurosci. 29, 14617–14626.

Tanila, H., Rämä, P., Carlson, S., 1996. The effects of prefrontal intracortical microinjections of an alpha-2 agonist, alpha-2 antagonist and lidocaine on the delayed alternation performance of aged rats. Brain Res. Bull. 40, 117–119. https://doi.org/10.1016/0361-9230(96)00026-3

Tattersall, A.J., Hockey, G.R.J., 1995. Level of operator control and changes in heart rate variability during simulated flight maintenance. Hum. Factors 37, 682–698. https://doi.org/10.1518/001872095778995517

Taylor, S.F., Welsh, R.C., Wager, T.D., Phan, K.L., Fitzgerald, K.D., Gehring, W.J., 2004. A functional neuroimaging study of motivation and executive function. NeuroImage 21, 1045–1054. https://doi.org/10.1016/j.neuroimage.2003.10.032

Tervo, D.G.R., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., Karpova, A.Y., 2014.

Behavioral Variability through Stochastic Choice and Its Gating by Anterior Cingulate Cortex. Cell 159, 21–32. https://doi.org/10.1016/j.cell.2014.08.037

Tibana, R., Manuel Frade de Sousa, N., Prestes, J., da Cunha Nascimento, D., Ernesto, C., Falk Neto, J., Kennedy, M., Azevedo Voltarelli, F., 2019. Is Perceived Exertion a Useful Indicator of the Metabolic and Cardiovascular Responses to a Metabolic Conditioning Session of Functional Fitness? Sports 7, 161. https://doi.org/10.3390/sports7070161

Tiesinga, P., Fellous, J.-M., Sejnowski, T.J., 2008. Regulation of spike timing in visual cortical circuits. Nat. Rev. Neurosci. 9, 97–107. https://doi.org/10.1038/nrn2315

Tillman, C.M., Wiens, S., 2011. Behavioral and ERP indices of response conflict in Stroop and flanker tasks. Psychophysiology 48, 1405–1411. https://doi.org/10.1111/j.1469-8986.2011.01203.x

Timberlake, W., 1994. Behavior systems, associationism, and Pavlovian conditioning. Psychon. Bull. Rev. 1, 405–420. https://doi.org/10.3758/BF03210945

Todd, M., Niv, Y., Cohen, J., 2008. Learning to Use Working Memory in Partially Observable Environments through Dopaminergic Reinforcement. Presented at the Advances in Neural Information Processing Systems, pp. 1689–1696.

Todd, P.M., Gigerenzer, G. (Eds.), 2012. Ecological rationality: intelligence in the world, Evolution and cognition series. Oxford University Press, Oxford ; New York.

Totah, N.K., Neves, R.M., Panzeri, S., Logothetis, N.K., Eschenko, O., 2018. The Locus Coeruleus Is a Complex and Differentiated Neuromodulatory System. Neuron 99, 1055-1068.e6. https://doi.org/10.1016/j.neuron.2018.07.037

Toyoda, H., Li, X.-Y., Wu, L.-J., Zhao, M.-G., Descalzi, G., Chen, T., Koga, K., Zhuo, M., 2011. Interplay of Amygdala and Cingulate Plasticity in Emotional Fear. Neural Plast. 2011, e813749. https://doi.org/10.1155/2011/813749

Travagli, R.A., Dunwiddie, T.V., Williams, J.T., 1995. Opioid Inhibition in Locus Coeruleus. J.

References

299

Neurophysiol. 74, 519–528. https://doi.org/10.1152/jn.1995.74.2.519

Treadway, M.T., Buckholtz, J.W., Schwartzman, A.N., Lambert, W.E., Zald, D.H., 2009. Worth the ‘EEfRT’? The Effort Expenditure for Rewards Task as an Objective Measure of Motivation and Anhedonia. PLOS ONE 4, e6598. https://doi.org/10.1371/journal.pone.0006598

Tsuda, K., Yokoo, H., Goldstein, M., 1989. Neuropeptide Y and galanin in norepinephrine release in hypothalamic slices. Hypertens. Dallas Tex 1979 14, 81–86. https://doi.org/10.1161/01.hyp.14.1.81

Tsukahara, J.S., Harrison, T.L., Engle, R.W., 2016. The relationship between baseline pupil size and intelligence. Cognit. Psychol. 91, 109–123. https://doi.org/10.1016/j.cogpsych.2016.10.001

Tsunematsu, S., 2001. Effort- and time-cost effects on demand curves for food by pigeons under short session closed economies. Behav. Processes 53, 47–56. https://doi.org/10.1016/S0376-6357(00)00147-9

Turetsky, B.I., Fein, G., 2002. α2-Noradrenergic effects on ERP and behavioral indices of auditory information processing. Psychophysiology 39, 147–157. https://doi.org/10.1111/1469-8986.3920147

Turner, J.R., Carroll, D., 1985. Heart Rate and Oxygen Consumption during Mental Arithmetic, a Video Game, and Graded Exercise: Further Evidence of Metabolically-Exaggerated Cardiac Adjustments? Psychophysiology 22, 261–267. https://doi.org/10.1111/j.1469-8986.1985.tb01597.x

Turner, J.R., Carroll, D., Courtney, H., 1983. Cardiac and Metabolic Responses to “Space Invaders”: An Instance of Metabolically-Exaggerated Cardiac Adjustment? Psychophysiology 20, 544–549. https://doi.org/10.1111/j.1469-8986.1983.tb03010.x

Tversky, A., Kahneman, D., 1974. Judgment under Uncertainty: Heuristics and Biases. Science 185, 1124–1131. https://doi.org/10.1126/science.185.4157.1124

Tzavara, E.T., Bymaster, F.P., Overshiner, C.D., Davis, R.J., Perry, K.W., Wolff, M., McKinzie, D.L.,

Witkin, J.M., Nomikos, G.G., 2006. Procholinergic and memory enhancing properties of the selective norepinephrine uptake inhibitor atomoxetine. Mol. Psychiatry 11, 187–195. https://doi.org/10.1038/sj.mp.4001763

Uematsu, A., Tan, B.Z., Johansen, J.P., 2015. Projection specificity in heterogeneous locus coeruleus cell populations: implications for learning and memory. Learn. Mem. 22, 444–451. https://doi.org/10.1101/lm.037283.114

Uematsu, A., Tan, B.Z., Ycu, E.A., Cuevas, J.S., Koivumaa, J., Junyent, F., Kremer, E.J., Witten, I.B., Deisseroth, K., Johansen, J.P., 2017. Modular organization of the brainstem noradrenaline system coordinates opposing learning states. Nat. Neurosci. 20, 1602–1611. https://doi.org/10.1038/nn.4642

Uhr, M.L., 1979. The influence of an enzyme on the direction of a reaction. Biochem. Educ. 7, 15–16. https://doi.org/10.1016/0307-4412(79)90014-1

Ullsperger, M., Cramon, D.Y. von, 2003. Error Monitoring Using External Feedback: Specific Roles of the Habenular Complex, the Reward System, and the Cingulate Motor Area Revealed by Functional Magnetic Resonance Imaging. J. Neurosci. 23, 4308–4314. https://doi.org/10.1523/JNEUROSCI.23-10-04308.2003

Umemoto, A., Holroyd, C.B., 2015. Task-specific effects of reward on task switching. Psychol. Res. 79, 698–707. https://doi.org/10.1007/s00426-014-0595-z

U’Prichard, D.C., Bechtel, W.D., Rouot, B.M., Snyder, S.H., 1979. Multiple apparent alpha-noradrenergic receptor binding sites in rat brain: effect of 6-hydroxydopamine. Mol. Pharmacol. 16, 47–60.

Usher, M., Cohen, J.D., Servan-Schreiber, D., Rajkowski, J., Aston-Jones, G., 1999. The Role of Locus Coeruleus in the Regulation of Cognitive Performance. Science 283, 549–554. https://doi.org/10.1126/science.283.5401.549

Usher, M., Davelaar, E.J., 2002. Neuromodulation of decision and response selection. Neural Netw. 15, 635–645. https://doi.org/10.1016/S0893-6080(02)00054-0

References

300

Valentin, V.V., Dickinson, A., O’Doherty, J.P., 2007. Determining the Neural Substrates of Goal-Directed Learning in the Human Brain. J. Neurosci. 27, 4019–4026.

Valentino, R.J., Foote, S.L., 1988. Corticotropin-releasing hormone increases tonic but not sensory-evoked activity of noradrenergic locus coeruleus neurons in unanesthetized rats. J. Neurosci. Off. J. Soc. Neurosci. 8, 1016–1025.

Valentino, R.J., Page, M.E., Curtis, A.L., 1991. Activation of noradrenergic locus coeruleus neurons by hemodynamic stress is due to local release of corticotropin-releasing factor. Brain Res. 555, 25–34. https://doi.org/10.1016/0006-8993(91)90855-p

Valentino, R.J., Van Bockstaele, E., 2008. Convergent Regulation of Locus Coeruleus Activity as an adaptive Response to Stress. Eur. J. Pharmacol. 583, 194–203. https://doi.org/10.1016/j.ejphar.2007.11.062

Van Bockstaele, E.J., Aston-Jones, G., 1995. Integration in the Ventral Medulla and Coordination of Sympathetic, Pain and Arousal Functions. Clin. Exp. Hypertens. 17, 153–165. https://doi.org/10.3109/10641969509087062

Van Bockstaele, E.J., Bajic, D., Proudfit, H., Valentino, R.J., 2001. Topographic architecture of stress-related pathways targeting the noradrenergic locus coeruleus. Physiol. Behav. 73, 273–283. https://doi.org/10.1016/s0031-9384(01)00448-6

Van Cutsem, J., Marcora, S., De Pauw, K., Bailey, S., Meeusen, R., Roelands, B., 2017. The Effects of Mental Fatigue on Physical Performance: A Systematic Review. Sports Med. 47, 1569–1588. https://doi.org/10.1007/s40279-016-0672-0

van der Wel, P., van Steenbergen, H., 2018. Pupil dilation as an index of effort in cognitive control tasks: A review. Psychon. Bull. Rev. 25, 2005–2015. https://doi.org/10.3758/s13423-018-1432-y

van Marle, H.J.F., Hermans, E.J., Qin, S., Fernández, G., 2010. Enhanced resting-state connectivity of amygdala in the immediate aftermath of acute psychological stress. NeuroImage 53, 348–354. https://doi.org/10.1016/j.neuroimage.2010.05.070

van Veen, V., Carter, C.S., 2002. The Timing of Action-Monitoring Processes in the Anterior Cingulate Cortex. J. Cogn. Neurosci. 14, 593–602. https://doi.org/10.1162/08989290260045837

van Veen, V., Cohen, J.D., Botvinick, M.M., Stenger, V.A., Carter, C.S., 2001. Anterior Cingulate Cortex, Conflict Monitoring, and Levels of Processing. NeuroImage 14, 1302–1308. https://doi.org/10.1006/nimg.2001.0923

Vankov, A., Hervé-Minvielle, A., Sara, S.J., 1995. Response to novelty and its rapid habituation in locus coeruleus neurons of the freely exploring rat. Eur. J. Neurosci. 7, 1180–1187. https://doi.org/10.1111/j.1460-9568.1995.tb01108.x

Varazzani, C., San-Galli, A., Gilardeau, S., Bouret, S., 2015. Noradrenaline and Dopamine Neurons in the Reward/Effort Trade-Off: A Direct Electrophysiological Comparison in Behaving Monkeys. J. Neurosci. 35, 7866–7877. https://doi.org/10.1523/JNEUROSCI.0454-15.2015

Vazey, E.M., Moorman, D.E., Aston-Jones, G., 2018. Phasic locus coeruleus activity regulates cortical encoding of salience information. Proc. Natl. Acad. Sci. U. S. A. 115, E9439–E9448. https://doi.org/10.1073/pnas.1803716115

Ventura, R., Latagliata, E.C., Morrone, C., La Mela, I., Puglisi-Allegra, S., 2008. Prefrontal Norepinephrine Determines Attribution of “High” Motivational Salience. PLoS ONE 3, e3044. https://doi.org/10.1371/journal.pone.0003044

Ventura, R., Morrone, C., Puglisi-Allegra, S., 2007. Prefrontal/accumbal catecholamine system determines motivational salience attribution to both reward- and aversion-related stimuli. Proc. Natl. Acad. Sci. 104, 5181–5186. https://doi.org/10.1073/pnas.0610178104

Verguts, T., Vassena, E., Silvetti, M., 2015. Adaptive effort investment in cognitive and physical tasks: a neurocomputational model. Front. Behav. Neurosci. 9. https://doi.org/10.3389/fnbeh.2015.00057

Verleger, R., Heide, W., Butt, C., Kömpf, D., 1994. Reduction of P3b in patients with temporo-parietal lesions. Brain Res. Cogn. Brain Res. 2, 103–116. https://doi.org/10.1016/0926-6410(94)90007-8

References

301

Vissing, J., Andersen, M., Diemer, N.H., 1996. Exercise-induced changes in local cerebral glucose utilization in the rat. J. Cereb. Blood Flow Metab. Off. J. Int. Soc. Cereb. Blood Flow Metab. 16, 729–736. https://doi.org/10.1097/00004647-199607000-00025

Von Neumann, J., Morgenstern, O., 2007. Theory of games and economic behavior, 60th anniversary ed. ed, Princeton classic editions. Princeton University Press, Princeton, N.J. ; Woodstock.

Von Neumann, J., Morgenstern, O., 1947. Theory of games and economic behavior, 2nd rev. ed, Theory of games and economic behavior, 2nd rev. ed. Princeton University Press, Princeton, NJ, US.

Wagatsuma, A., Okuyama, T., Sun, C., Smith, L.M., Abe, K., Tonegawa, S., 2018. Locus coeruleus input to hippocampal CA3 drives single-trial learning of a novel context. Proc. Natl. Acad. Sci. U. S. A. 115, E310–E316. https://doi.org/10.1073/pnas.1714082115

Wahn, B., Ferris, D.P., Hairston, W.D., König, P., 2016. Pupil Sizes Scale with Attentional Load and Task Experience in a Multiple Object Tracking Task. PloS One 11, e0168087. https://doi.org/10.1371/journal.pone.0168087

Walling, S.G., Harley, C.W., 2004. Locus ceruleus activation initiates delayed synaptic potentiation of perforant path input to the dentate gyrus in awake rats: a novel beta-adrenergic- and protein synthesis-dependent mammalian plasticity mechanism. J. Neurosci. Off. J. Soc. Neurosci. 24, 598–604. https://doi.org/10.1523/JNEUROSCI.4426-03.2004

Walls, A.B., Heimbürger, C.M., Bouman, S.D., Schousboe, A., Waagepetersen, H.S., 2009. Robust glycogen shunt activity in astrocytes: Effects of glutamatergic and adrenergic agents. Neuroscience 158, 284–292. https://doi.org/10.1016/j.neuroscience.2008.09.058

Walton, M.E., Bannerman, D.M., Alterescu, K., Rushworth, M.F.S., 2003. Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions. J. Neurosci. 23, 6475–6479.

https://doi.org/10.1523/JNEUROSCI.23-16-06475.2003

Walton, M.E., Bannerman, D.M., Rushworth, M.F.S., 2002. The role of rat medial frontal cortex in effort-based decision making. J. Neurosci. Off. J. Soc. Neurosci. 22, 10996–11003.

Walton, M.E., Croxson, P.L., Rushworth, M.F.S., Bannerman, D.M., 2005. The Mesocortical Dopamine Projection to Anterior Cingulate Cortex Plays No Role in Guiding Effort-Related Decisions. Behav. Neurosci. 119, 323–328. https://doi.org/10.1037/0735-7044.119.1.323

Walton, M.E., Devlin, J.T., Rushworth, M.F.S., 2004. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265. https://doi.org/10.1038/nn1339

Walton, M.E., Groves, J., Jennings, K.A., Croxson, P.L., Sharp, T., Rushworth, M.F.S., Bannerman, D.M., 2009. Comparing the role of the anterior cingulate cortex and 6-hydroxydopamine nucleus accumbens lesions on operant effort-based decision making. Eur. J. Neurosci. 29, 1678–1691. https://doi.org/10.1111/j.1460-9568.2009.06726.x

Walton, M.E., Kennerley, S.W., Bannerman, D.M., Phillips, P.E.M., Rushworth, M.F.S., 2006. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw. Off. J. Int. Neural Netw. Soc. 19, 1302–1314. https://doi.org/10.1016/j.neunet.2006.03.005

Wang, M., Ramos, B.P., Paspalas, C.D., Shu, Y., Simen, A., Duque, A., Vijayraghavan, S., Brennan, A., Dudley, A., Nou, E., Mazer, J.A., McCormick, D.A., Arnsten, A.F.T., 2007. α2A-Adrenoceptors Strengthen Working Memory Networks by Inhibiting cAMP-HCN Channel Signaling in Prefrontal Cortex. Cell 129, 397–410. https://doi.org/10.1016/j.cell.2007.03.015

Wang, X.-J., 2002. Probabilistic Decision Making by Slow Reverberation in Cortical Circuits. Neuron 36, 955–968. https://doi.org/10.1016/S0896-6273(02)01092-9

Watanabe, H., Passonneau, J.V., 1973. Factors affecting the turnover of cerebral glycogen and limit dextrin in vivo. J. Neurochem. 20, 1543–

References

302

1554. https://doi.org/10.1111/j.1471-4159.1973.tb00272.x

Waterhouse, B.D., Moises, H.C., Woodward, D.J., 1998. Phasic activation of the locus coeruleus enhances responses of primary sensory cortical neurons to peripheral receptive field stimulation. Brain Res. 790, 33–44. https://doi.org/10.1016/s0006-8993(98)00117-6

Waterhouse, B.D., Moises, H.C., Woodward, D.J., 1980. Noradrenergic modulation of somatosensory cortical neuronal responses to lontophoretically applied putative neurotransmitters. Exp. Neurol. 69, 30–49. https://doi.org/10.1016/0014-4886(80)90141-7

Waterhouse, B.D., Mouradian, R., Sessler, F.M., Lin, R.C.S., 2000. Differential modulatory effects of norepinephrine on synaptically driven responses of layer V barrel field cortical neurons. Brain Res. 868, 39–47. https://doi.org/10.1016/S0006-8993(00)02261-7

Waterhouse, B.D., Sessler, F.M., Jung-Tung, C., Woodward, D.J., Azizi, S.A., Moises, H.C., 1988. New evidence for a gating action of norepinephrine in central neuronal circuits of mammalian brain. Brain Res. Bull. 21, 425–432. https://doi.org/10.1016/0361-9230(88)90154-2

Waterhouse, B.D., Woodward, D.J., 1980. Interaction of norepinephrine with cerebrocortical activity evoked by stimulation of

somatosensory afferent pathways in the rat☆. Exp. Neurol. 67, 11–34. https://doi.org/10.1016/0014-4886(80)90159-4

Wei, H., Zhou, R., 2020. High working memory load impairs selective attention: EEG signatures. Psychophysiology 57, e13643. https://doi.org/10.1111/psyp.13643

Westbrook, A., Braver, T.S., 2015. Cognitive effort: A neuroeconomic approach. Cogn. Affect. Behav. Neurosci. 15, 395–415. https://doi.org/10.3758/s13415-015-0334-y

Westbrook, A., Kester, D., Braver, T.S., 2013. What Is the Subjective Cost of Cognitive Effort? Load, Trait, and Aging Effects Revealed by Economic Preference. PLOS ONE 8, e68210. https://doi.org/10.1371/journal.pone.0068210

Westbrook, J.A., Braver, T.S., 2013. The economics of cognitive effort. Behav. Brain Sci.

36, 704–705; discussion 707-726. https://doi.org/10.1017/S0140525X13001179

White, O., Davare, M., Andres, M., Olivier, E., 2013. The role of left supplementary motor area in grip force scaling. PloS One 8, e83812. https://doi.org/10.1371/journal.pone.0083812

Whittington, M.A., Cunningham, M.O., LeBeau, F.E.N., Racca, C., Traub, R.D., 2011. Multiple origins of the cortical gamma rhythm. Dev. Neurobiol. 71, 92–106. https://doi.org/10.1002/dneu.20814

Wickelgren, W.A., 1977. Speed-accuracy tradeoff and information processing dynamics. Acta Psychol. (Amst.) 41, 67–85. https://doi.org/10.1016/0001-6918(77)90012-9

Widge, A.S., Zorowitz, S., Basu, I., Paulk, A.C., Cash, S.S., Eskandar, E.N., Deckersbach, T., Miller, E.K., Dougherty, D.D., 2019. Deep brain stimulation of the internal capsule enhances human cognitive control and prefrontal cortex function. Nat. Commun. 10, 1536. https://doi.org/10.1038/s41467-019-09557-4

Wiecki, T.V., Frank, M.J., 2013. A computational model of inhibitory control in frontal cortex and basal ganglia. Psychol. Rev. 120, 329–355. https://doi.org/10.1037/a0031542

Williams, J.T., North, R.A., Shefner, S.A., Nishi, S., Egan, T.M., 1984. Membrane properties of rat locus coeruleus neurones. Neuroscience 13, 137–156. https://doi.org/10.1016/0306-4522(84)90265-3

Wilson, G.F., 1992. Applied use of cardiac and respiration measures: Practical considerations and precautions. Biol. Psychol. 34, 163–178. https://doi.org/10.1016/0301-0511(92)90014-L

Witte, E.A., Marrocco, R.T., 1997. Alteration of brain noradrenergic activity in rhesus monkeys affects the alerting component of covert orienting. Psychopharmacology (Berl.) 132, 315–323. https://doi.org/10.1007/s002130050351

Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W., Desimone, R., Engel, A.K., Fries, P., 2007. Modulation of neuronal interactions through neuronal synchronization. Science 316, 1609–1612. https://doi.org/10.1126/science.1139597

References

303

Worthy, D.A., Markman, A.B., Maddox, W.T., 2009. What is pressure? Evidence for social pressure as a type of regulatory focus. Psychon. Bull. Rev. 16, 344–349. https://doi.org/10.3758/PBR.16.2.344

Wright, P., 1975. Consumer Choice Strategies: Simplifying Vs. Optimizing. J. Mark. Res. 12, 60–67. https://doi.org/10.1177/002224377501200109

Wright, T.J., Boot, W.R., Morgan, C.S., 2013. Pupillary response predicts multiple object tracking load, error rate, and conscientiousness, but not inattentional blindness. Acta Psychol. (Amst.) 144, 6–11. https://doi.org/10.1016/j.actpsy.2013.04.018

Xu, Z.Q., Shi, T.J., Hökfelt, T., 1998. Galanin/GMAP- and NPY-like immunoreactivities in locus coeruleus and noradrenergic nerve terminals in the hippocampal formation and cortex with notes on the galanin-R1 and -R2 receptors. J. Comp. Neurol. 392, 227–251. https://doi.org/10.1002/(sici)1096-9861(19980309)392:2<227::aid-cne6>3.0.co;2-4

Yerkes, R.M., Dodson, J.D., 1908. The Relation of Strength of Stimulus to Rapidity of Habit Formation. J. Comp. Neurol. Psychol. 18, 459–482. https://doi.org/10.1002/cne.920180503

Yingling, C.D., Hosobuchi, Y., 1984. A subcortical correlate of P300 in man. Electroencephalogr. Clin. Neurophysiol. Evoked Potentials 59, 72–76. https://doi.org/10.1016/0168-5597(84)90022-4

Yoshida, W., Ishii, S., 2006. Resolution of uncertainty in prefrontal cortex. Neuron 50, 781–789. https://doi.org/10.1016/j.neuron.2006.05.006

Young, W.S., Kuhar, M.J., 1980. Noradrenergic alpha 1 and alpha 2 receptors: light microscopic autoradiographic localization. Proc. Natl. Acad. Sci. U. S. A. 77, 1696–1700. https://doi.org/10.1073/pnas.77.3.1696

Yu, A.J., Dayan, P., 2005. Uncertainty, Neuromodulation, and Attention. Neuron 46, 681–692. https://doi.org/10.1016/j.neuron.2005.04.026

Yu, A.J., Dayan, P., Cohen, J.D., 2009. Dynamics of attentional selection under conflict: Toward a rational Bayesian account. J. Exp. Psychol. Hum.

Percept. Perform. 35, 700–717. https://doi.org/10.1037/a0013553

Yuen, E.Y., Liu, W., Karatsoreos, I.N., Ren, Y., Feng, J., McEwen, B.S., Yan, Z., 2011. Mechanisms for acute stress-induced enhancement of glutamatergic transmission and working memory. Mol. Psychiatry 16, 156–170. https://doi.org/10.1038/mp.2010.50

Yuen, E.Y., Wei, J., Liu, W., Zhong, P., Li, X., Yan, Z., 2012. Repeated Stress Causes Cognitive Impairment by Suppressing Glutamate Receptor Expression and Function in Prefrontal Cortex. Neuron 73, 962–977. https://doi.org/10.1016/j.neuron.2011.12.033

Zénon, A., Sidibé, M., Olivier, E., 2015. Disrupting the Supplementary Motor Area Makes Physical Effort Appear Less Effortful. J. Neurosci. 35, 8737–8744. https://doi.org/10.1523/JNEUROSCI.3789-14.2015

Zénon, A., Sidibé, M., Olivier, E., 2014. Pupil size variations correlate with physical effort perception. Front. Behav. Neurosci. 8, 286. https://doi.org/10.3389/fnbeh.2014.00286

Zerbi, V., Floriou-Servou, A., Markicevic, M., Vermeiren, Y., Sturman, O., Privitera, M., von Ziegler, L., Ferrari, K.D., Weber, B., De Deyn, P.P., Wenderoth, N., Bohacek, J., 2019. Rapid Reconfiguration of the Functional Connectome after Chemogenetic Locus Coeruleus Activation. Neuron 103, 702-718.e5. https://doi.org/10.1016/j.neuron.2019.05.034

Zhao, W., Martin, A.D., Davenport, P.W., 2003. Magnitude estimation of inspiratory resistive loads by double-lung transplant recipients. J. Appl. Physiol. 94, 576–582. https://doi.org/10.1152/japplphysiol.00564.2002

Zhu, L., Mathewson, K.E., Hsu, M., 2012. Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning. Proc. Natl. Acad. Sci. U. S. A. 109, 1419–1424. https://doi.org/10.1073/pnas.1116783109

Zink, N., Bensmann, W., Arning, L., Stock, A.-K., Beste, C., 2019. CHRM2 Genotype Affects Inhibitory Control Mechanisms During Cognitive Flexibility. Mol. Neurobiol. 56, 6134–6141. https://doi.org/10.1007/s12035-019-1521-6

304

Zipf, G.K., 1949. Human behavior and the principle of least effort, Human behavior and the principle of least effort. Addison-Wesley Press, Oxford, England.

Zuend, M., Saab, A.S., Wyss, M.T., Ferrari, K.D., Hösli, L., Looser, Z.J., Stobart, J.L., Duran, J., Guinovart, J.J., Barros, L.F., Weber, B., 2020. Arousal-induced cortical activity triggers lactate

release from astrocytes. Nat. Metab. 2, 179–191. https://doi.org/10.1038/s42255-020-0170-4

Zwosta, K., Ruge, H., Wolfensteller, U., 2015. Neural mechanisms of goal-directed behavior: outcome-based response selection is associated with increased functional coupling of the angular gyrus. Front. Hum. Neurosci. 9. https://doi.org/10.3389/fnhum.2015.00180

List of figures

305

List of figures

Figure 1 : Physical effort carries disutility and increased effort requirements need to be

associated with increased rewards to be chosen (adapted from Walton et al., 2006) ............. 8

Figure 2 : Acceptance rate decreased when physical effort increased in a force exertion task

consisting in pressing a grip at four different force levels (adapted from Zénon et al., 2014) 10

Figure 3 : During force exertion, after an initial dip, pupil diameter increased and was positively

correlated with the exerted force (figure adapted from Zénon et al., 2014). ......................... 11

Figure 4: Positive relationship between RPEs and (A) oxygen uptake (percentage of maximum

uptake, VO2max) and (B) heart rate (percentage of maximum heart rate, HRmax) (adapted from

Andrade et al., 2020). ............................................................................................................... 12

Figure 5: Tasks used to test the different components of cognitive control ........................... 14

Figure 6: Pupil diameter increases during decisions, especially when subjects will respond

against their bias. ..................................................................................................................... 17

Figure 7: Behavioral demonstration of the aversiveness of mental effort (figure adapted from

Kool and Botvinick, 2018) ......................................................................................................... 22

Figure 8 : Results of the modified Stroop task used by Padmala and Pessoa (2011). RTs were

shorter, accuracy was higher and difficulty had lesser effects on RT and accuracy when

subjects were rewarded. .......................................................................................................... 23

Figure 9: Cognitive effort titration task (adapted from Westbrook et al., 2013). ................... 24

Figure 10 : The VOC model and the EVC model ....................................................................... 26

Figure 11: Representation of the determination of the optimal control intensity signal in a

certain state (payoffs and costs) and in case of increases in payoff or in difficulty (adapted from

Shenhav et al., 2013). ............................................................................................................... 28

Figure 12: Labor-leisure theory, in economics and applied to cognitive control (figure adapted

from Kool and Botvinick, 2014). ............................................................................................... 29

Figure 13 : Schematic representation of the different constraints on control exertion that

could account for limited control capacities and control aversiveness (adapted from Shenhav

et al., 2017). .............................................................................................................................. 32

Figure 14 : Ego-depletion after a controlled attention task, in a Stroop task with or without

glucose consumption (adapted from Gailliot et al., 2007). ..................................................... 34

Figure 15 : Glycogen as a buffer for brain metabolic resources (adapted from Christie and

Schrater, 2015) ......................................................................................................................... 35

List of figures

306

Figure 16: Influence of value difference (VD) and trial overall value (OV) or avoidance ratings

and on RTs in a value-based decision-making task .................................................................. 41

Figure 17 : The multiple-object tracking task (adapted from Alnæs et al., 2014). .................. 45

Figure 18: ERP recorded in EEG. ERPs can be decomposed into several subcomponents: N1

(positive), P2 (negative), N2 (positive) and P3 (negative) ....................................................... 47

Figure 19: Anatomy and histology of the LC in humans and monkeys .................................... 51

Figure 20: LC neurons respond to surprising stimuli with a characteristic activation-inhibition

response ................................................................................................................................... 52

Figure 21: ERPs evoked by LC optogenetic stimulation (adapted from Vazey et al., 2018). ... 58

Figure 22: Positive correlation between pupil diameter and LC activity ................................. 60

Figure 23 : Optogenetic inhibition of LC activity promotes sleep while stimulation promotes

wake (Adapted from Carter et al., 2010). ................................................................................ 62

Figure 24 : LC activity is modulated by stress, via the action of CRH. ..................................... 64

Figure 25: Optogenetic manipulations and Calcium imaging of LC activity demonstrate a causal

role of the LC/NA system in inhibitory control exertion (adapted from Bari et al., 2020) ...... 68

Figure 26: Pupil diameter increased more before exploration than exploitation, and individual

pupil diameter baseline was positively correlated with individual tendency to explore (adapted

from Jepma and Nieuwenhuis, 2011). ..................................................................................... 70

Figure 27: Increasing LC activity caused increased exploration and decision variability, while

decreasing LC activity decreased choice variability. ................................................................ 72

Figure 28: LC neurons encoded the exerted force during force exertion, and decreasing brain

NA decreased exerted force. .................................................................................................... 73

Figure 29: Effects of LC activity on action selection: increased LC activity, measured indirectly

through pupil dilation, was associated to decreased decision biases, while decreases in LC

activity by clonidine increased physical effort sensitivity. ....................................................... 76

Figure 30: LC neurons encoded engagement in the current trial at the time of cue onset and

in the next trial around fixation breaks i.e. disengagement from trials (adapted from Jahn et

al., 2020). .................................................................................................................................. 78

Figure 31: LC neurons differentiated between cues carrying novel information and cues

carrying no novel information. ................................................................................................. 80

Figure 32: LC neurons encoded some of the information provided by task cues on trial

contingencies, especially reward size. ..................................................................................... 81

List of figures

307

Figure 33: In forced-choice discrimination tasks, LC activity was better aligned on response

onset than on stimulus onset. LC neurons were only active if an action was executed,

regardless of reward expectation. ........................................................................................... 83

Figure 34: LC neurons encode task parameters just prior to action onset: reward negatively

and force positively. ................................................................................................................. 85

Figure 35: LC neurons respond to task-relevant cues. ............................................................. 91

Figure 36 : Effects of LC stimulation on activity of simultaneously recorded thalamic neurons

and somatosensory neurons. ................................................................................................... 93

Figure 37: In a spatial memory task, α2 agonist guanfacine enhanced while α2 antagonist

yohimbine decreased spatial-related responses of PFC neurons during memory retain period.

.................................................................................................................................................. 94

Figure 38: Yerkes-Dodson relationship between LC discharge and performance. .................. 96

Figure 39: The activation function, relating the input of a neuronal system (x-axis, Net input)

to the probability of firing in response to the input (y-axis, activation). ................................. 98

Figure 40: Network reset theory (adapted from Bouret and Sara, 2005). ............................ 102

Figure 41: Summary of the mean variations in rate for the 75 recorded neurons around cue

onset (A, B) and the outcome (C, D). ..................................................................................... 154

Figure 42: Summary of the modulation of rate by task parameters in large time intervals

around cue onset and the outcome ....................................................................................... 156

Figure 43: Summary of the dynamics of the modulation of rate by task parameters (Reward

and Delay) around cue onset (A,B,C), action onset (D, E, F) and the outcome (G, H, I). ....... 161

Figure 44: Summary of the correlations between baseline rate and the strength of the

response to task events and of modulation of rate by task parameters around task events.

................................................................................................................................................ 163

Figure 45: Two alternatives for the relationship between baseline rate and signal-to noise

ratio, represented using schematic representations of the mean rate of virtual example units

across time around an event of interest. ............................................................................... 165

Figure 46: Two-alternative choice task. ................................................................................. 175

Figure 47: Electrophysiological recordings: location in the brain and pharmacological

validation. ............................................................................................................................... 177

Figure 48: RT determination and decomposition: forces exerted in the two-alternative choice

task and eye-tracking procedure. .......................................................................................... 179

List of figures

308

Figure 49: Behavior associated to the middle press was compatible with a positive relationship

between RT and the cost of triggering the middle press. ...................................................... 184

Figure 50: Decisions to engage in the choice in the two-alternative choice task. ................. 186

Figure 51: Action-triggering time increased with difficulty to engage in the choice ............. 188

Figure 52: The subjective difficulty to engage in the choice scaled with the subjective difficulty

to engage in the middle press. ............................................................................................... 191

Figure 53: Binary choices between offers, and action-selection times. ................................ 194

Figure 54: Relationship between action-triggering time and choice accuracy: tenuous evidence

for a speed-accuracy tradeoff, and evidence for a “speed-cognitive control production”

tradeoff. .................................................................................................................................. 200

Figure 55: Negative relationship between maximum exerted force and engagement difficulty.

................................................................................................................................................ 203

Figure 56: Activity of LC neurons around middle press onset: LC neurons activated around the

middle press and encoded middle press RT before the middle press. .................................. 210

Figure 57: Increased LC activity before middle press onset was associated to decreased

difficulty to engage in the choice, as choice engagement increased and action-triggering time

decreased. .............................................................................................................................. 214

Figure 58: Pre-middle press spike count had a positive relation with middle press exerted

force........................................................................................................................................ 216

Figure 59: Rate of LC neurons around cues onset, and results of first and second order t-tests

assessing rate changes: after cues onset, LC activity increased. ........................................... 218

Figure 60: LC neurons encoded RT positively, and specifically its subcomponent action-

triggering time, over and above offer parameters, around cues onset. ............................... 220

Figure 61: Encoding of offer parameters (Forces and Rewards) in a late epoch ([200:500ms])

after cue onset: LC neurons only encoded Rewards. ............................................................. 223

Figure 62: LC activity around cues onset had a positive influence on the probability to choose

high-Force options, on the exerted force, especially in high-chosen Force conditions, and a

positive effect on performance. ............................................................................................. 225

Figure 63: Rate of LC neurons around side action onset, and results of first and second order

t-tests assessing rate changes. ............................................................................................... 231

Figure 64: LC neurons encoded the difficulty of the action during its execution. ................. 233

309

Figure 65: Increased spike count during the previous side press was associated to increased

Force sensitivity of the next Force-based choice. .................................................................. 235

Figure 66: LC neurons encoded performance of previous side press, but not engagement in

previous side press during the ITI ([-450;-150ms] before middle red dot onset). ................ 237

List of tables

Table 1: Latencies of the firing rate changes and task parameter encoding at cue onset,

action onset and the outcome. The latencies indicated correspond to the median and

confidence interval of the position of the windows in which a change was detected. ........ 159

Table 2: Results for the Pearson’s correlations of baseline rate with effect sizes

(maximum of the absolute values of coefficients from the spike count GLM) for reward, delay,

RT and willingness to work, at cue onset, action onset and the outcome. ........................... 166

List of abbreviations

310

List of abbreviations 4V: fourth ventricule

4x: trochlear decussation

ACC: anterior cingulate cortex

ADHD: attention-deficit hyperactivity disorder

AIC: Akaike Information Criteria

ANOVA: analysis of variance

AST: action-selection time

bc: brachium conjonctivium

BIC: Bayesian Information Criteria

BOLD: blood-oxygen-level dependent

cAMP: cyclic adenosine monophosphate

Cb: cerebellum

Cb2: cerebellum

CGPn: central gray of the rhombencephalon

ChR2: channelrhodopsin-2

CNO: clozapine-N-oxide

Corr: correct

CRH: corticotropin-release hormone

CS: conditioned stimulus

ctg: central tegmental tract

ctt: constant

D: delay

DA: dopamine

dACC: dorsal anterior cingulate cortex

DDM: drift-diffusion model

DREADD: Designer Receptor Exclusively Activated

by Designer Drugs

DβH: dopamine-β-hydroxylase

EEG: Electroencephalograms

eNpHR: halorhodopsin

ERP: event-related potential

EVC expected value of control

eYFP: yellow fluorescent protein

F: force

FA: false alarm

fMRI: functional magnetic resonance imaging

FR: number of lever presses

G-6-P: glucose-6-phosphate

GANE: glutamate amplification of noradrenaline

GLM: generalized linear model

HCN channel: hyperpolarization-activated cyclic

nucleotide gated cation channel

HF: high-force

HR: high-reward

HRmax: maximum heart rate

incorr: incorrect

ITI: inter-trial interval

LC: locus coeruleus

LR: low-reward

LTD: long-term depression

LTP: long-term potentiation

LV: lateral ventricule

mcp: middle cerebellar pedoncule

Me5: mesencephalic trigeminal nucleus

mlf: medial longitudinal fasciculus

Mo5N: rostral tip of the motor trigeminal nucleus

MPB: medial parabrachial nucleus

mPFC: medial prefrontal cortex

MRI: magnetic resonance imaging

mRNA: messenger ribonucleic acid

MVC: maximum voluntary contraction

n.s.: non significant

NA: noradrenaline

NAc: nucleus accumbens

NMDA: N-methyl-D-aspartate

NREM: non rapid eye movement

OV: overall value

PFC: prefrontal cortex

Pgi: nucleus Paragigantocellularis

Pn: pontine nuclei

PnO: pontine nuclei

prem: premature

PTG: protein targeting glycogen

PTSD: post traumatic stress disorder

R: reward

REM: rapid eye movement

RPE: rate of perceived exertion

RT: reaction time

scp: superior cerebellar pedoncule

SEM: standard error of the mean

SMA: supplementaru motor area

spk: spike

SSRT: stop signal reaction time

subCD: dorsal part of the subcoeruleus

subCV: ventral part pf the subcoeruleus

TPR: task-evoked pupillary responses

tVNS: transcutaneous vagus nerve stimulation

TW: time window

V: value

VD: value difference

ven: fourth ventricule

vmPFC: ventro-medial prefrontal cortex

VO2max: maximum oxygen uptake

VOC: value of computations

vsc: ventrat spinocerebellar tract

WTW: willingness to work

Résumé en français

311

Résumé en français

Allez-vous aller faire l’effort d’aller courir ce soir, ou resterez-vous dans votre canapé

devant un film? La prochaine fois que vous ferez vos courses, allez-vous faire l’effort de

calculer de tête le montant de tous vos articles ? Chaque jour, nous réalisons de nombreuses

décisions, et une grande partie d’entre elles repose sur un arbitrage entre un effort, physique

ou mental, et une récompense. L’effort est aversif, c’est-à-dire que les sujets agissent de telle

manière à éviter les actions qui impliqueraient de faire un effort. De plus, l’effort représente

les ressources qui sont mises en œuvre pour réaliser les actions, ce qui implique que faire un

effort est bénéfique pour la performance. Dans l’introduction du manuscrit, sont détaillées les

caractéristiques des deux types d’effort, physique et mental, ainsi que le lien entre ces deux

types d’effort, et dans un second temps, les éléments qui ont amené à la formulation de

l’hypothèse de l’implication du locus coeruleus (LC), noyau noradrénergique cérébral, dans la

gestion de l’effort.

Concernant l’effort physique, il a été démontré à de nombreuses reprises qu’il

constitue une variable de décision aversive, puisque les sujets évitent dans la mesure du

possible les options impliquant un effort physique, surtout si celui-ci est intense (Cousins and

Salamone, 1994; Hull, 1943; Salamone et al., 2007; Stevens et al., 2005; Walton et al., 2006).

Il s’agit de la loi du moindre effort formulée par Hull en 1944. La perception de l’effort par les

sujets ne repose pas sur des afférences sensitives provenant des muscles mis en jeu, mais

probablement d’une duplication de la commande motrice dans l’aire motrice supplémentaire

(White et al., 2013; Zénon et al., 2015). En conséquence, l’intensification de l’activité nerveuse

à l’origine du mouvement en cas d’administration de curares produit, pour une action

identique, une augmentation de la sensation d’effort (Gallagher et al., 2001). L’effort physique

est une notion intrinsèquement subjective, mais il peut être estimé en demandant aux sujets

de noter leur effort (Borg, 1982), en mesurant pour chaque niveau de difficulté le taux

d’acceptation des options (Jahn et al., 2020; Varazzani et al., 2015; Zénon et al., 2014), ou

encore la fréquence à laquelle les sujets acceptent de répéter les actions (Zénon et al., 2014).

Au-delà de ces mesures comportementales, plusieurs mesures physiologiques ont été

proposées : le diamètre pupillaire, la fréquence cardiaque et le taux sanguin de lactate

(Andrade et al., 2020; Da Silva et al., 2020; Tibana et al., 2019; Zénon et al., 2014). Les deux

Résumé en français

312

derniers sont sujets à des fluctuations lentes, et ne permettent donc pas d’évaluer l’effort

perçu par les sujets avec une résolution temporelle fine ou pour des actions brèves.

Concernant l’effort mental, il est défini comme la mobilisation d’énergie afin

d’atteindre un but (Gaillard, 2001, 1993), l’intensification subjective de l’activité mentale qui

relie les capacités d’un individu à effectuer une tâche à sa performance dans cette tâche (Kool

and Botvinick, 2018). Récemment, le concept d’effort mental a été unifié sous le terme de

contrôle cognitif, défini comme un ensemble de processus facilitant les comportements

dirigés vers un but (Cohen et al., 2001; Dixon and Christoff, 2012; Posner and Snyder, 2004;

Shenhav et al., 2017). Trois grandes catégories de tâches nécessitant d’exercer un effort

mental ont été identifiées : les tâches nécessitant de mettre à jour des informations stockées

dans la mémoire de travail (Kirchner, 1958), des tâches nécessitant d’exercer du contrôle

inhibiteur en particulier pour inhiber des réponses automatiques (Stroop, 1935), et enfin des

tâches nécessitant de changer régulièrement les règles appliquées (Monsell, 1996; Monsell et

al., 2003).

En ce qui concerne les mécanismes sous-jacents à l’allocation de contrôle cognitif, il a

été théorisé qu’on pouvait les décomposer en trois sous-processus (Shenhav et al., 2013). Le

premier est la régulation, qui à la modulation des processus bas-niveau de traitement

d’information. Le second est la spécification, qui se réfère aux décisions concernant

l’allocation de contrôle, définissant l’identité et l’intensité du signal de contrôle à

implémenter. Enfin, le suivi correspond à la collection d’informations concernant les

conséquences (récompenses, coûts, performance) de l’allocation de contrôle cognitif, afin

d’améliorer les futures spécifications.

Dans le contexte de la prise de décision, le contrôle cognitif est à la fois une variable à

prendre en compte, puisque les sujets s’efforcent d’éviter d’avoir à exercer du contrôle, et un

facteur d’amélioration de la prise de décision, puisque le contrôle cognitif permet d’optimiser

la prise de décision (Clark et al., 2004; Coricelli et al., 2005; De Martino et al., 2006; De Neys

et al., 2008; Eslinger and Damasio, 1985; Hinson et al., 2003; Manes et al., 2002; Mäntylä et

al., 2012; Missier et al., 2012, 2010). Ainsi, en l’absence de contrôle cognitif, les décisions sont

prises grâce à des stratégies plus simples, et probablement plus automatiques. Le contrôle

cognitif permet de faire appel à des stratégies décisionnelles plus complexes, nécessitant de

traiter plus d’informations par exemple (Gigerenzer, 2010; Gigerenzer and Goldstein, 1996;

Résumé en français

313

Sidarus et al., 2019; Simon, 1955; Tversky and Kahneman, 1974). Les sujets sont en

conséquence moins sensibles à leurs biais s’ils exercent du contrôle cognitif pour la prise de

décisions.

Concernant le rôle du contrôle cognitif comme variable de décision, il a été montré

que les sujets font un compromis entre le contrôle cognitif qu’ils investissent dans une tâche

et les bénéfices que cela peut leur rapporter (Kool et al., 2010; Kool and Botvinick, 2018, 2014;

Westbrook et al., 2013). Les sujets n’investissent généralement pas autant de contrôle cognitif

que ce dont ils seraient capables, et que ce qui permettrait de maximiser leur performanceun

constat résumé par la théorie de la rationalité limitée. Au cours des dernières années, de

nombreuses équipes ont formulé des modèles mathématiques formalisant les stratégies

d’allocation de contrôle cognitif (Griffiths et al., 2015; Lieder et al., 2014, 2012; Lieder and

Griffiths, 2017; Shenhav et al., 2017, 2013). Ces modèles se basent sur les principes

d’apprentissage par renforcement hiérarchique, c’est-à-dire qu’ils partent du principe que,

d’essai en essai, les sujets ajustent le contrôle qu’ils allouent à la tâche en fonction des

récompenses obtenues et des coûts du contrôle estimés et perçus. En d’autre termes, ils

identifient la quantité de contrôle cognitif qui leur permet de maximiser la valeur (bénéfices

moins coûts) de ce contrôle.

Puisque les sujets s’efforcent de minimiser le contrôle cognitif qu’ils allouent,

l’existence ressources qui seraient consommées pour exercer du contrôle est présumée. La

nature de ces ressources demeure hypothétique. Une première possibilité serait qu’il s’agisse

de substrats métaboliques, glucose ou glycogène (Baumeister et al., 1998; Baumeister and

Heatherton, 1996; Christie and Schrater, 2015; Muraven et al., 1998). Cette hypothèse repose

tout d’abord sur le fait que l’hypoglycémie est délétère pour l’effort mental, et la

consommation de glucose peut améliorer les performances, mais aussi sur certaines études

qui ont rapporté que les sujets voient leur glycémie diminuer après un effort mental (Benton

et al., 1987; Donohoe and Benton, 1999; Fairclough and Houston, 2004; Gailliot et al., 2007;

Kennedy and Scholey, 2000). Compte tenu que ce dernier résultat n’a pas été reproduit par la

suite, le rôle du glucose comme ressource limitante est maintenant discuté. Le glycogène, une

forme de stockage de glucose présente dans les astrocytes, a été proposé à sa place (Christie

and Schrater, 2015). Une seconde possibilité pour expliquer la dimension aversive du contrôle

cognitif repose sur l’existence d’une limite aux computations réalisables par le cerveau à un

instant donné (Allport, 1954; Anderson, 1983; Cohen et al., 1990; Logan, 1985). Toutefois,

Résumé en français

314

cette théorie ne permet d’expliquer que l’impossibilité à réaliser deux tâches cognitives en

parallèle (Shenhav et al., 2017). Enfin, une troisième possibilité purement motivationnelle

repose sur l’existence de coûts d’opportunité et sur la nécessité d’un équilibre entre

comportements d’exploitation et d’exploration (Chittka et al., 2003; Franks et al., 2003; Heitz

and Schall, 2012; Inzlicht et al., 2014; Inzlicht and Schmeichel, 2012; Wickelgren, 1977).

Compte tenu de l’impossibilité d’effectuer d’autres tâches en parallèle, allouer du contrôle à

une tâche en particulier implique de n’en effectuer aucune autre. L’allocation de contrôle

prolonge généralement les computations, ce qui veut dire qu’allouer plus de contrôle

augmente les coûts d’opportunité. En d’autres termes, l’évitement du contrôle cognitif

pourrait faciliter l’équilibre entre exploration et exploitation.

De même que l’effort physique, le contrôle cognitif est une notion subjective. Afin de

le mesurer, ou au moins de l’approximer, des mesures comportementales ont été proposées :

le temps de réaction et la performance. L’utilisation de la performance repose sur le principe

qu’à difficulté égale, augmenter le contrôle cognitif mobilisé pour la tâche augmentera la

performance (Norman and Bobrow, 1975). L’utilisation du temps de réaction est basée sur

l’idée simplificatrice que les ressources sont mobilisées avec un taux constant au cours du

temps, ce qui implique que plus de temps passé sur la tâche correspond à plus de ressources

mobilisées (Hunt et al., 2012; Krajbich et al., 2010; Krajbich and Rangel, 2011; Milosavljevic et

al., 2010; Morris et al., 2019; Philiastides and Ratcliff, 2013). Ces deux mesures

comportementales sont limitées car elles peuvent amener à un raisonnement circulaire

(Navon, 1984; Shenhav et al., 2017). En effet, il est difficile de faire la part des choses entre

des effets de la difficulté ou du contrôle cognitif mobilisé. Des mesures physiologiques ont

donc été proposées. Les principales sont les augmentations du rythme cardiaque (Carroll et

al., 1986a; Turner and Carroll, 1985) et de l’activité métabolique (Carroll et al., 1986a; Sims et

al., 1988; Sims and Carroll, 1990; Turner et al., 1983; Turner and Carroll, 1985), le diamètre

pupillaire (Alnæs et al., 2014; Hess and Polt, 1964; Kahneman and Beatty, 1966; Simpson and

Hale, 1969; Wahn et al., 2016; Wright et al., 2013), et les composants P3 et N2 des potentiels

cérébraux liés aux évènements mesurés par électroencéphalogramme (Clayson and Larson,

2013; Falkenstein et al., 1999; Folstein and Van Petten, 2008; Gajewski and Falkenstein, 2011;

Garavan et al., 1999; Jamadar et al., 2015; Kopp et al., 2020; Pratt et al., 2011; Qi et al., 2014;

Tillman and Wiens, 2011; Wei and Zhou, 2020).

Résumé en français

315

Il est intéressant de noter que malgré la catégorisation fréquente de l’effort en deux

sous catégories bien disjointes (physique et mental), il pourrait y avoir un certain

recouvrement entre ces deux concepts, et en particulier entre les ressources consommées. En

effet, faire un effort physique conduit à une diminution de la capacité pour les efforts mentaux

et vice versa (Boksem and Tops, 2008; Cadegiani and Kater, 2019; Martin et al., 2018;

McMorris, 2020; Meeusen et al., 2013; Schiphof-Godart et al., 2018; Smit, 1978; Van Cutsem

et al., 2017). Il est possible que cette relation entre les deux types d’effort vienne du fait

qu’exercer du contrôle cognitif est nécessaire à l’initiation et à la persévérance dans

l’exécution des efforts physiques, puisque le contrôle cognitif permet d’éviter les

comportements automatiques (Epstein, 1994; Sloman, 1996). Il est aussi possible que ces deux

formes d’effort consomment le même substrat métabolique : le glycogène (Christie and

Schrater, 2015; Matsui et al., 2017).

L’hypothèse de travail de cette thèse concerne l’implication du LC dans la gestion de

l’effort physique et mental. De fait, la deuxième partie de mon introduction porte sur les

caractéristiques de ce noyau noradrénergique du tronc cérébral, et sur les éléments qui ont

amené à émettre notre hypothèse. Le LC est un petit noyau dont tous les neurones produisent

de la noradrénaline (NA) (Dahlstroem and Fuxe, 1964; Goldman and Coleman, 1981;

Nieuwenhuys et al., 2008; Olszewski and Baxter, 1954; Sharma et al., 2010). C’est la seule

source de NA pour tout le cerveau antérieur. Il se situe dans le tronc cérébral, latéralement au

quatrième ventricule, ventralement à la partie la plus antérieure du cervelet, aux colliculi

inférieurs et au noyau du nerf trijumeau (Bouret and Richmond, 2009; Grant et al., 1988;

Kalwani et al., 2014). Les neurones du LC ont des caractéristiques électrophysiologiques qui

permettent de les reconnaître : un taux de décharge faible, des potentiels d’action larges, et

des réponses stéréotypées (activation puis longue inhibition) en cas de stimuli surprenants

(Bouret and Richmond, 2009; Bouret and Sara, 2004; Grant et al., 1988; Hervé-Minvielle and

Sara, 1995). Bien que les neurones du LC produisent tous de la NA, il ne s’agit pas d’un noyau

homogène : les neurones sont organisés en modules, contenant des cellules de morphologie

différente, qui produisent différents neurotransmetteurs en plus de la NA, expriment les

récepteurs à la NA en quantités différentes, reçoivent des afférences de zones cérébrales

différentes et projettent des axones vers des aires différentes (Aston-Jones et al., 1991;

Chamba et al., 1991; Léna et al., 1999; Luppi et al., 1995; Schwarz et al., 2015; Simpson et al.,

Résumé en français

316

1999; Sutin and Jacobowitz, 1991; Takeuchi et al., 2016; Xu et al., 1998; Young and Kuhar,

1980). Ajoutant à cette hétérogénéité anatomo-fonctionnelle, les effets de la NA sont

variables en fonction de l’aire cérébrale cible, car les différents récepteurs ne sont pas répartis

de manière homogène dans le cerveau, et ont des affinités différentes (Arnsten, 2000).

Compte tenu de sa localisation, l’activité du LC est difficilement mesurable, c’est

pourquoi de nombreuses équipes ont cherché à définir des mesures indirectes. Le diamètre

pupillaire (Jepma et al., 2011; Joshi et al., 2016; Murphy et al., 2014; Phillips et al., 2000;

Varazzani et al., 2015) et le composant P3 des potentiels cérébraux liés aux évènements

mesurés par électroencéphalogramme (de Rover et al., 2015; Nieuwenhuis, 2011;

Nieuwenhuis et al., 2011, 2005; Pineda et al., 1989; Swick et al., 1994; Vazey et al., 2018) sont

corrélés au taux de décharge des neurones du LC, bien qu’ils ne fournissent qu’une faible

résolution temporelle et qu’ils soient assez peu spécifiques.

Le LC et la NA sont impliqués dans plusieurs processus mobilisant des ressources. En

effet, le LC a un rôle prépondérant dans la balance entre veille et sommeil, car il promeut l’état

de veille et participe aux transitions entre sommeil et veille (Aston-Jones, 2005; Carter et al.,

2010; Foote et al., 1983; Jouvet, 1969; Saper et al., 2005; Sara, 2009; revue par Berridge and

Waterhouse, 2003). D’autre part, le LC participe aux réactions liées au stress, via des

connections à l’amygdale (Bingham et al., 2011; Bouret et al., 2003; Buffalari and Grace, 2007;

Cassens et al., 1981; Curtis et al., 2012; Koob, 1999; Sved et al., 2002; Valentino et al., 1991;

Van Bockstaele et al., 2001). Enfin, il est impliqué dans la régulation de l’attention (Bari et al.,

2020; Clark et al., 1987; Coull et al., 2001; Reynaud et al., 2019; Witte and Marrocco, 1997).

Au-delà de ces effets, le LC participe à la promotion de processus coûteux dirigés vers

un but. Il a été mentionné plus haut que la dilatation pupillaire corrèle avec le taux de

décharge du LC (Joshi et al., 2016; Murphy et al., 2014; Varazzani et al., 2015). Le diamètre

pupillaire corrèle avec quant à lui avec l’effort physique ou mental engagés dans une tâche

(Alnæs et al., 2014; Hess and Polt, 1964; Kahneman and Beatty, 1966; Simpson and Hale, 1969;

Wahn et al., 2016; Wright et al., 2013; Zénon et al., 2014). De la même manière, le composant

P3 des potentiels cérébraux liés aux évènements mesurés par électroencéphalogramme

corrèle à la fois avec l’activité du LC (de Rover et al., 2015; Nieuwenhuis, 2011; Nieuwenhuis

et al., 2011, 2005; Pineda et al., 1989; Swick et al., 1994; Vazey et al., 2018) et avec l’effort

mental (Clayson and Larson, 2013; Falkenstein et al., 1999; Folstein and Van Petten, 2008;

Gajewski and Falkenstein, 2011; Garavan et al., 1999; Jamadar et al., 2015; Kopp et al., 2020;

Résumé en français

317

Pratt et al., 2011; Qi et al., 2014; Tillman and Wiens, 2011; Wei and Zhou, 2020). Ces éléments

suggèrent que le LC pourrait participer à la gestion de l’effort. De plus, les neurones du LC

encodent la force exercée autour d’actions requérant de la force (Varazzani et al., 2015).

Par le biais de modifications pharmacologiques, chémogénétiques et optogénétiques

de l’activité du LC, il a été démontré que son activité et les taux cérébraux de NA sont

causalement impliqués dans l’exécution de tâches requérant du contrôle cognitif ou un effort

physique (Arnsten, 2006; Bari et al., 2020; Chamberlain et al., 2007, 2006a, 2006b; Coradazzi

et al., 2016; Faraone et al., 2005; Fernando et al., 2012; Gamo et al., 2010; Gilzenrat et al.,

2010; Jepma and Nieuwenhuis, 2011; Lapiz et al., 2007; Lapiz and Morilak, 2006; Li et al., 1999;

Li and Mei, 1994; Mao et al., 1999; McGaughy et al., 2008; Navarra et al., 2008; Overtoom et

al., 2003; Robinson, 2012; Robinson et al., 2008; Tait et al., 2007; Tzavara et al., 2006; Wang

et al., 2007). En particulier, nous avons démontré un rôle causal de la NA dans l’allocation

d’effort pour l’exécution d’actions nécessitant de produire de la force (Borderies et al., 2020;

Jahn et al., 2018) : suite à l’administration de clonidine, qui diminue l’activité des neurones du

LC, dans une tâche de compromis effort physique/récompense, les singes exerçaient moins

de force.

En ce qui concerne la prise de décisions, il a été mentionné plus haut que le contrôle

cognitif permet de l’optimiser. La pupille (dont le diamètre corrèle avec l’activité du LC) se

dilate au cours des décisions (Beatty, 1982; de Gee et al., 2014; Gilzenrat et al., 2010; Lempert

et al., 2015; Nassar et al., 2012). Le processus décisionnel peut se diviser en deux temps : la

sélection de l’action et le déclenchement de l’action choisie. Sélectionner une action nécessite

de traiter les informations sur les options disponibles. Les neurones du LC répondent à

l’apparition de stimuli informant les sujets sur les coûts et bénéfices des options proposées

(Bouret and Richmond, 2015; Jahn et al., 2020; Varazzani et al., 2015), d’autant plus qu’ils

fournissent des informations nouvelles (Jahn et al., 2020), ce qui serait compatible avec un

rôle de facilitation du traitement des informations. De plus, la NA a un rôle causal dans la

saillance motivationnelle, c’est-à-dire dans la régulation de l’intensité des comportements

facilitant l’accomplissement d’un objectif (Ventura et al., 2008, 2007). Par ailleurs, diminuer

l’activité du LC cause une augmentation de la sensibilité des choix à l’effort physique et une

diminution de la fréquence de sélection des options les plus difficiles (Borderies et al., 2020).

Compte tenu du biais qui consiste à éviter les efforts, engager du contrôle cognitif est

nécessaire pour choisir les options associées à un gros effort (Epstein, 1994; Sloman, 1996),

Résumé en français

318

ce qui suggère que la diminution de la fréquence des choix impliquant une action difficile

pourrait correspondre à une diminution de la capacité à engager du contrôle cognitif. En

d’autres termes, cela indiquerait que la noradrénaline pourrait être causalement impliquée

dans la mobilisation de contrôle cognitif pour la prise de décisions. L’idée du lien entre activité

du LC et contrôle cognitif est renforcée par les études qui ont montré le lien négatif entre

diamètre pupillaire et force des biais décisionnels (de Gee et al., 2020, 2017, 2014).

Pour ce qui est du déclenchement des actions choisies, les neurones du LC s’activent

généralement avant le déclenchement d’actions coûteuses et encodent la difficulté de l’action

déclenchée (Bouret et al., 2012; Bouret and Richmond, 2015, 2009; Bouret and Sara, 2004;

Clayton et al., 2004; Varazzani et al., 2015). Par ailleurs, inhiber l’activité du LC a tendance à

faire augmenter les temps de réaction (Jahn et al., 2018), qui peuvent représenter la difficulté

subjective à déclencher les actions.

Pour finir, l’hypothèse de l’implication du LC dans le suivi des conséquences des actions

et en particulier des erreurs a été émise, sur la base d’études de pupillométrie établissant un

lien entre l’amplitude des réponses pupillaires suite à une erreur et l’intensité du

ralentissement du temps de réaction suite à cette même erreur, c’est-à-dire l’intensité de

l’ajustement comportemental suite à une allocation d’effort insuffisante (Murphy et al.,

2016). De plus, des études augmentant avec peu de spécificité l’activité du LC (stimulation du

nerf vague) ont montré son lien de causalité avec le ralentissement post-erreur (Sellaro et al.,

2015).

Plusieurs mécanismes d’action de la NA, et donc de l’activité du LC pourraient

participer à son rôle dans la gestion de l’effort. Tout d’abord, la NA régule l’utilisation du

glycogène, qui a été mentionné plus haut comme potentielle ressource limitante pour

l’allocation de contrôle cognitif et d’effort physique. Dans les aires cérébrales cibles du LC, la

NA libérée facilite la glycogénolyse pour former du lactate, un substrat métabolique pour les

neurones (Magistretti, 1988; Quach et al., 1978; Sorg and Magistretti, 1991). A plus long

terme, la NA favorise la resynthèse de glycogène, en augmentant la production de protéines

critiques pour ce processus (Allaman et al., 2000; Sorg and Magistretti, 1992). Ces effets

métaboliques ont des conséquences positives sur les processus cognitifs (Gibbs, 2015; Gibbs

et al., 2010).

D’autre part, la NA facilite le traitement des informations sensorielles, puisque les

modifications électriques ou pharmacologiques de l’activité du LC augmentent le rapport

Résumé en français

319

signal/bruit, la précision temporelle des réponses, et le filtrage des stimuli dans les aires

cérébrales associées au traitement de l’information sensorielle (Ciombor et al., 1999; Devilbiss

et al., 2006; Foote et al., 1975; Halliday et al., 1994; Heath et al., 2006; Hurley et al., 2004;

Jiang et al., 1996; Kossl and Vater, 1989; Segal and Bloom, 1976; Turetsky and Fein, 2002;

Waterhouse et al., 2000, 1998). Cet effet a un intérêt particulier dans le contexte de l’effort,

car il a été mentionné plus tôt que le contrôle cognitif modifie l’identité des informations

traitées et l’intensité de leur traitement pour faciliter l’accomplissements des objectifs. Ces

effets facilitateurs du codage neuronal sont aussi présents dans des aires associées au

contrôle moteur (cervelet, Freedman et al., 1977; Hoffer et al., 1973, aire motrice primaire,

Plewnia et al., 2004) et au contrôle cognitif (cortex préfrontal, Li et al., 1999; Wang et al.,

2007).

Deux grandes théories qui donnent un contexte plus global à ces effets ont été

développées. Elles reposent sur la relation bien connue entre taux de décharge du LC et

performance cognitive, en forme de U inversé (Yerkes and Dodson, 1908). La première théorie

est celle de la régulation des états mentaux et des processus associés à ces états (Berridge and

Waterhouse, 2003). Elle formule l’idée que l’activité du LC pourrait mettre le cerveau dans un

certain état (par exemple veille ou sommeil) et qu’au sein de cet état elle pourrait moduler les

processus mentaux en cours. La théorie du gain adaptatif complémente cette première idée

(Aston-Jones and Cohen, 2005). Elle repose sur l’existence (maintenant discutée) de deux

modes d’activité du LC : tonique, avec un faible rapport signal/bruit, une activité de base

élevée et des réponses aux évènements modérées, et phasique, avec un fort rapport signal

bruit, une activité de base faible et des réponses aux évènements fortes. Le mode tonique

serait lié à la distractibilité, au désengagement des tâches et à l’exploration, tandis que le

mode phasique serait lié à l’engagement dans les tâches et à l’exploitation. L’ensemble de ces

effets serait lié à des modulations de gain, c’est-à-dire de rapport signal/bruit dans les cibles

du LC. Ces modulations de gain pourraient être responsables de variations de l’efficacité des

processus nécessaires à l’allocation d’effort.

De plus, les effets d’ajustement du gain dans les cibles du LC peuvent être prolongés

dans le temps par des effets de la NA sur la plasticité neuronale, principalement été démontrés

dans des aires sensitives et dans l’hippocampe (Bing et al., 1992, 1991; Gubits et al., 1989;

Hagena et al., 2016; Harley and Sara, 1992; Heginbotham and Dunwiddie, 1991; Hopkins and

Johnston, 1984; Lemon et al., 2009; Maity et al., 2020; Neuman and Harley, 1983; Salgado et

Résumé en français

320

al., 2012; Segal and Bloom, 1974; Stanton and Sarvey, 1985; Stone et al., 1997, 1991; Straube

et al., 2003; Walling and Harley, 2004). Notons que les effets de modulation du gain neuronal

et de facilitation de la plasticité pourraient être liés à une synergie entre glutamate et

noradrénaline (Mather et al., 2016).

Enfin, la noradrénaline modifie la connectivité à large échelle dans le cerveau (Borchert

et al., 2016; Brink et al., 2016; Coull et al., 1999; Guedj et al., 2017b; Zerbi et al., 2019), et ces

effets pourraient faciliter les transitions entre différents processus au cours d’une tâche ou

entre différentes tâches (Bouret and Sara, 2005; Corbetta et al., 2008; Sara and Bouret, 2012).

Concrètement, l’activité du LC augmente la connectivité entre les zones cérébrales impliquées

dans la tache en cours et diminue la connectivité dans les réseaux non pertinents. Cette idée

est cohérente avec les effets de facilitation de l’adaptation aux changements de règles par la

NA, et les modifications de connectivité cérébrale pourraient être une conséquence à plus

large échelle de modulations du gain neuronal, potentiellement via des modulations du

rythme Gamma (Guedj et al., 2017a).

Pour résumer, la NA facilite le traitement d’information dans les cibles du LC et ajuste

la connectivité cérébrale aux demandes de la tâche en cours. Le LC est causalement impliqué

dans des processus non dirigés vers un but qui augmentent la consommation de ressources :

éveil, stress et attention. En ce qui concerne les comportements dirigés vers un but, et en

particulier l’effort physique, les diminutions pharmacologiques de l’activité du LC ont

démontré un lien causal entre NA et difficulté subjective (coût de l’effort), une variable

computationnelle théorique qui affecte les choix et l’exécution des actions de manière

cohérente. Les neurones du LC s’activent et la pupille se dilate autour du déclenchement et

de l’exécution d’actions coûteuses. Dans le contexte de tâches de force, l’activité des

neurones et le diamètre pupillaire au cours de l’action sont corrélés positivement avec la force

requise et exécutée, bien que la dynamique exacte de ces modulations et les contributions

relatives de la difficulté objective et subjective n’aient pas été examinées en détail. En cas

d’allocation de contrôle cognitif, des mesures indirectes de l’activité du LC (P3 et diamètre

pupillaire) sont positivement corrélés à l’allocation d’effort, tandis que les modifications

pharmacologiques ou optogénétiques de l’activité du LC influencent la performance dans les

tâches impliquant du contrôle cognitif. Les neurones du LC répondent à la présentation

d’informations pertinente pour la tâche, et le diamètre pupillaire augmente au cours des

Résumé en français

321

décisions, prédisant négativement la force des biais décisionnels et donc le contrôle cognitif

alloué aux décisions. Enfin, les neurones du LC répondent aux erreurs, au moins les erreurs

d’engagement, et pourraient donc signaler les cas d’allocation insuffisante d’effort et faciliter

les ajustements comportementaux requis. En cohérence avec cette idée, les augmentations

non spécifiques de l’activité du LC ont des effets bénéfiques sur les modifications du

comportement en réponse aux erreurs.

Dans l’ensemble, ces résultats suggèrent un rôle critique de la NA et du LC dans la

gestion de l’effort physique et du contrôle cognitif. Précisément, nous avons premièrement

émis l’hypothèse que les neurones du LC modulent leur activité en fonction de l’effort alloué

au moment de l’exécution des actions requérant un effort, et signalent les conséquences de

cet effort, comme les erreurs. D’autre part, sur la base des résultats de modifications longues

de l’activité du LC, nous avons émis l’hypothèse que le LC aurait un rôle central dans la

sensibilité aux coûts de l’effort et la gestion de son allocation, puisqu’il semble faciliter les

processus nécessitant de l’effort et suivre ses coûts et conséquences, potentiellement dans le

but d’ajuster les futures décisions de faire des efforts.

Afin de confirmer cette hypothèse, plusieurs points avaient besoin d’être explorés. Il

était nécessaire d’obtenir des preuves d’un codage de l’effort au moment de l’exécution de

processus coûteux au-delà des preuves circonstancielles déjà disponibles, de généraliser ce

résultat au-delà de situations nécessitant un effort physique, et d’examiner les dynamiques

de ce codage. La première étude de ma thèse s’est focalisée sur ces aspects. Dans une seconde

étude, nous nous sommes intéressés aux conséquences de l’activation du LC sur le

comportement. Spécifiquement, nous avons examiné l’activité des neurones et son lien avec

la sensibilité à l’effort et la régulation de son allocation.

Les deux études de cette thèse ont consisté à analyser l’activité de neurones uniques

du LC (247 au total) dans des tâches de compromis effort-récompense chez des macaques

rhésus. Dans la première étude, l’activité de neurones du LC a été comparée dans trois tâches :

une de compromis Force-Récompense, une de compromis Délai-Récompense, et une tâche

de détection de signal. Dans la tâche de compromis Force-Récompense, les singes devaient

appuyer sur une poignée à un certain niveau de force pour obtenir une certaine récompense.

Le niveau de force et la récompense étaient indiqués par un indice visuel en début d’essai.

Dans la tâche de compromis Délai-Récompense, les singes devaient effectuer une action

Résumé en français

322

simple (relâcher une poignée) pour obtenir une récompense, qui serait distribuée après un

certain délai. De la même manière, délai et récompense étaient indiqués par un indice visuel

en début d’essai. Enfin, dans la tâche de détection de signal, le singe devait appuyer sur une

poignée lorsqu’un point à l’écran changeait de couleur. De fait, les deux tâches de compromis

Récompense-Délai ou Force manipulaient le contrôle cognitif nécessaire pour déclencher

l’action, puisque le comportement automatique aurait été de ne pas la réaliser. La tâche de

compromis Force-Récompense manipulait en plus l’effort physique. Enfin, les tâches de

compromis Force-Récompense et de détection de signal manipulaient les contraintes sensori-

motrices de la réponse. Afin de rechercher un codage de l’effort ou au moins de la difficulté

subjective du déclenchement et de l’exécution de l’action par les neurones du LC et de préciser

leurs dynamiques, nous nous sommes focalisés sur le moment du déclenchement et de

l’exécution des actions. Nous avons recherché des modulations de l’activité des neurones par

le temps de réaction (qui permet d’approximer la difficulté subjective et potentiellement

l’effort mental effectué pour déclencher l’action), et par la force exercée dans les deux tâches

impliquant d’appuyer sur la poignée. Nous avons découpé l’activité des neurones entre une

fenêtre de temps précédant l’action et une autre pendant l’action (pour les deux tâches

impliquant de la force, et donc une action maintenue dans le temps).

Nous avons constaté que l’activité des neurones du LC était modulée juste avant le

déclenchement des actions par le temps de réaction, uniquement lorsque le temps de réaction

reflétait des différences de difficulté à déclencher les actions (tâches de compromis Délai ou

Force-Récompense) mais pas lorsqu’il reflétait des différences sensori-motrices (tâche de

détection de signal). D’autre part, pendant l’exécution d’actions longues (force), nous avons

constaté que les neurones codaient la quantité de force appliquée sur la poignée, uniquement

lorsque les variations de force exercée étaient associées à des variations de difficulté

subjective (uniquement dans la tâche de compromis Force-Récompense). Nous en avons donc

conclu que les neurones du LC codaient la difficulté subjective des processus en cours, et

potentiellement l’effort mobilisé. La dynamique de ces modulations n’était toutefois pas

compatible avec des effets immédiats sur la mobilisation d’effort, puisque les neurones du LC

ont une vitesse de conduction lente (Aston-Jones et al., 1985). Par contre, elles pourraient

faciliter l’allocation d’effort pour les processus qui font suite à une action difficile, et servir de

signal du coût pour les structures ajustant l’allocation d’effort par la suite.

Résumé en français

323

Notons par ailleurs que l’analyse de l’activité des neurones dans la tâche de compromis

Délai-Récompense a mis en évidence plusieurs éléments. Au moment de l’apparition de

l’indice visuel, les neurones codaient la récompense promise et le délai associé, et il y avait un

codage marginalement significatif de la probabilité d’engagement dans la condition proposée,

c’est-à-dire potentiellement de la valeur subjective de l’essai. De plus, le codage de la

récompense et du délai étaient corrélés à travers les neurones, ce qui signifie que les neurones

avaient tendance à coder la valeur subjective de la récompense. D’autre part, nous avons

montré une hétérogénéité dans l’excitabilité des neurones, car ils différaient en terme de taux

de décharge de base, et ces différences étaient associées à des différences de force de

réponse aux évènements de la tâche et de modulation par les paramètres des essais (Délai et

récompense) et par les paramètres comportementaux (taux d’engagement, temps de

réaction). Ce résultat renforce l’idée de l’hétérogénéité du LC sur le plan électrophysiologique

et fonctionnel.

Dans une seconde étude, nous nous sommes focalisés sur les effets à moyen terme de

l’activation du LC. En effet, nous avons constaté dans la première étude que les neurones

codaient la difficulté subjective des processus en cours mais avec une dynamique

incompatible avec des effets directs sur ces processus. Pour cela, nous avons utilisé une tâche

de choix (2 alternatives, compromis Force-Récompense) et avons étudié l’activité de 54

neurones uniques du LC. Le singe devait d’abord appuyer sur une poignée placée devant lui

pour que deux offres, combinaisons de niveau de force à effectuer et de récompense promise,

lui soient présentées. Il devait choisir entre ces deux offres en appuyant sur une des deux

poignées placées à sa droite (option de droite) et à sa gauche (option de gauche). Nous avons

identifié plusieurs effets de l’activation du LC. Tout d’abord, pour chaque processus coûteux

(appuyer sur la poignée du milieu, faire le choix, déclencher l’action choisie, l’exécuter), les

neurones du LC s’activaient transitoirement, et cette activation reflétait généralement la

difficulté subjective du processus en cours. D’autre part, à l’échelle d’un essai, l’activation des

neurones conduisait par la suite à des temps de réaction réduits, des choix plus biaisés vers

les options difficiles, et des appuis plus forts, suggérant une diminution de la difficulté

subjective. A travers les essais, l’activité des neurones du LC au moment de l’appui choisi à

l’essai N avait pour conséquence un ajustement des choix basés sur l’effort à l’essai N+1, c’est-

à-dire qu’une activation plus forte pour l’appui d’un essai conduisait à des choix plus sensibles

Résumé en français

324

à l’effort à l’essai suivant. Enfin, nous avons mis en évidence que les neurones du LC

s’activaient après une erreur d’exécution (force insuffisante ou maintenue pendant une durée

trop faible). Nous avons constaté que cet effet pourrait en plus être associé à un codage de

l’erreur de prédiction de la récompense, même si nos résultats demandent à être confirmés.

Au bilan, de manière cohérente à travers trois taches comportementales de

compromis effort-récompense, nous avons montré que les neurones du LC modulent leur

activité en fonction de la difficulté subjective des processus en cours (déclenchement et

exécution des actions). De plus, la seconde étude a permis de préciser la relation de ces

activations avec la sensibilité à l’effort et la mobilisation d’effort physique et mental. Nous

avons montré qu’au sein d’un essai, les augmentations d’activité du LC précèdent des

diminutions de sensibilité à l’effort (augmentation de la fréquence de choix des options

difficiles, diminution du temps de réaction, augmentation de la force exercée). D’autre part,

nous avons montré que les neurones du LC répondent aux erreurs d’exécution, et que les

augmentations d’activité au moment de l’exécution des actions sont associées à des

modifications du compromis effort-récompense pour l’essai suivant, avec un poids plus

important pour l’effort.

Dans la dernière partie du manuscrit, plusieurs points sont discutés. Tout d’abord, la

dynamique des réponses du LC au cours des tâches cognitives étudiées dans le cadre ma thèse

ainsi que dans le cadre d’études réalisées dans le passé semblent compatibles avec le fait que

les neurones répondent à la fin des processus décisionnels, qu’il s’agisse de décisions de

s’engager dans des actions, de choix entre des options, ou de déclenchement d’actions

(décisions de s'engager : Bouret et al., 2012; Bouret and Richmond, 2009; Jahn et al., 2020;

Varazzani et al., 2015; déclenchement des actions: Bouret et al., 2012; Bouret and Richmond,

2015, 2009; Bouret and Sara, 2004; Clayton et al., 2004; Varazzani et al., 2015)..

Par ailleurs, le codage de la difficulté subjective par les neurones du LC, ainsi que le

potentiel codage de l’effort engagé, résonnent aves des études de pupillométrie qui ont mis

en évidence un lien entre effort physique et dilatation pupillaire (Zénon et al., 2014), et avec

des études d’imagerie calcique dans le locus coeruleus qui ont montré un lien entre l’activité

des neurones du LC et l’intensité du contrôle inhibiteur engagé dans une tâche de Posner (Bari

et al., 2020). Ce codage de l’effort pourrait provenir d’afférences du système nerveux

sympathique, et en particulier du noyau Paragigantocellularis (Aston-Jones et al., 1991, 1986;

Résumé en français

325

Ennis and Aston-Jones, 1988, 1986; Van Bockstaele and Aston-Jones, 1995), bien que leur

apparition semble tardive vu la vitesse de conduction rapide entre le noyau

Paragigantocellularis et le LC (Ennis and Aston-Jones, 1988). Il pourrait aussi provenir

d’afférences du cortex cingulaire antérieur, dont l’activité reflète l’effort et la difficulté dans

des tâches d’effort physique et mental (Croxson et al., 2009; Forstmann et al., 2006; Hillman

and Bilkey, 2010; Kennerley et al., 2009; Mars et al., 2005; Morris et al., 2019; Shenhav et al.,

2014; Walton et al., 2004; Yoshida and Ishii, 2006).

D’autre part, nous avons pu confirmer qu’au moment de la présentation d’indices

visuels informant sur les paramètres de l’essai à venir, les neurones du LC ne codent que les

paramètres de la récompense (taille et délai) et pas l’effort physique proposé (Bouret et al.,

2012; Bouret and Richmond, 2015; Jahn et al., 2020; Varazzani et al., 2015). Ces éléments

indiquent que le LC ne peut probablement pas coder la valeur subjective des options

impliquant de l’effort, mais qu’il code les paramètres de la récompense (taille, délai, et

probablement incertitude). De fait, l’activité liée aux indices visuels ne peut pas en elle-même

expliquer la relation causale entre activité du LC et les choix basés sur l’effort (Borderies et al.,

2020). Toutefois, dans la seconde étude, nous avons pu proposer un mécanisme par lequel

l’activité des neurones influencent ces choix, puisque nous avons montré que les

augmentations d’activité précédaient des diminutions de la difficulté subjective. Par ailleurs,

nous avons montré que l’activité des neurones du LC est compatible avec un suivi de l’effort

engagé et de ses conséquences (erreurs), et pourrait participer à l’ajustement des choix futurs

en fonction de l’effort engagé par le passé. Ce rôle pourrait faire intervenir les connections

entre le cortex cingulaire antérieur et le LC (Schwarz et al., 2015; Shenhav et al., 2013). D’autre

part, la diminution de la difficulté subjective des processus impliquant de l’effort pourrait être

liée aux effets de la NA sur le métabolisme des aires cérébrales cibles et/ou à des effets de

synergie entre le glutamate présent en grande quantité dans les aires les plus actives et la NA.

Les effets de promotion de l’effort et de la variabilité comportementale par le LC

(Borderies et al., 2020; Dubois et al., 2021; Jahn et al., 2018; Kane et al., 2017) pourraient

sembler indépendants, pourtant, si le LC intervient dans la régulation essai par essai de la

difficulté subjective et de l’effort engagé à l’essai suivant, les variations de l’activité des

neurones pourraient au moins en partie expliquer le bruit dans les choix.

Pour finir, nous avons montré dans la première étude que les neurones du LC sont

hétérogènes sur le plan du taux de décharge, et que le taux de décharge de base prédit la

326

force des réponses aux évènements et la force de la modulation par les paramètres de la

tâches et le comportement. Nous ne pouvons pas exclure que les effets multiples de l’activité

du LC, sur l’effort physique, mental et la variabilité dans les choix pourraient provenir de

l’activité de différents modules de ce noyau. Cette idée est cohérente avec les différents effets

observés en fonction du dosage des agents pharmacologiques utilisés pour inhiber l’activité

du LC (Borderies et al., 2020; Jahn et al., 2018). Il serait en conséquence intéressant pour de

futures études de disséquer les effets des différentes projections du LC.

327

Annex: Pharmacological evidence for the implication

of noradrenaline in effort, Borderies et al., 2020

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

*

345

346

347

348

349

350

351

352

353