Handbook of Reward and Decision Making

Jean-Claude Dreher, Leon Tremblay0123746205, 978-0-12-374620-7

Table of contents :
Cover Page
……Page 1
Copyright Page……Page 2
Preface……Page 3
List of contributors……Page 7
Monkey Anatomical and Electrophysiological Studies on Reward and Decision Making……Page 10
Introduction……Page 11
Prefrontal cortex……Page 12
Special features of the ventral striatum……Page 15
Cortical projections to the ventral striatum……Page 16
Thalamic projections to the ventral striatum……Page 20
Ventral pallidum……Page 21
The midbrain DA neurons……Page 23
Afferent projections……Page 24
The striato-nigro-striatal network……Page 25
The thalamus……Page 26
Complex network features of the reward circuit……Page 28
References……Page 29
Key points……Page 36
Definitions and foundations……Page 37
Temporal contiguity……Page 38
Prediction error……Page 39
Models……Page 41
Views from microeconomic theory……Page 42
Electrophysiological characteristics of dopamine neurons and afferent inputs……Page 44
Mechanisms of dopamine transmission……Page 45
Aversive events……Page 46
Primary rewards and reward-predicting stimuli……Page 47
Value coding……Page 49
Reward uncertainty……Page 50
Acknowledgements……Page 52
References……Page 53
Key points……Page 58
Introduction……Page 59
The relation between the cortex and the basal ganglia: input and output……Page 60
The heterogeneous projections of the ventral striatum……Page 62
The dopamine pathway……Page 63
The direct and indirect pathways of the ventral striatum……Page 65
A tool to study the neuronal basis of reward processes, motivation, and decision-making……Page 66
The neuronal activity in the striatum……Page 69
Reward processing in ventral striatum: goal of action and reinforcer……Page 70
Motivation in ventral striatum: from expectation to the goal determination……Page 72
Decision-making in ventral striatum: guiding choice and building new associations……Page 73
Dysfunction in ventral striatum and perturbation of different motivational domains……Page 75
References……Page 79
Introduction……Page 85
Reward-related sustained neuronal activity in the LPFC……Page 86
Coding the response outcome in prefrontal neurons……Page 90
Integration of motivational and cognitive information in prefrontal neurons……Page 92
Prefrontal neurons related to perceptual decision-making and forced choice discrimination……Page 95
Implicit decision-making and prefrontal neuronal activity……Page 96
Decision-making and multiple reward signals in the primate brain……Page 98
References……Page 100
Introduction……Page 103
A connectional and functional framework……Page 105
Taste reward……Page 106
Olfactory reward……Page 109
Oral texture and temperature reward……Page 110
Somatosensory and temperature inputs to the orbitofrontal cortex, and affective value……Page 111
Visual inputs to the orbitofrontal cortex, and visual stimulus–reinforcement association learning and reversal……Page 112
Reward prediction error neurons……Page 115
Social reinforcers such as face and voice expression……Page 117
Top-down effects of cognition and attention on the reward value of affective stimuli……Page 118
Individual differences in reward processing and emotion……Page 120
Beyond the representation of reward value to choice decision-making……Page 122
Cortical networks that make choices between rewards: is there a common currency?……Page 125
A computational basis for stimulus–reinforcer association learning and reversal in the orbitofrontal cortex involving conditional reward neurons and negative reward prediction error neurons……Page 127
Acknowledgements……Page 129
References……Page 130
fMRI studies on reward and decision making……Page 140
Basic computations involved in decision making……Page 141
Animal electrophysiology on prediction error……Page 143
Human neuroimaging studies on prediction error……Page 144
Computing various uncertainty signals in the brain……Page 147
Discounting the value of costly options in delay and effort discounting studies……Page 150
The concept of common neural currency……Page 151
Common and distinct brain regions involved in processing primary and secondary rewards……Page 152
Distinguishing two brain systems involved in choosing between different types of rewards……Page 155
Goal-value and decision-value signals……Page 156
Variation in dopamine genes influence reward processing……Page 158
Conclusions……Page 161
References……Page 162
Key points……Page 168
Introduction……Page 169
General neuroanatomy of reward processing……Page 170
Orbital and medial prefrontal cortex……Page 172
Dorsal and ventral striatum……Page 173
Dissociating OFC and striatum contributions to subjective value calculations……Page 175
Decision-making computations that contribute to valuation……Page 176
Action selection……Page 177
Risk: uncertainty and ambiguity……Page 178
Social interactions……Page 180
Future directions and open questions……Page 181
References……Page 182
Key points……Page 188
Introduction……Page 189
Sensory evidence representation……Page 191
Accumulation of sensory evidence and decision formation……Page 192
Perceptual decision making in humans……Page 193
Integration of sensory evidence and formation of the decision variable……Page 194
Uncertainty, attention, and task difficulty……Page 195
The role of the motor system……Page 196
Performance and error monitoring……Page 197
Single-trial EEG reveals temporal characteristics of decision making……Page 198
The role of synchronized activity in perceptual decision making……Page 202
Spatiotemporal characterization of decision making by integrating EEG with fMRI……Page 203
Neuromodulation of perceptual decision making……Page 207
Valuation-based/reward-based decision making……Page 208
References……Page 209
Introduction……Page 216
Activity related to the anticipation of the outcome……Page 217
Orbitofrontal cortex……Page 219
Activity related to the occurrence of the outcome……Page 220
Lateral prefrontal cortex……Page 221
Dissociation of the role of OFC, ACC, and dorsolateral prefrontal cortex (DLPFC) in reward-based decision-making……Page 222
References……Page 226
Introduction……Page 232
An example experiment……Page 234
Neural implementation……Page 236
Variations on TD(0)……Page 238
Computational neuroimaging……Page 239
The BOLD response……Page 240
Dopamine and the BOLD response……Page 241
Reward prediction errors in dopamine target regions……Page 242
Direct measurement in the human VTA……Page 245
Conclusions……Page 246
References……Page 247
Brain Disorders Involving Dysfunctions of Reward and Decision Making Processes……Page 251
Key points……Page 252
Theories of altered reward processing in schizophrenia……Page 253
Dopamine, schizophrenia, and reinforcement learning……Page 255
The possible importance of glutamate……Page 256
Studies of reward processing/reinforcement learning in psychosis: behavioral studies……Page 257
Studies of reward processing/reinforcement learning in psychosis: neuroimaging studies……Page 258
Delusions……Page 260
Anhedonia……Page 263
Summary……Page 265
References……Page 266
Key points……Page 271
Preamble: idiopathic Parkinson’s disease and the monkey MPTP model……Page 272
Energization……Page 273
Decision making……Page 275
Learning……Page 276
Theories: what is the basal ganglia dysfunction induced by dopamine depletion?……Page 277
Decoupling……Page 278
Disinhibition……Page 279
Reinforcement……Page 280
Explanations: how can basal ganglia dysfunction account for reward-seeking deficits?……Page 281
References……Page 282
Key points……Page 290
Brief history of neuropsychology……Page 291
Phineas Gage……Page 292
Somatic marker hypothesis……Page 293
Emotion regulation……Page 295
Making decisions……Page 297
Social interactions and moral judgments……Page 298
The aging VMPC……Page 300
Conclusions……Page 303
References……Page 304
Genetic and Hormonal Influences on the Reward System……Page 306
Key points……Page 307
Introduction……Page 308
Effects of testosterone on cognitive capacities……Page 309
Effects of testosterone on reward processing and decision making……Page 310
Behavioral studies……Page 311
Testosterone effects on economic decision making……Page 312
Effects of estradiol and progesterone on cognitive capacities……Page 313
Menstrual cycle effects on reward processing and decision making……Page 314
Dissociating the roles of estrogen and progesterone on reward processing……Page 317
Dopaminergic dysfunction in aging……Page 318
Hormonal replacement therapy (HRT) in aging men and women……Page 320
Actions of gonadal steroid hormones in the brain……Page 321
Conclusions……Page 322
References……Page 323
Key points……Page 333
CNS arousal underlying the activation of motivated behaviors……Page 334
Estrogens and CNS arousal……Page 336
Sources of sexual arousal……Page 339
Future directions……Page 340
References……Page 341
Key points……Page 342
Dopaminergic reward system and addictive behavior……Page 343
Pathological gambling: model for addiction without a substance……Page 344
Individual differences and vulnerability to addictive behaviors……Page 347
Candidate gene studies of dopamine transmission: DAT and COMT……Page 348
Imaging genetics of the DAT/COMT interaction and reward sensitivity……Page 349
Looking forward……Page 350
References……Page 352
Introduction……Page 358
COMT effects on brain activation elicited by working memory and executive control……Page 359
COMT effects and negative mood states……Page 361
COMT effects in the brain reward system……Page 363
References……Page 364
Computational Models of the Reward System and Decision Making……Page 370
Introduction……Page 371
Neurobiology of decision processes in the cortex……Page 372
Optimal stopping criterion for two alternatives……Page 373
Comparison with experimental data……Page 375
Optimal stopping criterion for multiple alternatives……Page 377
Neural implementation……Page 379
Comparison with experimental data……Page 380
Representing probabilities in integrators’ firing rate……Page 382
Incorporating prior probabilities……Page 383
Comparison with experimental data……Page 384
Open questions……Page 385
Appendix A: Diffusion model implements SPRT……Page 386
Appendix B: Diffusion model maximizes reward rate……Page 387
Appendix C: MSPRT……Page 388
Appendix D: STN and GP can compute the optimal conflict……Page 389
Appendix E: Basal ganglia model is unaffected by inhibition of integrators……Page 390
References……Page 391
Key points……Page 394
BG circuitry and function……Page 395
Basic model……Page 397
Empirical support for basic model……Page 399
PD and medication effects……Page 401
Pharmacological studies in healthy participants……Page 402
Complications and qualifications……Page 403
Genetics……Page 404
Subthalamic nucleus……Page 406
DA/NE interactions and attention deficit/hyperactivity disorder……Page 409
Instructional control……Page 410
Working memory and dorsolateral PFC……Page 412
References……Page 413
Key points……Page 421
Reward-based emotions: emotions as affective evaluations of outcomes……Page 422
Reward-based emotions implement learning……Page 426
What should adaptive learning do?……Page 427
Private and social rewards……Page 428
References……Page 430
Key points……Page 434
Inference……Page 436
Decision criteria……Page 437
Bayesian model and diffusion model……Page 438
Estimating sensory likelihoods on line……Page 439
Prediction in a motion discrimination task……Page 442
Reaction time task……Page 443
Fixed delay tasks……Page 444
Comparison of model predictions with experimental results……Page 446
Why does human behavior fit better the prediction of a diffusion model?……Page 447
Full Bayesian model……Page 448
Simplified Bayesian model……Page 449
References……Page 450
Key points……Page 452
Introduction……Page 453
The Multistep Risk Example……Page 455
Theoretical values of expected reward and risk……Page 456
One-step-ahead prediction risk and total reward risk……Page 463
Temporal discounting……Page 464
Discussion……Page 465
References……Page 466
A……Page 468
C……Page 469
D……Page 470
E……Page 471
F……Page 472
L……Page 473
M……Page 474
N……Page 475
P……Page 476
R……Page 477
T……Page 479
V……Page 480
W……Page 481