Dopamine, Reinforcement Learning, and Addiction

Abstract

Dopamine is intimately linked with the modes of action of drugs of addiction. However, although its role in the initiation of drug abuse seems relatively uncomplicated, its possible involvement in the development of compulsive drug taking, and indeed vulnerability and relapse, is less clear. We first describe a modern reinforcement learning view of affective control, focusing on the roles for dopamine. We then use this as a framework to sketch various notions of the neuromodulator's possible participation in initiation and compulsion. We end with some pointers towards future theoretical developments.

References

• 1 Ahmed SH, Koob GF. Transition from moderate to excessive drug intake: change in hedonic set point.  Science. 1998;  282 ((5387)) 298-300
• 2 Ainslie G. Breakdown of Will. Cambridge University Press, 3 2001
• 3 Anagnostaras SG, Schallert T, Robinson TE. Memory processes governing amphetamine-induced psychomotor sensitization.  Neuropsychopharmacology. 2002;  26 ((6)) 703-715
• 4 Aragona BJ, Cleaveland NA, Stuber GD. et al . Preferential enhancement of dopamine transmission within the nucleus accumbens shell by cocaine is attributable to a direct increase in phasic dopamine release events.  J Neurosci. 2008;  28 ((35)) 8821-8831
• 5 Baird L. Advantage updating. Technical Report WL-TR-93-1146. Wright-Patterson Air Force Base, OH 1993
• 6 Balleine BW. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits.  Physiol Behav. 2005;  86 ((5)) 717-730
• 7 Barto A. Adaptive critics and the basal ganglia. In Houk J, Davis J and Beiser D eds. Models of Information Processing in the Basal Ganglia. Cambridge, MA, MIT Press 1995: 215-232
• 8 Barto A, Sutton R, Anderson C. Neuron-like adaptive elements that that can learn difficult control problems. IEEE Trans.  on Systems Man and Cybernetics. 1983;  13 ((5)) 835-846
• 9 Berridge KC. Pleasures of the brain.  Brain Cogn. 2003;  52 ((1)) 106-128
• 10 Berridge KC. The debate over dopamine's role in reward: the case for incentive salience.  Psychopharmacology (Berl). 2007;  191 ((3)) 391-431
• 11 Blanchard DC, Blanchard RJ. Ethoexperimental approaches to the biology of emotion.  Annu Rev Psychol. 1988;  39 43-68
• 12 Bolles RC. Species-specific defense reactions and avoidance learning.  Psychol Rev. 1970;  77 32-48
• 13 Breland K, Breland M. The misbehavior of organisms.  American Psychologist. 1961;  16 ((9)) 681-684
• 14 Breland K, Breland M. Animal behavior. Macmillan New York 1966
• 15 Cardinal RN, Everitt BJ. Neural and psychological mechanisms underlying appetitive learning: links to drug addiction.  Curr Opin Neurobiol. 2004;  14 ((2)) 156-162
• 16 Cheer JF, Wassum KM, Heien MLAV. et al . Cannabinoids enhance subsecond dopamine release in the nucleus accumbens of awake rats.  J Neurosci. 2004;  24 ((18)) 4393-4400
• 17 Dalley JW, Fryer TD, Brichard L. et al . Nucleus accumbens d2/3 receptors predict trait impulsivity and cocaine reinforcement.  Science. 2007;  315 ((5816)) 1267-1270
• 18 Daw N. Reinforcement learning models of the dopamine system and their behavioral implications. PhD thesis, Computer Science Dept, CMU 2003
• 19 Daw ND, Kakade S, Dayan P. Opponent interactions between serotonin and dopamine.  Neural Netw. 2002;  15 ((4–6)) 603-616
• 20 Daw ND, Niv Y, Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control.  Nat Neurosci. 2005;  8 ((12)) 1704-1711
• 21 Daw ND, O’Doherty JP, Dayan P. et al . Cortical substrates for exploratory decisions in humans.  Nature. 2006;  441 ((7095)) 876-879
• 22 Day JJ, Roitman MF, Wightman RM. et al . Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens.  Nat Neurosci. 2007;  10 ((8)) 1020-1028
• 23 Dayan P. The role of value systems in decision-making. In Engel C and Singer W, eds. Better than Conscious. Ernst Strüngmann Forum, MIT Press: Cambridge MA 2008: 51-70
• 24 Dayan P, Balleine BW. Reward, motivation, and reinforcement learning.  Neuron. 2002;  36 ((2)) 285-298
• 25 Dayan P, Huys Q. Serotonin and affective control. Annual Review of Neuroscience. 2009; 
• 26 Dayan P, Huys QJM. Serotonin, inhibition, and negative mood.  PLoS Comput Biol. 2008;  4 ((2)) e4
• 27 Dayan P, Niv Y, Seymour B. et al . The misbehavior of value and the discipline of the will.  Neural Netw. 2006;  19 ((8)) 1153-1160
• 28 Dayan P, Seymour B. Values and actions in aversion. In: Glimcher P, Camerer C, Poldrack R and Fehr E, eds. Neuroeconomics: Decision making and the brain. New York, NY: Academic Press, New York, NY 2008: 175-191
• 29 Chiara G Di. Nucleus accumbens shell and core dopamine: differential role in behavior and addiction.  Behav Brain Res. 2002;  137 ((1–2)) 75-114
• 30 Chiara G Di, Imperato A. Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats.  Proc Natl Acad Sci USA. 1988;  85 ((14)) 5274-5278
• 31 Dickinson A, Balleine B. The role of learning in motivation. In: Gallistel C, ed. Stevens’ handbook of experimental psychology, volume. Wiley, New York, NY 2002: 497-5
• 32 Dickinson A, Smith J, Mirenowicz J. Dissociation of pavlovian and instrumental incentive learning under dopamine antagonists.  Behav Neurosci. 2000;  114 ((3)) 468-483
• 33 Everitt BJ, Belin D, Economidou D. et al . Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction.  Philos Trans R Soc Lond B Biol Sci. 2008;  363 ((1507)) 3125-3135
• 34 Everitt BJ, Hutcheson DM, Ersche KD. et al . The orbital prefrontal cortex and drug addiction in laboratory animals and humans.  Ann N Y Acad Sci. 2007;  1121 576-597
• 35 Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion.  Nat Neurosci. 2005;  8 ((11)) 1481-1489
• 36 Fields HL, Heinricher MM, Mason P. Neurotransmitters in nociceptive modulatory circuits.  Annu Rev Neurosci. 1991;  14 219-245
• 37 Flagel SB, Akil H, Robinson TE. Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction.  Neuropharmacology. 2008; 
• 38 Frank MJ. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism.  J Cogn Neurosci. 2005;  17 ((1)) 51-72
• 39 Frank MJ. Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making.  Neural Netw. 2006;  19 ((8)) 1120-1136
• 40 Frank MJ, Samanta J, Moustafa AA. et al . Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism.  Science. 2007;  318 ((5854)) 1309-1312
• 41 Frank MJ, Seeberger LC, O’reilly RC. By carrot or by stick: cognitive reinforcement learning in parkinsonism.  Science. 2004;  306 ((5703)) 1940-1943
• 42 Friston KJ, Tononi G, Reeke GN. et al . Value-dependent selection in the brain: simulation in a synthetic neural model.  Neuroscience. 1994;  59 ((2)) 229-243
• 43 Fuchs RA, Evans KA, Parker MC. et al . Differential involvement of the core and shell subregions of the nucleus accumbens in conditioned cue-induced reinstatement of cocaine seeking in rats.  Psychopharmacology (Berl). 2004;  176 ((3–4)) 459-465
• 44 Gallistel CR. The role of the dopaminergic projections in MFB self-stimulation.  Behav Brain Res. 1986;  20 ((3)) 313-321
• 45 Garavan H, Pankiewicz J, Bloom A. et al . Cue-induced cocaine craving: neuroanatomical specificity for drug users and drug stimuli.  Am J Psychiatry. 2000;  157 ((11)) 1789-1798
• 46 Gittins JC. Multi-Armed Bandit Allocation Indices (Wiley Interscience Series in Systems and Optimization). John Wiley & Sons Inc. 3 1989
• 47 Goto Y, Otani S, Grace AA. The yin and yang of dopamine release: a new perspective.  Neuropharmacology. 2007;  53 ((5)) 583-587
• 48 Grossberg S. Processing of expected and unexpected events during conditioning and attention: a psychophysiological theory.  Psychol Rev. 1982;  89 ((5)) 529-572
• 49 Grossberg S. Some normal and abnormal behavioral syndromes due to transmitter gating of opponent processes.  Biol Psychiatry. 1984;  19 ((7)) 1075-1118
• 50 Haber SN, Fudge JL, MacFarland NR. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum.  J Neurosci. 2000;  20 ((6)) 2369-2382
• 51 Haruno M, Kawato M. Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning.  Neural Netw. 2006;  19 ((8)) 1242-1254
• 52 Heien MLAV, Khan AS, Ariansen JL. et al . Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats.  Proc Natl Acad Sci USA. 2005;  102 ((29)) 10023-10028
• 53 Heinz A, Siessmeier T, Wrase J. et al . Correlation of alcohol craving with striatal dopamine synthesis capacity and d2/3 receptor availability: a combined [18f]dopa and [18f]dmfp pet study in detoxified alcoholic patients.  Am J Psychiatry. 2005;  162 ((8)) 1515-1520
• 54 Heinz A, Siessmeier T, Wrase J. et al . Correlation between dopamine d(2) receptors in the ventral striatum and central processing of alcohol cues and craving.  Am J Psychiatry. 2004;  161 ((10)) 1783-1789
• 55 Hernandez G, Hamdani S, Rajabi H. et al . Prolonged rewarding stimulation of the rat medial forebrain bundle: neurochemical and behavioral consequences.  Behav Neurosci. 2006;  120 ((4)) 888-904
• 56 Hershberger W. An approach through the looking-glass.  Learning & Behavior. 1986;  14 ((4)) 443-451
• 57 Hnasko TS, Sotak BN, Palmiter RD. Morphine reward in dopamine-deficient mice.  Nature. 2005;  438 ((7069)) 854-857
• 58 Horvitz JC, Stewart T, Jacobs BL. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat.  Brain Res. 1997;  759 ((2)) 251-258
• 59 Hyman SE. Addiction: a disease of learning and memory.  Am J Psychiatry. 2005;  162 ((8)) 1414-1422
• 60 Hyman SE, Malenka RC, Nestler EJ. Neural mechanisms of addiction: the role of reward-related learning and memory.  Annu Rev Neurosci. 2006;  29 565-598
• 61 Ikemoto S. Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex.  Brain Res Rev. 2007;  56 ((1)) 27-78
• 62 Joel D, Weiner I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum.  Neuroscience. 2000;  96 ((3)) 451-474
• 63 Johnson J, Li W, Li J. et al . A computational model of learned avoidance behavior in a one-way avoidance experiment.  Adaptive Behavior. 2002;  9 ((2)) 91-104
• 64 Kakade S, Dayan P. Dopamine: generalization and bonuses.  Neural Netw. 2002;  15 ((4–6)) 549-559
• 65 Kalivas PW, MacFarland K. Brain circuitry and the reinstatement of cocaine-seeking behavior.  Psychopharmacology (Berl). 2003;  168 ((1–2)) 44-56
• 66 Kalivas PW, O’Brien C. Drug addiction as a pathology of staged neuroplasticity.  Neuropsychopharmacology. 2008;  33 ((1)) 166-180
• 67 Kamin LJ. Predictability, surprise, attention and conditioning. In Campbell BA, Church RM, eds. Punishment and aversive behavior. Appleton-Century-Crofts, New York 1969
• 68 Kelley AE, Berridge KC. The neuroscience of natural rewards: relevance to addictive drugs.  J Neurosci. 2002;  22 ((9)) 3306-3311
• 69 Koob GF. Drugs of abuse: anatomy, pharmacology and function of reward pathways.  Trends Pharmacol Sci. 1992;  13 ((5)) 177-184
• 70 Koob GF, Moal ML. Addiction and the brain antireward system.  Annu Rev Psychol. 2008;  59 29-53
• 71 Koob GF, Moal ML. Neurobiological mechanisms for opponent motivational processes in addiction.  Philos Trans R Soc Lond B Biol Sci. 2008;  363 ((1507)) 3113-3123
• 72 Kumar P, Waiter G, Ahearn T. et al . Abnormal temporal difference reward-learning signals in major depression.  Brain. 2008;  131 ((Pt 8)) 2084-2093
• 73 Marr D. Vision: A computational investigation into the human representation and processing of visual information. Henry Holt and Co., Inc. New York, NY, USA 1982
• 74 Mazzoni P, Hristova A, Krakauer JW. Why don’t we move faster? Parkinson's disease, movement vigor, and implicit motivation.  J Neurosci. 2007;  27 ((27)) 7105-7116
• 75 MacClure SM, Daw ND, Montague PR. A computational substrate for incentive salience.  Trends Neurosci. 2003;  26 ((8)) 423-428
• 76 MacFarland K, Kalivas PW. The circuitry mediating cocaine-induced reinstatement of drug-seeking behavior.  J Neurosci. 2001;  21 ((21)) 8655-8663
• 77 MacNaughton N, Corr PJ. A two-dimensional neuropsychology of defense: fear/anxiety and defensive distance.  Neurosci Biobehav Rev. 2004;  28 ((3)) 285-305
• 78 Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive hebbian learning.  J Neurosci. 1996;  16 ((5)) 1936-1947
• 79 Morris G, Nevet A, Arkadir D. et al . Midbrain dopamine neurons encode decisions for future action.  Nat Neurosci. 2006;  9 ((8)) 1057-1063
• 80 Moutoussis M, Bentall RP, Williams J. et al . A temporal difference account of avoidance learning.  Network. 2008;  19 ((2)) 137-160
• 81 Moutoussis M, Williams J, Dayan P. et al . Persecutory delusions and the conditioned avoidance paradigm: towards an integration of the psychology and biology of paranoia.  Cognit Neuropsychiatry. 2007;  12 ((6)) 495-510
• 82 Mowrer O. On the dual nature of learning: A reinterpretation of conditioning and problem solving.  Harvard Educational Review. 1947;  17 ((2)) 102-150
• 83 Nieoullon A, Coquerel A. Dopamine: A key regulator to adapt action, emotion, motivation and cognition.  Current Opinion in Neurology. 2003;  16 S3
• 84 Niv Y, Daw ND, Joel D. et al . Tonic dopamine: opportunity costs and the control of response vigor.  Psychopharmacology (Berl). 2007;  191 ((3)) 507-520
• 85 Owesson-White CA, Cheer JF, Beyene M. et al . Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation.  Proc Natl Acad Sci USA. 2008;  105 ((33)) 11957-11962
• 86 Panksepp J. Affective Neuroscience. Oxford University Press, New York, NY 1998
• 87 Panlilio LV, Thorndike EB, Schindler CW. Blocking of conditioning to a cocaine-paired stimulus: testing the hypothesis that cocaine perpetually produces a signal of larger-than-expected reward.  Pharmacol Biochem Behav. 2007;  86 ((4)) 774-777
• 88 Peciña S, Berridge KC. Central enhancement of taste pleasure by intraventricular morphine.  Neurobiology (Bp). 1995;  3 ((3–4)) 269-280
• 89 Pessiglione M, Seymour B, Flandin G. et al . Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.  Nature. 2006;  442 ((7106)) 1042-1045
• 90 Phillips PEM, Stuber GD, Heien MLAV. et al . Subsecond dopa-mine release promotes cocaine seeking.  Nature. 2003;  422 ((6932)) 614-618
• 91 Redish AD. Addiction as a computational process gone awry.  Science. 2004;  306 ((5703)) 1944-1947
• 92 Redish AD, Jensen S, Johnson A. Addiction as vulnerabilities in the decision process.  Behav Brain Sci. 2008;  31 ((4)) 461-487
• 93 Redish AD, Jensen S, Johnson A. et al . Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling.  Psychol Rev. 2007;  114 ((3)) 784-805
• 94 Rescorla R, Wagner A. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement.  Classical conditioning II: Current research and theory. 1972;  64-99
• 95 Reynolds JN, Hyland BI, Wickens JR. A cellular mechanism of reward-related learning.  Nature. 2001;  413 ((6851)) 67-70
• 96 Reynolds SM, Berridge KC. Positive and negative motivation in nucleus accumbens shell: bivalent rostrocaudal gradients for GABA-elicited eating, taste “liking/disliking” reactions, place preference/avoidance, and fear.  J Neurosci. 2002;  22 ((16)) 7308-7320
• 97 Reynolds SM, Berridge KC. Glutamate motivational ensembles in nucleus accumbens: rostrocaudal shell gradients of fear and feeding.  Eur J Neurosci. 2003;  17 ((10)) 2187-2200
• 98 Reynolds SM, Berridge KC. Emotional environments retune the valence of appetitive versus fearful functions in nucleus accumbens.  Nat Neurosci. 2008;  11 ((4)) 423-425
• 99 Robinson TE, Berridge KC. Incentive-sensitization and addiction.  Addiction. 2001;  96 ((1)) 103-114
• 100 Robinson TE, Berridge KC. Addiction.  Annu Rev Psychol. 2003;  54 25-53
• 101 Robinson TE, Berridge KC. The incentive sensitization theory of addiction: some current issues.  Philos Trans R Soc Lond B Biol Sci. 2008;  363 ((1507)) 3137-3146
• 102 Roesch MR, Calu DJ, Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards.  Nat Neurosci. 2007;  10 ((12)) 1615-1624
• 103 Salamone JD, Correa M, Farrar A. et al . Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits.  Psychopharmacology (Berl). 2007;  191 ((3)) 461-482
• 104 Salamone JD, Correa M, Mingote S. et al . Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse.  J Pharmacol Exp Ther. 2003;  305 ((1)) 1-8
• 105 Satoh T, Nakai S, Sato T. et al . Correlated coding of motivation and outcome of decision by dopamine neurons.  J Neurosci. 2003;  23 ((30)) 9913-9923
• 106 Schmajuk N, Zanutto B. Escape, avoidance, and imitation: A neural network approach.  Adaptive Behavior. 1997;  6 ((1)) 63
• 107 Schultz W. Getting formal with dopamine and reward.  Neuron. 2002;  36 ((2)) 241-263
• 108 Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward.  Science. 1997;  275 ((5306)) 1593-1599
• 109 See RE, Elliott JC, Feltenstein MW. The role of dorsal vs. ventral striatal pathways in cocaine-seeking behavior after prolonged abstinence in rats.  Psychopharmacology (Berl). 2007;  194 ((3)) 321-331
• 110 Solomon RL, Corbit JD. An opponent-process theory of motivation. i. temporal dynamics of affect.  Psychol Rev. 1974;  81 ((2)) 119-145
• 111 Spanagel R, Herz A, Shippenberg TS. Opposing tonically active endogenous opioid systems modulate the mesolimbic dopaminergic pathway.  Proc Natl Acad Sci USA. 1992;  89 ((6)) 2046-2050
• 112 Spanagel R, Weiss F. The dopamine hypothesis of reward: past and current status.  Trends Neurosci. 1999;  22 ((11)) 521-527
• 113 Stewart J. Psychological and neural mechanisms of relapse.  Philos Trans R Soc Lond B Biol Sci. 2008;  363 ((1507)) 3147-3158
• 114 Stinus L, Cador M, Moal ML. Interaction between endogenous opioids and dopamine within the nucleus accumbens.  Ann N Y Acad Sci. 1992;  654 254-273
• 115 Stinus L, Koob GF, Ling N. et al . Locomotor activation induced by infusion of endorphins into the ventral tegmental area: evidence for opiate-dopamine interactions.  Proc Natl Acad Sci USA. 1980;  77 ((4)) 2323-2327
• 116 Suri RE, Schultz W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task.  Neuroscience. 1999;  91 ((3)) 871-890
• 117 Sutton R. Learning to predict by the methods of temporal differences.  Machine Learning. 1988;  3 ((1)) 9-44
• 118 Sutton RS, Barto AG. Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning).  The MIT Press, 3. 1998; 
• 119 Tzschentke TM. Measuring reward with the conditioned place preference paradigm: a comprehensive review of drug effects, recent progress and new issues.  Prog Neurobiol. 1998;  56 ((6)) 613-672
• 120 Verdejo-Garcia A, Lawrence AJ, Clark L. Impulsivity as a vulnerability marker for substance-use disorders: review of findings from high-risk research, problem gamblers and genetic association studies.  Neurosci Biobehav Rev. 2008;  32 ((4)) 777-810
• 121 Volkow ND, Fowler JS, Wang G-J. et al . Dopamine in drug abuse and addiction: results from imaging studies and treatment implications.  Mol Psychiatry. 2004;  9 ((6)) 557-569
• 122 Volkow ND, Fowler JS, Wang G-J. et al . Dopamine in drug abuse and addiction: results of imaging studies and treatment implications.  Arch Neurol. 2007;  64 ((11)) 1575-1579
• 123 Watkins C. Learning from Delayed Rewards. PhD thesis, University of Cambridge 1989
• 124 Wickens J. Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model.  J Neural Transm Gen Sect. 1990;  80 ((1)) 9-31
• 125 Wickens JR, Reynolds JNJ, Hyland BI. Neural mechanisms of reward-related motor learning.  Curr Opin Neurobiol. 2003;  13 ((6)) 685-690
• 126 Williams DR, Williams H. Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement.  J Exp Anal Behav. 1969;  12 ((4)) 511-520
• 127 Williams J, Dayan P. Dopamine, learning, and impulsivity: a biological account of attention-deficit/hyperactivity disorder.  J Child Adolesc Psychopharmacol. 2005;  15 ((2)) 160-179 , discussion 157–159
• 128 Williams R. Simple statistical gradient-following algorithms for connectionist reinforcement learning.  Reinforcement Learning. 1992;  8 229-256
• 129 Wise RA. Opiate reward: sites and substrates.  Neurosci Biobehav Rev. 1989;  13 ((2–3)) 129-133
• 130 Wise RA. Forebrain substrates of reward and motivation.  J Comp Neurol. 2005;  493 ((1)) 115-121
• 131 Wise RA, Bozarth MA. A psychomotor stimulant theory of addiction.  Psychol Rev. 1987;  94 ((4)) 469-492
• 132 Wrase J, Schlagenhauf F, Kienast T. et al . Dysfunction of reward processing correlates with alcohol craving in detoxified alcoholics.  Neuroimage. 2007;  35 ((2)) 787-794
• 133 Wyvell CL, Berridge KC. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward “wanting” without enhanced “liking” or response reinforcement.  J Neurosci. 2000;  20 ((21)) 8122-8130
• 134 Yu AJ, Dayan P. Uncertainty, neuromodulation, and attention.  Neuron. 2005;  46 ((4)) 681-692

1 Note that this is equivalent to the first D(S_l) term in Eq. 4 of [91] (P1945).

Correspondence