Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards (original) (raw)

References

  1. Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).
    Article CAS Google Scholar
  2. Schultz, W. Getting formal with dopamine and reward. Neuron 36, 241–263 (2002).
    Article CAS Google Scholar
  3. Dayan, P. & Balleine, B.W. Reward, motivation and reinforcement learning. Neuron 36, 285–298 (2002).
    Article CAS Google Scholar
  4. Day, J.J., Roitman, M.F., Wightman, R.M. & Carelli, R.M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020–1028 (2007).
    Article CAS Google Scholar
  5. Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024–1027 (1994).
    Article CAS Google Scholar
  6. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).
    Article CAS Google Scholar
  7. Tobler, P.N., Dickinson, A. & Schultz, W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 23, 10402–10410 (2003).
    Article CAS Google Scholar
  8. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).
    Article CAS Google Scholar
  9. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).
    Article CAS Google Scholar
  10. Montague, P.R., Dayan, P. & Sejnowski, T.J. A framework for mesencephalic dopamine systems based on predictive hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
    Article CAS Google Scholar
  11. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).
    Article CAS Google Scholar
  12. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269–280 (2004).
    Article CAS Google Scholar
  13. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).
    Article CAS Google Scholar
  14. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057–1063 (2006).
    Article CAS Google Scholar
  15. Kawagoe, R., Takikawa, Y. & Hikosaka, O. Reward-predicting activity of dopamine and caudate neurons—a possible mechanism of motivational control of saccadic eye movement. J. Neurophysiol. 91, 1013–1024 (2004).
    Article Google Scholar
  16. Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W. & Everitt, B.J. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292, 2499–2501 (2001).
    Article CAS Google Scholar
  17. Evenden, J.L. & Ryan, C.N. The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl.) 128, 161–170 (1996).
    Article CAS Google Scholar
  18. Herrnstein, R.J. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272 (1961).
    Article CAS Google Scholar
  19. Ho, M.Y., Mobini, S., Chiang, T.J., Bradshaw, C.M. & Szabadi, E. Theory and method in the quantitative analysis of “impulsive choice” behaviour: implications for psychopharmacology. Psychopharmacology (Berl.) 146, 362–372 (1999).
    Article CAS Google Scholar
  20. Mobini, S. et al. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl.) 160, 290–298 (2002).
    Article CAS Google Scholar
  21. Kahneman, D. & Tverskey, A. Choices, values and frames. Am. Psychol. 39, 341–350 (1984).
    Article Google Scholar
  22. Kalenscher, T. et al. Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task. Curr. Biol. 15, 594–602 (2005).
    Article CAS Google Scholar
  23. Lowenstein, G.E.J. Choice Over Time (Russel Sage Foundation, New York, 1992).
  24. Thaler, R. Some empirical evidence on dynamic inconsistency. Econ. Lett. 8, 201–207 (1981).
    Article Google Scholar
  25. Winstanley, C.A., Theobald, D.E., Cardinal, R.N. & Robbins, T.W. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24, 4718–4722 (2004).
    Article CAS Google Scholar
  26. Cardinal, R.N., Winstanley, C.A., Robbins, T.W. & Everitt, B.J. Limbic corticostriatal systems and delayed reinforcement. Ann. NY Acad. Sci. 1021, 33–50 (2004).
    Article Google Scholar
  27. Kheramin, S. et al. Effects of orbital prefrontal cortex dopamine depletion on intertemporal choice: a quantitative analysis. Psychopharmacology (Berl.) 175, 206–214 (2004).
    Article CAS Google Scholar
  28. Wade, T.R., de Wit, H. & Richards, J.B. Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology (Berl.) 150, 90–101 (2000).
    Article CAS Google Scholar
  29. Cardinal, R.N., Robbins, T.W. & Everitt, B.J. The effects of d-amphetamine, chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology (Berl.) 152, 362–375 (2000).
    Article CAS Google Scholar
  30. Roesch, M.R., Takahashi, Y., Gugsa, N., Bissonette, G.B. & Schoenbaum, G. Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude. J. Neurosci. 27, 245–250 (2007).
    Article CAS Google Scholar
  31. Roesch, M.R., Taylor, A.R. & Schoenbaum, G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509–520 (2006).
    Article CAS Google Scholar
  32. Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).
    Article CAS Google Scholar
  33. Kiyatkin, E.A. & Rebec, G.V. Heterogeneity of ventral tegmental area neurons: single-unit recording and iontophoresis in awake, unrestrained rats. Neuroscience 85, 1285–1309 (1998).
    Article CAS Google Scholar
  34. Bunney, B.S., Aghajanian, G.K. & Roth, R.H. Comparison of effects of L-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nat. New Biol. 245, 123–125 (1973).
    Article CAS Google Scholar
  35. Skirboll, L.R., Grace, A.A. & Bunney, B.S. Dopamine auto- and postsynaptic receptors: electrophysiological evidence for differential sensitivity to dopamine agonists. Science 206, 80–82 (1979).
    Article CAS Google Scholar
  36. Niv, Y., Daw, N.D. & Dayan, P. Choice values. Nat. Neurosci. 9, 987–988 (2006).
    Article CAS Google Scholar
  37. Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).
    Article CAS Google Scholar
  38. Joel, D. & Weiner, I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451–474 (2000).
    Article CAS Google Scholar
  39. Yin, H.H., Knowlton, B.J. & Balleine, B.W. Lesions of dorsolateral striatum preserve outcome expectancy, but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004).
    Article Google Scholar
  40. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).
    Article CAS Google Scholar
  41. Knowlton, B.J., Mangels, J.A. & Squire, L. A neostriatal habit learning system in humans. Science 273, 1399–1402 (1996).
    Article CAS Google Scholar
  42. Hatfield, T., Han, J.S., Conley, M., Gallagher, M. & Holland, P. Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256–5265 (1996).
    Article CAS Google Scholar
  43. Gallagher, M., McMahan, R.W. & Schoenbaum, G. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610–6614 (1999).
    Article CAS Google Scholar
  44. Baxter, M.G., Parker, A., Lindner, C.C.C., Izquierdo, A.D. & Murray, E.A. Control of response selection by reinforcer value requires interaction of amygdala and orbitofrontal cortex. J. Neurosci. 20, 4311–4319 (2000).
    Article CAS Google Scholar
  45. Gottfried, J.A., O'Doherty, J. & Dolan, R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104–1107 (2003).
    Article CAS Google Scholar
  46. Parkinson, J.A., Cardinal, R.N. & Everitt, B.J. Limbic cortical-ventral striatal systems underlying appetitive conditioning. Prog. Brain Res. 126, 263–285 (2000).
    Article CAS Google Scholar
  47. Lu, L. et al. Central amygdala ERK signaling pathway is critical to incubation of cocaine craving. Nat. Neurosci. 8, 212–219 (2005).
    Article CAS Google Scholar

Download references