The temporal precision of reward prediction in dopamine neurons (original) (raw)

References

1. Buhusi, C.V. & Meck, W.H. What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765 (2005).
   Article CAS PubMed Google Scholar
2. Mauk, M.D. & Buonomano, D.V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).
   Article CAS PubMed Google Scholar
3. Rao, S.M., Mayer, A.R. & Harrington, D.L. The evolution of brain activation during temporal processing. Nat. Neurosci. 4, 317–323 (2001).
   Article CAS PubMed Google Scholar
4. Gibbon, J., Malapani, C., Dale, C.L. & Gallistel, C.R. Toward a neurobiology of temporal cognition: advances and challenges. Curr. Opin. Neurobiol. 7, 170–184 (1997).
   Article CAS PubMed Google Scholar
5. Meck, W.H. Neuropharmacology of timing and time perception. Brain Res. Cogn. Brain Res. 3, 227–242 (1996).
   Article CAS PubMed Google Scholar
6. Matell, M.S. & Meck, W.H. Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes. Brain Res. Cogn. Brain Res. 21, 139–170 (2004).
   Article PubMed Google Scholar
7. Houk, J., Adams, J. & Barto, A. A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models of Information Processing in the Basal Ganglia (eds Houk, J., Davis, J. & Beiser, D.) 249–270 (MIT Press, Cambridge, Massachusetts, 1995).
   Google Scholar
8. Montague, P.R., Dayan, P. & Sejnowski, T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
   Article CAS PubMed Google Scholar
9. Schultz, W., Dayan, P. & Montague, R.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
   Article CAS PubMed Google Scholar
10. Berns, G.S. & Sejnowski, T.J. A computational model of how the basal ganglia produce sequences. J. Cogn. Neurosci. 10, 108–121 (1998).
    Article CAS PubMed Google Scholar
11. Brown, J., Bullock, D. & Grossberg, S. How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J. Neurosci. 19, 10502–10511 (1999).
    Article CAS PubMed Google Scholar
12. Contreras-Vidal, J.L. & Schultz, W. A predictive reinforcement model of dopamine neurons for learning approach behavior. J. Comput. Neurosci. 6, 191–214 (1999).
    Article CAS PubMed Google Scholar
13. Suri, R.E. & Schultz, W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871–890 (1999).
    Article CAS PubMed Google Scholar
14. Daw, N.D., Courville, A.C. & Touretzky, D.S. Representation and timing in theories of the dopamine system. Neural Comput. 18, 1637–1677 (2006).
    Article PubMed Google Scholar
15. Schultz, W., Apicella, P. & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900–913 (1993).
    Article CAS PubMed Google Scholar
16. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).
    Article CAS PubMed Google Scholar
17. Schultz, W. The predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).
    Article CAS PubMed Google Scholar
18. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).
    Article CAS PubMed Google Scholar
19. Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913–9923 (2003).
    Article CAS PubMed Google Scholar
20. Nakahara, H., Itho, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269–280 (2004).
    Article CAS PubMed Google Scholar
21. Morris, G., Arkadir, D., Nevet, A., Vaadia, E. & Bergman, H. Coincident, but distinct, messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133–143 (2004).
    Article CAS PubMed Google Scholar
22. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).
    Article CAS PubMed Google Scholar
23. Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).
    Article CAS PubMed Google Scholar
24. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).
    Article CAS PubMed PubMed Central Google Scholar
25. Rakitin, B.C. et al. Scalar expectancy theory and peak-interval timing in humans. J. Exp. Psychol. Anim. Behav. Process. 24, 15–33 (1998).
    Article CAS PubMed Google Scholar
26. Kacelnik, A. & Brito e Abreu, F. Risky choice and Weber's Law. J. Theor. Biol. 194, 289–298 (1998).
    Article CAS PubMed Google Scholar
27. Janssen, P. & Shadlen, M.N. A representation of the hazard rate of elapsed time in macaque area LIP. Nat. Neurosci. 8, 234–241 (2005).
    Article CAS PubMed Google Scholar
28. Ghose, G.M. & Maunsell, J.H.R. Attentional modulation in visual cortex depends on task-timing. Nature 419, 616–620 (2002).
    Article CAS PubMed Google Scholar
29. Bateson, M. & Kacelnik, A. Accuracy of memory for amount in the foraging starling, Sturnus vulgaris. Anim. Behav. 50, 431–443 (1995).
    Article Google Scholar
30. Komura, Y. et al. Retrospective and prospective coding for predicted reward in the sensory thalamus. Nature 412, 546–549 (2001).
    Article CAS PubMed Google Scholar
31. Brody, C.D., Hernandez, A., Zanos, A. & Romo, R. Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. Cereb. Cortex 13, 1196–1207 (2003).
    Article Google Scholar
32. Leon, M.I. & Shadlen, M.N. Representation of time by neurons in the posterior parietal cortex of the macaque. Neuron 38, 317–327 (2003).
    Article CAS PubMed Google Scholar
33. Renoult, L.Roux. S. & Riehle, A. Time is a rubberband: neuronal activity in monkey motor cortex in relation to time estimation. Eur. J. Neurosci. 23, 3098–3108 (2006).
    Article PubMed Google Scholar
34. O'Reilly, R.C., Frank, M.J., Hazy, T.E. & Watz, B. PVLV: the primary value and learned value Pavlovian learning algorithm. Behav. Neurosci. 121, 31–49 (2007).
    Article PubMed Google Scholar
35. Grossberg, S. & Schmajuk, N.A. Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Netw. 2, 79–102 (1989).
    Article Google Scholar
36. Medina, J.F., Nores, W.L. & Mauk, M.D. Inhibition of climbing fibers is a signal for the extinction of conditioned eyelid responses. Nature 416, 330–333 (2002).
    Article CAS PubMed Google Scholar
37. Schultz, W. & Romo, R. Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 57, 201–217 (1987).
    Article CAS PubMed Google Scholar
38. Nieder, A. & Miller, E.K. Coding of cognitive magnitude: compressed scaling of numerical information in the primate prefrontal cortex. Neuron 37, 149–157 (2003).
    Article CAS PubMed Google Scholar

Download references