Dopamine responses comply with basic assumptions of formal learning theory (original) (raw)

References

  1. Thorndike, E. L. Animal Intelligence: Experimental Studies (MacMillan, New York, 1911).
    Book Google Scholar
  2. Pavlov, I. P. Conditional Reflexes (Oxford Univ. Press, London, 1927).
    Google Scholar
  3. Rescorla, R. A. & Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A. H. & Prokasy, W. F.) 64–99 (Appleton Century Crofts, New York, 1972).
    Google Scholar
  4. Mackintosh, N. J. A theory of attention: Variations in the associability of stimulus with reinforcement. Psychol. Rev. 82, 276–298 (1975).
    Article Google Scholar
  5. Pearce, J. M. & Hall, G. A. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532–552 (1980).
    Article CAS Google Scholar
  6. Dickinson, A. Contemporary Animal Learning Theory (Cambridge Univ. Press, Cambridge, 1980).
    Google Scholar
  7. Schultz, W. & Dickinson, A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473–500 (2000).
    Article CAS Google Scholar
  8. Widrow, G. & Hoff, M. E. Adaptive switching circuits. IRE Western Electron. Show Convention, Convention Record Part 4, 96–104 (1960).
  9. Kalman, R. E. A new approach to linear filtering and prediction problems. J. Basic Eng. Trans. ASME 82, 35–45 (1960).
    Article Google Scholar
  10. Widrow, G. & Sterns, S. D. Adaptive Signal Processing (Prentice-Hall, Englewood Cliffs, 1985).
    Google Scholar
  11. Marr, D. A theory of cerebellar cortex. J. Physiol. 202, 437–470 (1969).
    Article CAS Google Scholar
  12. Ito, M. Long-term depression. Ann. Rev. Neurosci. 12, 85–102 (1989).
    Article CAS Google Scholar
  13. Thompson, R. F. & Gluck, M. A. in Perspectives on Cognitive Neuroscience (eds Lister, R. G. & Weingartner, H.) 25–45 (Oxford Univ. Press, New York, 1991).
    Google Scholar
  14. Kawato, M. & Gomi, H. The cerebellum and VOR/OKR learning models. Trends Neurosci. 15, 445–453 (1992).
    Article CAS Google Scholar
  15. Kim, J. J., Krupa, D. J. & Thompson, R. F. Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science 279, 570–573 (1998).
    Article ADS CAS Google Scholar
  16. Sutton, R. S. & Barto, A. G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88, 135–170 (1981).
    Article CAS Google Scholar
  17. Sutton, R. S. & Barto, A. G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).
    Google Scholar
  18. Fibiger, H. C. & Phillips, A. G. in Handbook of Physiology—The Nervous System IV (ed. Bloom, F. E.) 647–675 (Williams and Wilkins, Baltimore, 1986).
    Google Scholar
  19. Wise, R. A. & Hoffman, C. D. Localization of drug reward mechanisms by intracranial injections. Synapse 10, 247–263 (1992).
    Article CAS Google Scholar
  20. Robinson, T. E. & Berridge, K. C. The neural basis for drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev. 18, 247–291 (1993).
    Article CAS Google Scholar
  21. Robbins, T. W. & Everitt, B. J. Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 6, 228–236 (1996).
    Article CAS Google Scholar
  22. Romo, R. & Schultz, W. Dopamine neurons of the monkey midbrain: Contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol. 63, 592–606 (1990).
    Article CAS Google Scholar
  23. Schultz, W. & Romo, R. Dopamine neurons of the monkey midbrain: Contingencies of responses to stimuli eliciting immediate behavioural reactions. J. Neurophysiol. 63, 607–624 (1990).
    Article CAS Google Scholar
  24. Ljungberg, T., Apicella, P. & Schultz, W. Responses of monkey dopamine neurons during learning of behavioural reactions. J. Neurophysiol. 67, 145–163 (1992).
    Article CAS Google Scholar
  25. Schultz, W., Apicella, P. & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900–913 (1993).
    Article CAS Google Scholar
  26. Schultz, W., Dayan, P. & Montague, R. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
    Article CAS Google Scholar
  27. Hollerman, J. R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neurosci. 1, 304–309 (1998).
    Article CAS Google Scholar
  28. Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).
    Article CAS Google Scholar
  29. Salamone, J. D. The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav. Brain Res. 61, 117–133 (1994).
    Article CAS Google Scholar
  30. Horvitz, J. C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651–656 (2000).
    Article CAS Google Scholar
  31. Sutton, R. S. & Barto, A. G. in Learning and Computational Neuroscience: Foundations of Adaptive Networks (eds Gabriel, M. & Moore, J.) 497–537 (MIT Press, Cambridge, Massachusetts, 1990).
    Google Scholar
  32. Mackintosh, N. J. Conditioning and Associative Learning (Oxford Univ. Press, New York, 1983).
    Google Scholar
  33. Friston, K. J., Tononi, G., Reeke, G. N. Jr, Sporns, O. & Edelman, G. M. Value-dependent selection in the brain: simulation in a synthetic neural model. Neuroscience 59, 229–243 (1994).
    Article CAS Google Scholar
  34. Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
    Article CAS Google Scholar
  35. Houk, J. C., Adams, J. L. & Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 249–270 (MIT Press, Cambridge, Massachusetts, 1995).
    Google Scholar
  36. Suri, R. & Schultz, W. A neural network with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871–890 (1999).
    Article CAS Google Scholar
  37. Kamin, L. J. in Fundamental Issues in Instrumental Learning (eds Mackintosh, N. J. & Honig, W. K.) 42–64 (Dalhousie Univ. Press, Dalhousie, 1969).
    Google Scholar
  38. Martin, I. & Levey, A. B. Blocking observed in human eyelid conditioning. Q. J. Exp. Psychol. B 43, 233–255 (1991).
    CAS PubMed Google Scholar
  39. Dickinson, A. Causal learning: An associative analysis. Q. J. Exp. Psychol. B 54, 3–25 (2001).
    Article CAS Google Scholar
  40. Holland, P. C. Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavioural theory. Anim. Learn. Behav. 25, 373–399 (1997).
    Article Google Scholar
  41. Calabresi, P., Maj, R., Pisani, A., Mercuri, N. B. & Bernardi, G. Long-term synaptic depression in the striatum: Physiological and pharmacological characterization. J. Neurosci. 12, 4224–4233 (1992).
    Article CAS Google Scholar
  42. Garcia-Munoz, M., Young, S. J. & Groves, P. Presynaptic long-term changes in excitability of the corticostriatal pathway. NeuroReport 3, 357–360 (1992).
    Article CAS Google Scholar
  43. Wickens, J. R., Begg, A. J. & Arbuthnott, G. W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 70, 1–5 (1996).
    Article CAS Google Scholar
  44. Calabresi, P. et al. Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. J. Neurosci. 17, 4536–4544 (1997).
    Article CAS Google Scholar
  45. Otani, S., Blond, O., Desce, J. M. & Crepel, F. Dopamine facilitates long-term depression of glutamatergic transmission in rat prefrontal cortex. Neuroscience 85, 669–676 (1998).
    Article CAS Google Scholar
  46. Otani, S., Auclair, N., Desce, J., Roisin, M. P. & Crepel, F. J. Neurosci. 19, 9788–9802 (1999).
    Article CAS Google Scholar
  47. Centonze, D. et al. Unilateral dopamine denervation blocks corticostriatal LTP. J. Neurophysiol. 82, 3575–3579 (1999).
    Article CAS Google Scholar
  48. Minsky, M. L. Steps toward artificial intelligence. Proc. Inst. Radio Engineers 49, 8–30 (1961).
    MathSciNet Google Scholar
  49. Rauhut, A. S., McPhee, J. E. & Ayres, J. J. B. Blocked and overshadowed stimuli are weakened in their ability to serve as blockers and second-order reinforcers in Pavlovian fear conditioning. J. Exp. Psychol: Anim. Behav. Process 25, 45–67 (1999).
    CAS Google Scholar
  50. Schultz, W. & Romo, R. Responses of nigrostriatal dopamine neurons to high intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 57, 201–217 (1987).
    Article CAS Google Scholar

Download references