Dopamine responses comply with basic assumptions of formal learning theory (original) (raw)
References
Thorndike, E. L. Animal Intelligence: Experimental Studies (MacMillan, New York, 1911). Book Google Scholar
Pavlov, I. P. Conditional Reflexes (Oxford Univ. Press, London, 1927). Google Scholar
Rescorla, R. A. & Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A. H. & Prokasy, W. F.) 64–99 (Appleton Century Crofts, New York, 1972). Google Scholar
Mackintosh, N. J. A theory of attention: Variations in the associability of stimulus with reinforcement. Psychol. Rev.82, 276–298 (1975). Article Google Scholar
Pearce, J. M. & Hall, G. A. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev.87, 532–552 (1980). ArticleCAS Google Scholar
Dickinson, A. Contemporary Animal Learning Theory (Cambridge Univ. Press, Cambridge, 1980). Google Scholar
Schultz, W. & Dickinson, A. Neuronal coding of prediction errors. Annu. Rev. Neurosci.23, 473–500 (2000). ArticleCAS Google Scholar
Widrow, G. & Hoff, M. E. Adaptive switching circuits. IRE Western Electron. Show Convention, Convention Record Part 4, 96–104 (1960).
Kalman, R. E. A new approach to linear filtering and prediction problems. J. Basic Eng. Trans. ASME82, 35–45 (1960). Article Google Scholar
Widrow, G. & Sterns, S. D. Adaptive Signal Processing (Prentice-Hall, Englewood Cliffs, 1985). Google Scholar
Thompson, R. F. & Gluck, M. A. in Perspectives on Cognitive Neuroscience (eds Lister, R. G. & Weingartner, H.) 25–45 (Oxford Univ. Press, New York, 1991). Google Scholar
Kawato, M. & Gomi, H. The cerebellum and VOR/OKR learning models. Trends Neurosci.15, 445–453 (1992). ArticleCAS Google Scholar
Kim, J. J., Krupa, D. J. & Thompson, R. F. Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science279, 570–573 (1998). ArticleADSCAS Google Scholar
Sutton, R. S. & Barto, A. G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev.88, 135–170 (1981). ArticleCAS Google Scholar
Sutton, R. S. & Barto, A. G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998). Google Scholar
Fibiger, H. C. & Phillips, A. G. in Handbook of Physiology—The Nervous System IV (ed. Bloom, F. E.) 647–675 (Williams and Wilkins, Baltimore, 1986). Google Scholar
Wise, R. A. & Hoffman, C. D. Localization of drug reward mechanisms by intracranial injections. Synapse10, 247–263 (1992). ArticleCAS Google Scholar
Robinson, T. E. & Berridge, K. C. The neural basis for drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev.18, 247–291 (1993). ArticleCAS Google Scholar
Robbins, T. W. & Everitt, B. J. Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol.6, 228–236 (1996). ArticleCAS Google Scholar
Romo, R. & Schultz, W. Dopamine neurons of the monkey midbrain: Contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol.63, 592–606 (1990). ArticleCAS Google Scholar
Schultz, W. & Romo, R. Dopamine neurons of the monkey midbrain: Contingencies of responses to stimuli eliciting immediate behavioural reactions. J. Neurophysiol.63, 607–624 (1990). ArticleCAS Google Scholar
Ljungberg, T., Apicella, P. & Schultz, W. Responses of monkey dopamine neurons during learning of behavioural reactions. J. Neurophysiol.67, 145–163 (1992). ArticleCAS Google Scholar
Schultz, W., Apicella, P. & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci.13, 900–913 (1993). ArticleCAS Google Scholar
Schultz, W., Dayan, P. & Montague, R. R. A neural substrate of prediction and reward. Science275, 1593–1599 (1997). ArticleCAS Google Scholar
Hollerman, J. R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neurosci.1, 304–309 (1998). ArticleCAS Google Scholar
Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol.80, 1–27 (1998). ArticleCAS Google Scholar
Salamone, J. D. The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav. Brain Res.61, 117–133 (1994). ArticleCAS Google Scholar
Horvitz, J. C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience96, 651–656 (2000). ArticleCAS Google Scholar
Sutton, R. S. & Barto, A. G. in Learning and Computational Neuroscience: Foundations of Adaptive Networks (eds Gabriel, M. & Moore, J.) 497–537 (MIT Press, Cambridge, Massachusetts, 1990). Google Scholar
Mackintosh, N. J. Conditioning and Associative Learning (Oxford Univ. Press, New York, 1983). Google Scholar
Friston, K. J., Tononi, G., Reeke, G. N. Jr, Sporns, O. & Edelman, G. M. Value-dependent selection in the brain: simulation in a synthetic neural model. Neuroscience59, 229–243 (1994). ArticleCAS Google Scholar
Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci.16, 1936–1947 (1996). ArticleCAS Google Scholar
Houk, J. C., Adams, J. L. & Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 249–270 (MIT Press, Cambridge, Massachusetts, 1995). Google Scholar
Suri, R. & Schultz, W. A neural network with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience91, 871–890 (1999). ArticleCAS Google Scholar
Kamin, L. J. in Fundamental Issues in Instrumental Learning (eds Mackintosh, N. J. & Honig, W. K.) 42–64 (Dalhousie Univ. Press, Dalhousie, 1969). Google Scholar
Martin, I. & Levey, A. B. Blocking observed in human eyelid conditioning. Q. J. Exp. Psychol. B43, 233–255 (1991). CASPubMed Google Scholar
Dickinson, A. Causal learning: An associative analysis. Q. J. Exp. Psychol. B54, 3–25 (2001). ArticleCAS Google Scholar
Holland, P. C. Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavioural theory. Anim. Learn. Behav.25, 373–399 (1997). Article Google Scholar
Calabresi, P., Maj, R., Pisani, A., Mercuri, N. B. & Bernardi, G. Long-term synaptic depression in the striatum: Physiological and pharmacological characterization. J. Neurosci.12, 4224–4233 (1992). ArticleCAS Google Scholar
Garcia-Munoz, M., Young, S. J. & Groves, P. Presynaptic long-term changes in excitability of the corticostriatal pathway. NeuroReport3, 357–360 (1992). ArticleCAS Google Scholar
Wickens, J. R., Begg, A. J. & Arbuthnott, G. W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience70, 1–5 (1996). ArticleCAS Google Scholar
Calabresi, P. et al. Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. J. Neurosci.17, 4536–4544 (1997). ArticleCAS Google Scholar
Otani, S., Blond, O., Desce, J. M. & Crepel, F. Dopamine facilitates long-term depression of glutamatergic transmission in rat prefrontal cortex. Neuroscience85, 669–676 (1998). ArticleCAS Google Scholar
Otani, S., Auclair, N., Desce, J., Roisin, M. P. & Crepel, F. J. Neurosci.19, 9788–9802 (1999). ArticleCAS Google Scholar
Centonze, D. et al. Unilateral dopamine denervation blocks corticostriatal LTP. J. Neurophysiol.82, 3575–3579 (1999). ArticleCAS Google Scholar
Minsky, M. L. Steps toward artificial intelligence. Proc. Inst. Radio Engineers49, 8–30 (1961). MathSciNet Google Scholar
Rauhut, A. S., McPhee, J. E. & Ayres, J. J. B. Blocked and overshadowed stimuli are weakened in their ability to serve as blockers and second-order reinforcers in Pavlovian fear conditioning. J. Exp. Psychol: Anim. Behav. Process25, 45–67 (1999). CAS Google Scholar
Schultz, W. & Romo, R. Responses of nigrostriatal dopamine neurons to high intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol.57, 201–217 (1987). ArticleCAS Google Scholar