Choosing the greater of two goods: neural currencies for valuation and decision making (original) (raw)

• Bauby, J. D. The Diving Bell and The Butterfly: A Memoir of Life in Death (Alfred A. Knopf Inc., New York, 1997).
  Google Scholar
• Glimcher, P. W. & Rustichini, A. R. Neuroeconomics: the consilience of brain and decision. Science 306, 447–452 (2004).
  CAS PubMed Google Scholar
• Parker, A. J. & Newsome, W. T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998). An exploration of the link between neural activity and sensory perception, summarizing work that comprises the background to recent neurophysiological studies of perceptual decision making.
  CAS PubMed Google Scholar
• Rizzolatti, G. & Luppino, G. The cortical motor system. Neuron 31, 889–901 (2001).
  CAS PubMed Google Scholar
• Graziano, M. S., Taylor, C. S., Moore, T. & Cooke, D. F. The cortical control of movement revisited. Neuron 36, 349–362 (2002).
  CAS PubMed Google Scholar
• Andersen, R. A. & Bruneo, C. A. Intentional maps in posterior parietal cortex. Annu. Rev. Neurosci. 25, 189–220 (2002).
  CAS PubMed Google Scholar
• Leon, M. I. & Shadlen, M. N. Exploring the neurophysiology of decisions. Neuron 21, 669–672 (1998).
  CAS PubMed Google Scholar
• Georgopoulos, A. P. Neural aspects of cognitive motor control. Curr. Opin. Neurobiol. 10, 238 (2000).
  CAS PubMed Google Scholar
• Schall, J. D. Neural basis of deciding, choosing and acting. Nature Rev. Neurosci. 2, 33–42 (2001).
  CAS Google Scholar
• Gold, J. I. & Shadlen, M. N. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5, 10–16 (2001). Proposes that neural systems implement categorical decisions about perceptual stimuli by computing a decision variable related to the logarithm of the likelihood ratio in favour of one or another alternative. Discusses how this framework can be used to interpret neural signals recorded from eye movement planning centres during perceptual discrimination tasks.
  PubMed Google Scholar
• Glimcher, P. W. Making choices: the neurophysiology of visual-saccadic decision making. Trends Neurosci. 24, 654–659 (2001).
  CAS PubMed Google Scholar
• Romo, R. & Salinas, E. Touch and go: decision making mechanisms in somatosensation. Annu. Rev. Neurosci. 24, 107–137 (2001).
  CAS PubMed Google Scholar
• Romo, R., Hernandez, A., Zainos, A., Lemus, L. & Brody, C. Neural correlates of decision-making in secondary somatosensory cortex. Nature Neurosci. 5, 1217–1225 (2002).
  CAS PubMed Google Scholar
• Romo, R. & Salinas, E. Flutter discrimination: neural codes, perception, memory and decision making. Nature Rev. Neurosci. 4, 203–218 (2003).
  CAS Google Scholar
• Graham, N. V. S. Visual Pattern Analysers (Oxford Univ. Press, Oxford, 1989).
  Google Scholar
• Shadlen, M. N. & Newsome, W. T. Motion perception: seeing and deciding. Proc. Natl Acad. Sci. USA 93, 628–633 (1996). One of the first demonstrations of a correlation between the activity of single cortical neurons and an evolving perceptual decision.
  CAS PubMed PubMed Central Google Scholar
• Shadlen, M. N. & Newsome, W. T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916–1936 (2001).
  CAS PubMed Google Scholar
• Kim, J. N. & Shadlen, M. N. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nature Neurosci. 2, 176–185 (1999).
  PubMed Google Scholar
• Horwitz, G. D. & Newsome, W. T. Target selection for saccadic eye movements: prelude activity in the superior colliculus during a direction-discrimination task. J. Neurophysiol. 86, 2543–2558 (2001).
  CAS PubMed Google Scholar
• Roitman, J. D. & Shadlen, M. N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).
  CAS PubMed PubMed Central Google Scholar
• Hanks, T. D. & Shadlen, M. N. Microstimulation of macaque area LIP affects decision making in a motion discrimination task. Soc. Neurosci. Abstr. 20.9 (2004).
• Bisley, J. W. & Goldberg, M. E. Neuronal activity in the lateral intraparietal area and spatial attention. Science 299, 81–86 (2003).
  CAS PubMed Google Scholar
• Platt, M. L. & Glimcher, P. W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1998).
  Google Scholar
• Kahneman, D. & Tversky, A. (eds) Choices, Values, and Frames (Cambridge Univ. Press, Cambridge, 2000).
  Google Scholar
• Berridge, K. C. & Robinson, T. E. Parsing reward. Trends Neurosci. 26, 507–513 (2003). Discusses the distinct psychological components of reward and their underlying neural substrates. Emphasizes the distinction between systems devoted to the affective ('liking') and motivational ('wanting') aspects of reward, and between motivational processes based on simple Pavlovian associations and those involving more cognitive representations of value.
  CAS PubMed Google Scholar
• Olds, J. & Milner, P. M. Positive reinforcement produced by electrical stimulation of septal area and other regions of the rat brain. J. Comp. Physiol. Psychol. 47, 419–427 (1954).
  CAS PubMed Google Scholar
• Shizgal, P. Neural basis of utility estimation. Curr. Opin. Neurobiol. 7, 198–208 (1997). Summarizes findings from a series of elegant experiments involving BSR in the rat that indicate the existence of a final common neural representation for reward information.
  CAS PubMed Google Scholar
• Wise, R. A. & Bozarth, M. A. A psychomotor stimulant theory of addiction. Psychol. Rev. 94, 469–492 (1987).
  CAS PubMed Google Scholar
• Hyman, S. E. & Malenka, R. C. Addiction and the brain: the neurobiology of compulsion and its persistence. Nature Rev. Neurosci. 2, 695–703 (2001).
  CAS Google Scholar
• Kelley, A. E. Memory and addiction. Shared neural circuitry and molecular mechanisms. Neuron 44, 161–167 (2004).
  CAS PubMed Google Scholar
• Berridge, K. C. & Robinson, T. E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Rev. 28, 309–369 (1998).
  CAS PubMed Google Scholar
• Garris, P. A. et al. Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature 398, 67–69 (1999).
  CAS PubMed Google Scholar
• Pecina, S., Cagniard, B., Berridge, K. C., Aldridge, J. W. & Zhuang, X. Hyperdopaminergic mutant mice have higher 'wanting' but not 'liking' for sweet rewards. J. Neurosci. 23, 9395–9402 (2003).
  CAS PubMed PubMed Central Google Scholar
• Liu, Z. et al. DNA targeting of rhinal cortex D2 receptor protein reversibly blocks learning of cues that predict reward. Proc. Natl Acad. Sci. USA 101, 12336–12341 (2004).
  CAS PubMed PubMed Central Google Scholar
• Schultz, W., Apicella, P. & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900–913 (1993).
  CAS PubMed PubMed Central Google Scholar
• Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).
  CAS PubMed Google Scholar
• Hollerman, J. R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neurosci. 1, 304–309 (1998).
  CAS PubMed Google Scholar
• Sutton, R. S. & Barto, A. G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).
  Google Scholar
• Dayan, P. & Abbott, L. F. Theoretical Neuroscience Ch. 9 (MIT Press, Cambridge, Massachusetts, 2001).
  Google Scholar
• Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997). Reviews evidence indicating that midbrain dopamine neurons signal an error in the prediction of future reward (see references 35–37). Proposes that the function of these neurons is particularly well described by a specific class of reinforcement learning algorithms, and shows how a model that uses a dopamine-like signal to implement such an algorithm can learn to predict future rewards and guide action selection.
  CAS PubMed Google Scholar
• Schultz, W. Getting formal with dopamine and reward. Neuron 36, 241–263 (2002).
  CAS PubMed Google Scholar
• Montague, P. R., Dayan, P., Person, C. & Sejnowski, T. Bee foraging in uncertain environments using predictive Hebbian learning. Nature 377, 725–728 (1995).
  CAS PubMed Google Scholar
• Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995).
  Google Scholar
• Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neurosci. 1, 411–416 (1998).
  CAS PubMed Google Scholar
• Lauwereyns, J., Watanabe, K., Coe, B. & Hikosaka, O. A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413–417 (2002).
  CAS PubMed Google Scholar
• Gold, J. I. Linking reward expectation to behavior in the basal ganglia. Trends Neurosci. 26, 12–14 (2003).
  CAS PubMed Google Scholar
• Montague, P. R. & Berns, G. S. Neural economics and the biological substrates of valuation. Neuron 36, 265–284 (2002). Among the first of a series of recent attempts (see also references 41,48,49) to integrate our emerging understanding of the role of dopamine with other reward-related neural signals for the purpose of developing a coherent framework for understanding the neural basis of reward and valuation.
  CAS PubMed Google Scholar
• McClure, S. M., Daw, N. D. & Montague, P. R. A computational substrate for incentive salience. Trends Neurosci. 26, 423–428 (2003).
  CAS PubMed Google Scholar
• Montague, P. R., Hyman, S. E. & Cohen, J. D. Computational roles for dopamine in behavioral control. Nature 431, 760–767 (2004).
  CAS PubMed Google Scholar
• Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269–280 (2004).
  CAS PubMed Google Scholar
• Rolls, E. T., Critchley, H., Mason, R. & Wakeman, E. A. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J. Neurophysiol. 75, 1970–1978 (1996).
  CAS PubMed Google Scholar
• Tremblay, L. & Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704–708 (1999).
  CAS PubMed Google Scholar
• Tremblay, L. & Schultz, W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 83, 1877–1885 (2000).
  CAS PubMed Google Scholar
• Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996).
  CAS PubMed Google Scholar
• Leon, M. I. & Shadlen, M. N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).
  CAS PubMed Google Scholar
• Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M. & Hikosaka, O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).
  PubMed Google Scholar
• Wallis, J. D. & Miller, E. K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18, 2069–2081 (2003).
  PubMed Google Scholar
• Roesch, M. R. & Olson, C. R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).
  PubMed Google Scholar
• Roesch, M. R. & Olson, C. R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004).
  CAS PubMed Google Scholar
• Shidara, M. & Richmond, B. J. Anterior cingulate: single neuron signals related to the degree of reward expectancy. Science 296, 1709–1711 (2002).
  PubMed Google Scholar
• McCoy, A. N., Crowley, J. C., Haghighian, G., Dean, H. L. & Platt, M. L. Saccade reward signals in posterior cingulate cortex. Neuron 40, 1031–1040 (2003).
  CAS PubMed Google Scholar
• Ito, S., Stuphorn, V., Brown, J. W. & Schall, J. D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).
  CAS PubMed Google Scholar
• Fuster, J. in Cerebral Cortex (eds Peter, J. & Jones, E.) 151–177 (Plenum, New York, 1985).
  Google Scholar
• Segraves, M. A. & Goldberg, M. E. Functional properties of corticotectal neurons in the monkey's frontal eye field. J. Neurophysiol. 58, 1387–1419 (1987).
  CAS PubMed Google Scholar
• Funahashi, S., Bruce, C. J. & Goldman-Rakic, P. S. Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J. Neurophysiol. 61, 331–349 (1989).
  CAS PubMed Google Scholar
• Romo, R., Brody, C. D., Hernandez, A. & Lemus, L. Neuronal correlates of parametric working memory in the prefrontal cortex. Nature 339, 470–473 (1999).
  Google Scholar
• Gnadt, J. W. & Andersen, R. A. Memory related motor planning activity in posterior parietal cortex of macaque. Exp. Brain Res. 70, 216–220 (1988).
  CAS PubMed Google Scholar
• Barash, S., Bracewell, R. M., Fogassi, L., Gnadt, J. W. & Andersen, R. A. Saccade-related activity in the lateral intraparietal area. I. Temporal properties; comparison with area 7a. J. Neurophysiol. 66, 1095–1108 (1991).
  CAS PubMed Google Scholar
• Colby, C. L., Duhamel, J. R. & Goldberg, M. E. Visual, presaccadic, and cognitive activation of single neurons in monkey intraparietal area. J. Neurophysiol. 76, 2841–2852 (1996).
  CAS PubMed Google Scholar
• Bechara, A., Tranel, D., Damasio, H. & Damasio, A. R. Deciding advantageously before knowing the advantageous strategy. Science 275, 1293–1295 (1997).
  CAS PubMed Google Scholar
• Iversen, S. D. & Mishkin, M. Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity. Exp. Brain Res. 11, 376–386 (1970).
  CAS PubMed Google Scholar
• Rolls, E. T. The orbitofrontal cortex and reward. Cereb. Cortex 10, 284–294 (2000).
  CAS PubMed Google Scholar
• Damasio, A. R. Descartes' Error: Emotion, Reason, and the Human Brain (Grosset/Putnam, New York, 1994).
  Google Scholar
• Musallam, S., Corneil, B. D., Greger, B., Scherberger, H. & Andersen, R. A. Cognitive control signals for neural prosthetics. Science 305, 258–262 (2004).
  CAS PubMed Google Scholar
• Schultz, W. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioral ecology. Curr. Opin. Neurobiol. 14, 139–147 (2004). An up-to-date survey of the types and locations of reward-related signals that have been documented using neurophysiological techniques.
  CAS PubMed Google Scholar
• Barraclough, D. J., Conroy, M. L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nature Neurosci. 7, 404–410 (2004).
  CAS PubMed Google Scholar
• Sugrue, L. P., Corrado, G. S. & Newsome, W. T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004).
  CAS PubMed Google Scholar
• Dorris, M. C. & Glimcher, P. W. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44, 365–378 (2004). References 76–78 describe studies that all used a novel free-choice approach to investigate the behavioural and neural basis of value-based decision making in awake monkeys. These papers are discussed at length in the main text of this review.
  CAS PubMed Google Scholar
• Erev, I. & Roth, A. E. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am. Econ. Rev. 88, 848–881 (1998).
  Google Scholar
• Platt, M. L. & Glimcher, P. W. Response fields of intraparietal neurons quantified with multiple saccadic targets. Exp. Brain Res. 121, 65–75 (1998).
  CAS PubMed Google Scholar
• Gottlieb, J. P., Kusunoki, M. & Goldberg, M. E. The representation of visual salience in monkey parietal cortex. Nature 391, 481–484 (1998).
  CAS PubMed Google Scholar
• Colby, C. L. & Goldberg, M. E. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319–349 (1999).
  CAS PubMed Google Scholar
• Glimcher, P. W. The neurobiology of visual-saccadic decision making. Annu. Rev. Neurosci. 26, 133–179 (2003).
  CAS PubMed Google Scholar
• Glimcher, P. W., Dorris, M. C. & Bayer, H. M. Physiological utility theory and the neuroeconomics of choice. Games Econ. Behav. (in the press).
• Gold, J. I. & Shadlen, M. N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36, 299–308 (2002).
  CAS PubMed Google Scholar
• Gold, J. I. & Shadlen, M. N. Representation of a perceptual decision in developing oculomotor commands. Nature 404, 390–394 (2000).
  CAS PubMed Google Scholar
• Shadlen, M. N. Pursuing commitments. Nature Neurosci. 5, 819–821 (2002).
  CAS PubMed Google Scholar
• Horwitz, G. D., Batista, A. P. & Newsome, W. T. Representation of an abstract perceptual decision in macaque superior colliculus. J. Neurophysiol. 91, 2281–2296 (2004).
  PubMed Google Scholar
• Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192 (2004).
  CAS PubMed Google Scholar
• Maunsell, J. H. Neuronal representations of cognitive state: reward of attention? Trends Cogn. Sci. 8, 261–265 (2004).
  PubMed Google Scholar
• Koch, C. & Ullman, S. Shifts in selective visual attention: toward the underlying neural circuitry. Hum. Neurobiol. 4, 219–227 (1985).
  CAS PubMed Google Scholar
• Itti, L. & Koch, C. Computational modeling of visual attention. Nature Rev. Neurosci. 2, 194–203 (2001).
  CAS Google Scholar
• Kahneman, D. in Les Prix Nobel 2002 (ed. Frangsmyr, T.) 416–499 (2002). A clear summary of the pioneering work of Kahneman and Tversky on the psychology of valuation and choice, which emphasizes the importance of intuitive over deliberative decision making and challenges basic economic assumptions of rationality.
  Google Scholar
• Dayan, P. & Balleine, B. W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).
  CAS PubMed Google Scholar
• Dickinson, A. in Animal Learning and Cognition (ed. Mackintosh, N. J.) 45–79 (Academic, Orlando, 1994).
  Google Scholar
• Nevin, J. A. Analyzing Thorndike's law of effect: the question of stimulus-response bonds. J. Exp. Anal. Behav. 72, 447–450 (1999).
  CAS PubMed PubMed Central Google Scholar
• Colwill, R. M. & Rescorla, R. A. Instrumental responding remains sensitive to reinforcer devaluation after extensive training. J. Exp. Psychology Anim. Behav. Process. 11, 520–536 (1985).
  Google Scholar
• Nash, J. F. Equilibrium points in n-person games. Proc. Natl Acad. Sci. USA 36, 48–49 (1950).
  CAS PubMed PubMed Central Google Scholar
• Holt, C. A. & Roth, A. E. The Nash equilibrium: a perspective. Proc. Natl Acad. Sci. USA 101, 3999–4002 (2004).
  CAS PubMed PubMed Central Google Scholar
• Herrnstein, R. J. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272 (1961).
  CAS PubMed PubMed Central Google Scholar
• Stephens, D. W. & Krebs, J. R. Foraging Theory (Princeton Univ. Press, Princeton, New Jersey, 1986).
  Google Scholar
• Baum, W. M. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 36, 387–401 (1981).
  CAS PubMed PubMed Central Google Scholar
• Gallistel, C. R. & Gibbon, J. Time, rate and conditioning. Psychol. Rev. 107, 289–344 (2000). A refreshing, if controversial, look at the subject of operant choice. Argues for distinct decision mechanisms in choice situations that involve opting for the single best alternative, compared with those settings in which an animal must decide in what proportion to allocate its time or responses between alternatives.
  CAS PubMed Google Scholar
• Heyman, G. M. A Markov model description of changeover probabilities on concurrent variable-interval schedules. J. Exp. Anal. Behav. 31, 41–51 (1979).
  CAS PubMed PubMed Central Google Scholar
• Green, D. M. & Swets, J. A. Signal Detection Theory and Psychophysics (Wiley, New York, 1966). A seminal work that contributed the key theoretical foundation to research on perceptual decision making.
  Google Scholar
• Van Essen, D. C. in The Visual Neurosciences (eds Chalupa, L. & Werner, J. S.) 507–521 (MIT Press, Cambridge, Massachusetts, 2004).
  Google Scholar
• Van Essen, D. C. et al. An integrated software suite for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 8, 443–459 (2001).
  CAS PubMed PubMed Central Google Scholar