Modulators of decision making (original) (raw)

References

1. Doya, K., Ishii, S., Pouget, A. & Rao, R. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning (MIT Press, Cambridge, Massachusetts, USA, (2007).
   Google Scholar
2. Doya, K. Reinforcement learning: computational theory and biological mechanisms. HFSP J. 1, 30–40 (2007).
   Article PubMed PubMed Central Google Scholar
3. Sutton, R.S. & Barto, A.G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, USA, (1998).
   Google Scholar
4. Ho, M.Y., Mobini, S., Chiang, T.J., Bradshaw, C.M. & Szabadi, E. Theory and method in the quantitative analysis of “impulsive choice” behaviour: implications for psychopharmacology. Psychopharmacology (Berl.) 146, 362–372 (1999).
   Article CAS Google Scholar
5. Berns, G.S., Laibson, D. & Loewenstein, G. Intertemporal choice–toward an integrative framework. Trends Cogn. Sci. 11, 482–488 (2007).
   Article PubMed Google Scholar
6. Laibson, D.I. Golden eggs and hyperbolic discounting. Q. J. Econ. 62, 443–477 (1997).
   Article Google Scholar
7. von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton Univ. Press, Princeton, New Jersey, USA, (1944).
   Google Scholar
8. Kahneman, D. & Tversky, A. Prospect theory: an analysis of decision under risk. Econometrica 47, 263–291 (1979).
   Article Google Scholar
9. Ishii, S., Yoshida, W. & Yoshimoto, J. Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw. 15, 665–687 (2002).
   Article PubMed Google Scholar
10. Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004).
    Article CAS PubMed Google Scholar
11. Herrnstein, R.J. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272 (1961).
    Article CAS PubMed PubMed Central Google Scholar
12. Baum, W.M. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 36, 387–403 (1981).
    Article CAS PubMed PubMed Central Google Scholar
13. Puterman, M.L. Markov Decision Processes: Discrete Dynamic Stochastic Programming (Wiley, New York, (1994).
    Book Google Scholar
14. Balleine, B.W., Delgado, M.R. & Hikosaka, O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161–8165 (2007).
    Article CAS PubMed PubMed Central Google Scholar
15. Balleine, B.W. & Killcross, S. Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci. 29, 272–279 (2006).
    Article CAS PubMed Google Scholar
16. Kakade, S. & Dayan, P. Acquisition and extinction in autoshaping. Psychol. Rev. 109, 533–544 (2002).
    Article PubMed Google Scholar
17. Daw, N.D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).
    Article CAS PubMed Google Scholar
18. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507–520 (2007).
    Article CAS Google Scholar
19. Schweighofer, N. et al. Humans can adopt optimal discounting strategy under real-time constraints. PLOS Comput. Biol. 2, e152 (2006).
    Article Google Scholar
20. Doya, K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732–739 (2000).
    Article CAS PubMed Google Scholar
21. Matsumoto, K., Suzuki, W. & Tanaka, K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301, 229–232 (2003).
    Article CAS PubMed Google Scholar
22. Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).
    Article CAS PubMed Google Scholar
23. Schultz, W., Tremblay, L. & Hollerman, J.R. Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 10, 272–284 (2000).
    Article CAS PubMed Google Scholar
24. Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996).
    Article CAS PubMed Google Scholar
25. Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416 (1998).
    Article CAS PubMed Google Scholar
26. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).
    Article CAS PubMed Google Scholar
27. Pasquereau, B. et al. Shaping of motor responses by incentive values through the basal ganglia. J. Neurosci. 27, 1176–1183 (2007).
    Article CAS PubMed PubMed Central Google Scholar
28. Komura, Y., Tamura, R., Uwano, T., Nishijo, H. & Ono, T. Auditory thalamus integrates visual inputs into behavioral gains. Nat. Neurosci. 8, 1203–1209 (2005).
    Article CAS PubMed Google Scholar
29. Minamimoto, T., Hori, Y. & Kimura, M. Complementary process to response bias in the centromedian nucleus of the thalamus. Science 308, 1798–1801 (2005).
    Article CAS PubMed Google Scholar
30. Montague, P.R., Dayan, P. & Sejnowski, T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
    Article CAS PubMed PubMed Central Google Scholar
31. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
    Article CAS PubMed Google Scholar
32. McClure, S.M., Berns, G.S. & Montague, P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003).
33. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).
    Article CAS PubMed Google Scholar
34. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).
    Article CAS PubMed Google Scholar
35. Seymour, B. et al. Temporal difference models describe higher-order learning in humans. Nature 429, 664–667 (2004).
    Article CAS PubMed Google Scholar
36. Reynolds, J.N. & Wickens, J.R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15, 507–521 (2002).
    Article PubMed Google Scholar
37. Wickens, J.R., Begg, A.J. & Arbuthnott, G.W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 70, 1–5 (1996).
    Article CAS PubMed Google Scholar
38. Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495–506 (2002).
    Article PubMed Google Scholar
39. Yacubian, J. et al. Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J. Neurosci. 26, 9530–9537 (2006).
    Article CAS PubMed PubMed Central Google Scholar
40. Belova, M.A., Paton, J.J., Morrison, S.E. & Salzman, C.D. Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55, 970–984 (2007).
    Article CAS PubMed PubMed Central Google Scholar
41. Seymour, B., Daw, N., Dayan, P., Singer, T. & Dolan, R. Differential encoding of losses and gains in the human striatum. J. Neurosci. 27, 4826–4831 (2007).
    Article CAS PubMed PubMed Central Google Scholar
42. Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913–9923 (2003).
    Article CAS PubMed PubMed Central Google Scholar
43. Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).
    Article PubMed Google Scholar
44. Matsumoto, M. & Hikosaka, O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111–1115 (2007).
    Article CAS PubMed Google Scholar
45. Denk, F. et al. Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology (Berl.) 179, 587–596 (2005).
    Article CAS Google Scholar
46. Kuhnen, C.M. & Knutson, B. The neural basis of financial risk taking. Neuron 47, 763–770 (2005).
    Article CAS PubMed Google Scholar
47. Tobler, P.N., O'Doherty, J.P., Dolan, R.J. & Schultz, W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 97, 1621–1632 (2007).
    Article PubMed Google Scholar
48. Cardinal, R.N. Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw. 19, 1277–1301 (2006).
    Article PubMed Google Scholar
49. Tanaka, S.C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887–893 (2004).
    Article CAS PubMed Google Scholar
50. Tanaka, S.C. et al. Brain mechanism of reward prediction under predictable and unpredictable environmental dynamics. Neural Netw. 19, 1233–1241 (2006).
    Article PubMed Google Scholar
51. McClure, S.M., Laibson, D.I., Loewenstein, G. & Cohen, J.D. Separate neural systems value immediate and delayed monetary rewards. Science 306, 503–507 (2004).
    Article CAS PubMed Google Scholar
52. McClure, S.M., Ericson, K.M., Laibson, D.I., Loewenstein, G. & Cohen, J.D. Time discounting for primary rewards. J. Neurosci. 27, 5796–5804 (2007).
    Article CAS PubMed PubMed Central Google Scholar
53. Kable, J.W. & Glimcher, P.W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10, 1625–1633 (2007).
    Article CAS PubMed PubMed Central Google Scholar
54. Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M. & Rushworth, M.F. Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161–1168 (2006).
    Article CAS PubMed Google Scholar
55. Winstanley, C.A., Theobald, D.E., Dalley, J.W., Cardinal, R.N. & Robbins, T.W. Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice. Cereb. Cortex 16, 106–114 (2006).
    Article PubMed Google Scholar
56. Tanaka, S.C. et al. Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS ONE 2, e1333 (2007).
    Article PubMed PubMed Central CAS Google Scholar
57. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
    Article CAS PubMed Google Scholar
58. Bouret, S. & Sara, S.J. Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582 (2005).
    Article CAS PubMed Google Scholar
59. Dayan, P. & Yu, A.J. Phasic norepinephrine: A neural interrupt signal for unexpected events. Network 17, 335–350 (2006).
    Article PubMed Google Scholar
60. Aston-Jones, G. & Cohen, J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450 (2005).
    Article CAS PubMed Google Scholar
61. Clarke, H.F., Dalley, J.W., Crofts, H.S., Robbins, T.W. & Roberts, A.C. Cognitive inflexibility after prefrontal serotonin depletion. Science 304, 878–880 (2004).
    Article CAS PubMed Google Scholar
62. Floresco, S.B., Tse, M.T. & Ghods-Sharifi, S. Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology, published online 5 September 2007 (doi:10.1038/sj.npp.1301565).
    Article PubMed CAS Google Scholar
63. Walton, M.E., Bannerman, D.M. & Rushworth, M.F. The role of rat medial frontal cortex in effort-based decision making. J. Neurosci. 22, 10996–11003 (2002).
    Article CAS PubMed PubMed Central Google Scholar
64. Rogers, R.D., Lancaster, M., Wakeley, J. & Bhagwagar, Z. Effects of beta-adrenoceptor blockade on components of human decision-making. Psychopharmacology (Berl.) 172, 157–164 (2004).
    Article CAS Google Scholar
65. Cardinal, R.N. & Howes, N.J. Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 6, 37 (2005).
66. Winstanley, C.A., Dalley, J.W., Theobald, D.E. & Robbins, T.W. Fractionating impulsivity: contrasting effects of central 5-HT depletion on different measures of impulsive behavior. Neuropsychopharmacology 29, 1331–1343 (2004).
    Article CAS PubMed Google Scholar
67. van Gaalen, M.M., van Koten, R., Schoffelmeer, A.N. & Vanderschuren, L.J. Critical involvement of dopaminergic neurotransmission in impulsive decision making. Biol. Psychiatry 60, 66–73 (2006).
    Article CAS PubMed Google Scholar
68. Kheramin, S. et al. Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl.) 175, 206–214 (2004).
    Article CAS Google Scholar
69. Robinson, E.S. et al. Similar effects of the selective noradrenaline reuptake inhibitor atomoxetine on three distinct forms of impulsivity in the rat. Neuropsychopharmacology, published online 18 July 2007 (doi:10.1038/sj.npp.1301487).
    Article PubMed CAS Google Scholar
70. Rudebeck, P.H., Buckley, M.J., Walton, M.E. & Rushworth, M.F. A role for the macaque anterior cingulate gyrus in social valuation. Science 313, 1310–1312 (2006).
    Article CAS PubMed Google Scholar
71. Mobini, S. et al. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl.) 160, 290–298 (2002).
    Article CAS Google Scholar
72. Kheramin, S. et al. Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl.) 165, 9–17 (2002).
    Article CAS Google Scholar
73. Kheramin, S. et al. The effect of orbital prefrontal cortex lesions on performance on a progressive ratio schedule: implications for models of inter-temporal choice. Behav. Brain Res. 156, 145–152 (2005).
    Article CAS PubMed Google Scholar
74. Winstanley, C.A., Theobald, D.E., Cardinal, R.N. & Robbins, T.W. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24, 4718–4722 (2004).
    Article CAS PubMed PubMed Central Google Scholar
75. Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W. & Everitt, B.J. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292, 2499–2501 (2001).
    Article CAS PubMed Google Scholar
76. Pothuizen, H.H., Jongen-Relo, A.L., Feldon, J. & Yee, B.K. Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats. Eur. J. Neurosci. 22, 2605–2616 (2005).
    Article PubMed Google Scholar
77. Hariri, A.R. et al. Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. J. Neurosci. 26, 13213–13217 (2006).
    Article CAS PubMed PubMed Central Google Scholar
78. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).
    Article CAS PubMed Google Scholar
79. Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).
    Article CAS PubMed Google Scholar
80. Rogers, R.D. et al. Tryptophan depletion impairs stimulus-reward learning while methylphenidate disrupts attentional control in healthy young adults: implications for the monoaminergic basis of impulsive behaviour. Psychopharmacology (Berl.) 146, 482–491 (1999).
    Article CAS Google Scholar
81. Rogers, R.D. et al. Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: evidence for monoaminergic mechanisms. Neuropsychopharmacology 20, 322–339 (1999).
    Article CAS PubMed Google Scholar
82. Chamberlain, S.R. et al. Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311, 861–863 (2006).
    Article CAS PubMed PubMed Central Google Scholar
83. Clarke, H.F., Walker, S.C., Dalley, J.W., Robbins, T.W. & Roberts, A.C. Cognitive inflexibility after prefrontal serotonin depletion is behaviorally and neurochemically specific. Cereb. Cortex 17, 18–27 (2007).
    Article CAS Google Scholar
84. van der Plasse, G. et al. Medial prefrontal serotonin in the rat is involved in goal-directed behaviour when affect guides decision making. Psychopharmacology (Berl.) 195, 435–449 (2007).
    Article CAS Google Scholar
85. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
    Article CAS PubMed PubMed Central Google Scholar
86. Corbit, L.H. & Balleine, B.W. The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res. 146, 145–157 (2003).
    Article PubMed Google Scholar
87. Balleine, B.W. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol. Behav. 86, 717–730 (2005).
    Article CAS PubMed Google Scholar
88. Rachlin, H., Raineri, A. & Cross, D. Subjective probability and delay. J. Exp. Anal. Behav. 55, 233–244 (1991).
    Article CAS PubMed PubMed Central Google Scholar

Download references