Value, search, persistence and model updating in anterior cingulate cortex (original) (raw)

References

  1. Neubert, F.X., Mars, R.B., Sallet, J. & Rushworth, M.F. Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex. Proc. Natl. Acad. Sci. USA 112, E2695–E2704 (2015).
    Article CAS PubMed PubMed Central Google Scholar
  2. Behrens, T.E., Fox, P., Laird, A. & Smith, S.M. What is the most interesting part of the brain? Trends Cogn. Sci. 17, 2–4 (2013).
    Article PubMed Google Scholar
  3. Kolling, N., Behrens, T.E., Mars, R.B. & Rushworth, M.F. Neural mechanisms of foraging. Science 336, 95–98 (2012).
    Article CAS PubMed PubMed Central Google Scholar
  4. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).
    Article CAS PubMed PubMed Central Google Scholar
  5. Wittmann, M. et al. Predicting how your luck will change: decision making driven by multiple time-linked reward representations in anterior cingulate cortex. Nat. Commun. http://dx.doi.org/10.1038/ncomms12327 (2016).
  6. Bernacchia, A., Seo, H., Lee, D. & Wang, X.J. A reservoir of time constants for memory traces in cortical neurons. Nat. Neurosci. 14, 366–372 (2011).
    Article CAS PubMed PubMed Central Google Scholar
  7. Tervo, D.G. et al. Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21–32 (2014).
    Article CAS PubMed Google Scholar
  8. Karlsson, M.P., Tervo, D.G. & Karpova, A.Y. Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty. Science 338, 135–139 (2012).
    Article CAS PubMed Google Scholar
  9. O'Reilly, J.X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 110, E3660–E3669 (2013).
    CAS PubMed PubMed Central Google Scholar
  10. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).
    Article CAS PubMed Google Scholar
  11. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
    Article CAS PubMed Google Scholar
  12. Quilodran, R., Rothé, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325 (2008).
    Article CAS PubMed Google Scholar
  13. Narayanan, N.S., Cavanagh, J.F., Frank, M.J. & Laubach, M. Common medial frontal mechanisms of adaptive control in humans and rodents. Nat. Neurosci. 16, 1888–1895 (2013).
    Article CAS PubMed PubMed Central Google Scholar
  14. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).
    Article PubMed Google Scholar
  15. Warren, C.M., Hyman, J.M., Seamans, J.K. & Holroyd, C.B. Feedback-related negativity observed in rodent anterior cingulate cortex. J. Physiol. Paris 109, 87–94 (2015).
    Article PubMed Google Scholar
  16. Phillips, J.M. & Everling, S. Event-related potentials associated with performance monitoring in non-human primates. Neuroimage 97, 308–320 (2014).
    Article PubMed Google Scholar
  17. Scholl, J. et al. The good, the bad, and the irrelevant: neural mechanisms of learning real and hypothetical rewards and effort. J. Neurosci. 35, 11233–11251 (2015).
    Article CAS PubMed PubMed Central Google Scholar
  18. Shenhav, A., Botvinick, M.M. & Cohen, J.D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).
    Article CAS PubMed PubMed Central Google Scholar
  19. Shenhav, A., Straccia, M.A., Cohen, J.D. & Botvinick, M.M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254 (2014).
    Article CAS PubMed PubMed Central Google Scholar
  20. Cole, M.W., Yeung, N., Freiwald, W.A. & Botvinick, M. Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci. 32, 566–574 (2009).
    Article CAS PubMed PubMed Central Google Scholar
  21. Amiez, C. & Petrides, M. Neuroimaging evidence of the anatomo-functional organization of the human cingulate motor areas. Cereb. Cortex 24, 563–578 (2014).
    Article PubMed Google Scholar
  22. Procyk, E. et al. Midcingulate motor map and feedback detection: converging data from humans and monkeys. Cereb. Cortex 26, 467–476 (2016).
    PubMed Google Scholar
  23. O'Reilly, J.X. et al. Causal effect of disconnection lesions on interhemispheric functional connectivity in rhesus monkeys. Proc. Natl. Acad. Sci. USA 110, 13982–13987 (2013).
    Article CAS PubMed PubMed Central Google Scholar
  24. Kolling, N., Wittmann, M. & Rushworth, M.F. Multiple neural mechanisms of decision making and their competition under changing risk pressure. Neuron 81, 1190–1202 (2014).
    Article CAS PubMed PubMed Central Google Scholar
  25. Boorman, E.D., Rushworth, M.F. & Behrens, T.E. Ventromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multi-alternative choice. J. Neurosci. 33, 2242–2253 (2013).
    Article CAS PubMed PubMed Central Google Scholar
  26. Neubert, F.X., Mars, R.B., Thomas, A.G., Sallet, J. & Rushworth, M.F. Comparison of human ventral frontal cortex areas for cognitive control and language with areas in monkey frontal cortex. Neuron 81, 700–713 (2014).
    Article CAS PubMed Google Scholar
  27. Vogt, B.A. in Cingulate Neurobiology and Disease (ed. Vogt, B.A.) (Oxford Univ. Press, 2009).
  28. Passingham, R.E. & Wise, S.P. The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight (Oxford Univ. Press, 2012).
  29. Strick, P.L., Dum, R.P. & Picard, N. Motor areas on the medial wall of the hemisphere. Novartis Found. Symp. 218, 64–75, discussion 75–80, 104–108 (1998).
    CAS PubMed Google Scholar
  30. Van Hoesen, G.W., Morecraft, R.J. & Vogt, B.A. in Neurobiology of Cingulate Cortex and Limbic Thalamus (eds. Vogt, B.A. & Gabriel, M.) 249–284 (Birkhauser, 1993).
  31. Haber, S.N., Kim, K.S., Mailly, P. & Calzavara, R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J. Neurosci. 26, 8368–8376 (2006).
    Article CAS PubMed PubMed Central Google Scholar
  32. Porrino, L.J. & Goldman-Rakic, P.S. Brainstem innervation of prefrontal and anterior cingulate cortex in the rhesus monkey revealed by retrograde transport of HRP. J. Comp. Neurol. 205, 63–76 (1982).
    Article CAS PubMed Google Scholar
  33. Eblen, F. & Graybiel, A.M. Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J. Neurosci. 15, 5999–6013 (1995).
    Article CAS PubMed PubMed Central Google Scholar
  34. Friedman, A. et al. A corticostriatal path targeting striosomes controls decision-making under conflict. Cell 161, 1320–1333 (2015).
    Article CAS PubMed PubMed Central Google Scholar
  35. Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178 (2009).
    Article PubMed PubMed Central Google Scholar
  36. Kennerley, S.W., Behrens, T.E. & Wallis, J.D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581–1589 (2011).
    Article CAS PubMed PubMed Central Google Scholar
  37. Rudebeck, P.H., Mitz, A.R., Chacko, R.V. & Murray, E.A. Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex. Neuron 80, 1519–1531 (2013).
    Article CAS PubMed Google Scholar
  38. Hayden, B.Y., Pearson, J.M. & Platt, M.L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933–939 (2011).
    Article CAS PubMed PubMed Central Google Scholar
  39. Blanchard, T.C. & Hayden, B.Y. Neurons in dorsal anterior cingulate cortex signal postdecisional variables in a foraging task. J. Neurosci. 34, 646–655 (2014).
    Article CAS PubMed PubMed Central Google Scholar
  40. Cai, X. & Padoa-Schioppa, C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J. Neurosci. 32, 3791–3808 (2012).
    Article CAS PubMed PubMed Central Google Scholar
  41. Luk, C.H. & Wallis, J.D. Choice coding in frontal cortex during stimulus-guided or action-guided decision-making. J. Neurosci. 33, 1864–1871 (2013).
    Article CAS PubMed PubMed Central Google Scholar
  42. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).
    Article CAS PubMed Google Scholar
  43. Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).
    Article PubMed Google Scholar
  44. Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884–908 (2005).
    Article PubMed Google Scholar
  45. Ebitz, R.B. & Platt, M.L. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron 85, 628–640 (2015).
    Article CAS PubMed PubMed Central Google Scholar
  46. Sheth, S.A. et al. Human dorsal anterior cingulate cortex neurons mediate ongoing behavioural adaptation. Nature 488, 218–221 (2012).
    Article CAS PubMed PubMed Central Google Scholar
  47. McGuire, J.T. & Kable, J.W. Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence. Nat. Neurosci. 18, 760–766 (2015).
    Article CAS PubMed PubMed Central Google Scholar
  48. Cai, X. & Padoa-Schioppa, C. Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation. Neuron 81, 1140–1151 (2014).
    Article CAS PubMed PubMed Central Google Scholar
  49. Boorman, E.D., Behrens, T.E., Woolrich, M.W. & Rushworth, M.F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743 (2009).
    Article CAS PubMed Google Scholar
  50. Strait, C.E., Blanchard, T.C. & Hayden, B.Y. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82, 1357–1366 (2014).
    Article CAS PubMed PubMed Central Google Scholar
  51. Amemori, K. & Graybiel, A.M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785 (2012).
    Article CAS PubMed PubMed Central Google Scholar
  52. Wittmann, M.K. et al. Self-other mergence in frontal cortex during cooperation and competition. Neuron 91, 482–493 (2016).
    Article CAS PubMed PubMed Central Google Scholar
  53. De Martino, B., Fleming, S.M., Garrett, N. & Dolan, R.J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).
    Article CAS PubMed Google Scholar
  54. Lebreton, M., Abitbol, R., Daunizeau, J. & Pessiglione, M. Automatic integration of confidence in the brain valuation signal. Nat. Neurosci. 18, 1159–1167 (2015).
    Article CAS PubMed Google Scholar
  55. Mars, R.B. et al. Short-latency influence of medial frontal cortex on primary motor cortex during action selection under conflict. J. Neurosci. 29, 6926–6931 (2009).
    Article CAS PubMed PubMed Central Google Scholar
  56. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
    Article CAS PubMed PubMed Central Google Scholar
  57. Hare, T.A., Schultz, W., Camerer, C.F., O'Doherty, J.P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. USA 108, 18120–18125 (2011).
    Article CAS PubMed PubMed Central Google Scholar
  58. Wang, X.J. Decision making in recurrent neuronal circuits. Neuron 60, 215–234 (2008).
    Article CAS PubMed PubMed Central Google Scholar
  59. Hunt, L.T. et al. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 15, 470–476, S1–S3 (2012).
    Article CAS PubMed PubMed Central Google Scholar
  60. Freidin, E. & Kacelnik, A. Rational choice, context dependence, and the value of information in European starlings (Sturnus vulgaris). Science 334, 1000–1002 (2011).
    Article CAS PubMed Google Scholar
  61. Pearson, J.M., Watson, K.K. & Platt, M.L. Decision making: the neuroethological turn. Neuron 82, 950–965 (2014).
    Article CAS PubMed PubMed Central Google Scholar
  62. Logiaco, L., Quilodran, R., Procyk, E. & Arleo, A. Spatiotemporal spike coding of behavioral adaptation in the dorsal anterior cingulate cortex. PLoS Biol. 13, e1002222 (2015).
    Article PubMed PubMed Central CAS Google Scholar
  63. Voloh, B., Valiante, T.A., Everling, S. & Womelsdorf, T. Theta-gamma coordination between anterior cingulate and prefrontal cortex indexes correct attention shifts. Proc. Natl. Acad. Sci. USA 112, 8457–8462 (2015).
    Article CAS PubMed PubMed Central Google Scholar
  64. Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M. & Rushworth, M.F. Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161–1168 (2006).
    Article CAS PubMed Google Scholar
  65. Posner, M.I. & Petersen, S.E. The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42 (1990).
    Article CAS PubMed Google Scholar
  66. Fellows, L.K. & Farah, M.J. Is anterior cingulate cortex necessary for cognitive control? Brain 128, 788–796 (2005).
    Article PubMed Google Scholar
  67. Rushworth, M.F.S., Hadland, K.A., Gaffan, D. & Passingham, R.E. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 15, 338–353 (2003).
    Article CAS PubMed Google Scholar
  68. Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998).
    Article CAS PubMed Google Scholar
  69. Mansouri, F.A., Buckley, M.J. & Tanaka, K. Mnemonic function of the dorsolateral prefrontal cortex in conflict-induced behavioral adjustment. Science 318, 987–990 (2007).
    Article CAS PubMed Google Scholar
  70. Camille, N., Tsuchida, A. & Fellows, L.K. Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J. Neurosci. 31, 15048–15052 (2011).
    Article CAS PubMed PubMed Central Google Scholar
  71. Buckley, M.J. et al. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 325, 52–58 (2009).
    Article CAS PubMed Google Scholar
  72. Thaler, D., Chen, Y.C., Nixon, P.D., Stern, C.E. & Passingham, R.E. The functions of the medial premotor cortex. I. Simple learned movements. Exp. Brain Res. 102, 445–460 (1995).
    Article CAS PubMed Google Scholar
  73. Stoll, F.M., Fontanier, V. & Procyk, E. Specific frontal neural dynamics contribute to decisions to check. Nat. Commun. http://dx.doi.org/10.1038/ncomms11990 (2016).
  74. Meder, D. et al. Tuning the brake while raising the stake: network dynamics during sequential decision-making. J. Neurosci. 36, 5417–5426 (2016).
    Article CAS PubMed PubMed Central Google Scholar
  75. Klein-Flügge, M.C., Barron, H.C., Brodersen, K.H., Dolan, R.J. & Behrens, T.E. Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J. Neurosci. 33, 3202–3211 (2013).
    Article PubMed PubMed Central CAS Google Scholar

Download references