Inference based decisions in a hidden state foraging task: differential contributions of prefrontal cortical areas (original) (raw)
Related papers
Brain-wide representations of prior information in mouse decision-making
bioRxiv (Cold Spring Harbor Laboratory), 2023
To address this problem, we analyzed brain-wide data from the International Brain Laboratory, which provides electrophysiological recordings from 159 brain regions, defined by the Allen Common Coordinate Framework (Wang et al., 2020), as well as from widefield imaging (WFI) data from layers 2/3 of cortex of mice performing the same decision-making task (The International Brain Laboratory et al., 2021)(cite BWM). Our results suggest that the prior is encoded cortically and subcortically, across all levels of the brain, including early sensory regions. Mice use the prior to optimize their performance Mice were trained to discriminate whether a visual stimulus, of varying contrast, appeared in the right or left visual field (Fig. 1a). Importantly, the prior probability that the stimulus appeared on the right side switched in a random and uncued manner between 0.2 and 0.8 in blocks of 20-100 trials (Fig. 1b). Knowledge of the current prior would help the mice perform well; in particular, on zero contrast trials, the prior is the only source of information, as the probability of reward on these trials is determined by the block probability. We refer to the experimentally-determined prior as the 'true block prior'. Since the presence of the blocks was not explicitly cued, mice could only form a subjective estimate of the true block prior from the trial history. At best, they can compute the estimate of the true block prior given full knowledge of the task structure and the sequence of previous stimulus sides since the start of a session, which we refer thereafter as the Bayes-optimal prior (see Methods, Fig. 2a). Analyzing choice behavior revealed that mice leverage the block structure to improve their performance. Psychometric curves conditioned on right and left blocks, averaged across all animals and all sessions, were displaced relative to each other, in a direction consistent with the true block prior (2-tailed signed-rank Wilcoxon paired test between proportion of right choices on zero contrast trials: t=3, p=1.4E-20, N=115 mice; Fig. 1c). As a result, animals performed at 59.0% ± 0.4% (mean ± sem) correct for zero contrast trials, statistically significantly better than chance (2-tailed signed-rank Wilcoxon t=12, p=2.6E-20, N=115 mice) but significantly worse than an observer that generates actions by sampling from the Bayes-optimal prior, which performs at 61.3% ± 1.8% (mean ± std; 2-tailed signed-rank Wilcoxon paired test t=1547, p=6.0e-7, N=115 mice). Tracking performance around block switches provided further evidence that the animals estimated and used the prior. Indeed, around block switches performance dropped, presumably because of the mismatch between the subjective and true block prior. Thus, performance on zero contrast trials recovered with a decay constant of 5.16 trials (jackknife median, see Methods). This is slower than the previously introduced observer that generates actions by sampling the Bayes-optimal prior (jackknife median: 2.46 , 2-tailed paired t-test t 114 =2.94, p=0.004, N=115 jackknife replicates; Fig. S1).
Revisiting foraging approaches in neuroscience
Many complex real-world decisions, such as deciding which house to buy or whether to switch jobs, involve trying to maximise reward across a sequence of choices. Optimal Foraging Theory is well suited to study these kinds of choices because it provides formal models for reward-maximisation in sequential situations. In this article, we review recent insights from foraging neuroscience, behavioural ecology and computational modelling. We find that a commonly used approach in foraging neuroscience, in which choice items are encountered at random, does not reflect the way animals direct their foraging efforts in real-world settings, nor does it reflect efficient reward-maximising behaviour. Based on this, we propose that task designs allowing subjects to encounter choice items strategically will further improve the ecological validity of foraging approaches used in neuroscience, as well as give rise to new behavioural and neural predictions that deepen our understanding of sequential, v...
Engaging and Exploring: Cortical Circuits for Adaptive Foraging Decisions
Impulsivity is a profound source of poor decision making, often bringing suffering to both person and polity. Although impulsivity attends psychiatric disorders such as addiction, pathological gambling, attention deficit hyperactivity disorder, and obsessive-compulsive disorder, almost everyone makes impulsive decisions that disregard the long-term consequences of our actions in favor of the near-term allure of immediate temptations. Deliberating between long-term benefits and short-term rewards is also a hallmark of foraging decisions, probably the most fundamental of all challenges confronted by mobile organisms. Behavioral studies confirm theoretical predictions that foragers compute the value of current offers, track background reward rates over different temporal and spatial scales, and update strategies in response to changes in the environment. These observations suggest that the execution of foraging computations is fundamental for understanding the organization of the nervous system. Here we describe a process model for making foraging choices that integrates the value of short-term options and compares that value to a decision threshold determined by long-term reward rates. In addition, the role of interrupts and optimization routines are here incorporated for the first time into a foraging framework, by adapting decision thresholds to changes in the environment. A core network of brain areas, including the ventromedial prefrontal cortex, the anterior cingulate cortex, and the posterior cingulate cortex, under the modulatory influence of dopamine and norepinephrine, executes these computations and implements these processes. Our model provocatively implies that maladaptive impulsive choices can result from dysregulated foraging neurocircuitry.
The neural basis of spontaneous perceptual selection
2003
What do we do with the large proportion of our waking lives when we are not concerned about satisfying the standard needs for survival and reproductive success (e.g., satisfying hunger, avoidance of harm, etc.)? Possible Answer We attempt to satisfy another drive, one that maximizes the rate at which we acquire new but interpretable information. That is, we are infovores.
Competitive Frontoparietal Interactions Mediate Implicit Inferences
2019
Frequent experience with regularities in our environment allows us to use predictive information to guide our decision process. However, contingencies in our environment are not always explicitly present and sometimes need to be inferred. Heretofore, it remained unknown how predictive information guides decision-making when explicit knowledge is absent and how the brain shapes such implicit inferences. In the present experiment, 17 human participants (9 females) performed a discrimination task in which a target stimulus was preceded by a predictive cue. Critically, participants had no explicit knowledge that some of the cues signaled an upcoming target, allowing us to investigate how implicit inferences emerge and guide decision-making. Despite unawareness of the cue-target contingencies, participants were able to use implicit information to improve performance. Concurrent EEG recordings demonstrate that implicit inferences rely upon interactions between internally and externally oriented networks, whereby prefrontal regions inhibit parietal cortex under internal implicit control.
Neural mechanisms of observational learning
Proceedings of the National Academy of Sciences of the United States of America, 2010
Individuals can learn by interacting with the environment and experiencing a difference between predicted and obtained outcomes (prediction error). However, many species also learn by observing the actions and outcomes of others. In contrast to individual learning, observational learning cannot be based on directly experienced outcome prediction errors. Accordingly, the behavioral and neural mechanisms of learning through observation remain elusive. Here we propose that human observational learning can be explained by two previously uncharacterized forms of prediction error, observational action prediction errors (the actual minus the predicted choice of others) and observational outcome prediction errors (the actual minus predicted outcome received by others). In a functional MRI experiment, we found that brain activity in the dorsolateral prefrontal cortex and the ventromedial prefrontal cortex respectively corresponded to these two distinct observational learning signals.
2013
Selective attention in real-world environments enables subjects to direct information gathering systems towards relevant stimuli in the presence of distractors. In a dynamic visual environment that lacks cues identifying the relevance of stimuli for reaching goals, attention requires internal mechanisms to track valuable targets. Reinforcement learning (RL) provides a theoretical framework that is typically applied to describe the mechanisms underlying optimal action control (Lee et al., 2012; Schultz et al., 1997; Sutton and Barto, 1998). Here we propose a RL approach to resolve the credit assignment problem for the deployment of covert attentional selection (Balcarras et al., 2013; Kaping et al., 2011). We compared two types of RL models to explain behavior of two macaques performing a selective attention task during foraging in the space of stimulus features: we show that despite being extensively reinforced on the relevance of stimulus color for reward, monkey behavior does not ...
Neurons in Dorsal Anterior Cingulate Cortex Signal Postdecisional Variables in a Foraging Task
Journal of Neuroscience, 2014
The dorsal anterior cingulate cortex (dACC) is a key hub of the brain's executive control system. Although a great deal is known about its role in outcome monitoring and behavioral adjustment, whether and how it contributes to the decision process remain unclear. Some theories suggest that dACC neurons track decision variables (e.g., option values) that feed into choice processes and is thus "predecisional." Other theories suggest that dACC activity patterns differ qualitatively depending on the choice that is made and is thus "postdecisional." To compare these hypotheses, we examined responses of 124 dACC neurons in a simple foraging task in which monkeys accepted or rejected offers of delayed rewards. In this task, options that vary in benefit (reward size) and cost (delay) appear for 1 s; accepting the option provides the cued reward after the cued delay. To get at dACC neurons' contributions to decisions, we focused on responses around the time of choice, several seconds before the reward and the end of the trial. We found that dACC neurons signal the foregone value of the rejected option, a postdecisional variable. Neurons also signal the profitability (that is, the relative value) of the offer, but even these signals are qualitatively different on accept and reject decisions, meaning that they are also postdecisional. These results suggest that dACC can be placed late in the decision process and also support models that give it a regulatory role in decision, rather than serving as a site of comparison.
Neural circuitry of judgment and decision mechanisms
Brain Research Reviews, 2005
Tracing the neural circuitry of decision formation is a critical step in the understanding of higher cognitive function. To make a decision, the primate brain coordinates dynamic interactions between several cortical and subcortical areas that process sensory, cognitive, and reward information. In selecting the optimal behavioral response, decision mechanisms integrate the accumulating evidence with reward expectation and knowledge from prior experience, and deliberate about the choice that matches the expected outcome. Linkages between sensory input and behavioral output responsible for response selection are shown in the neural activity of structures from the prefrontal-basal ganglia-thalamo-cortical loop. The deliberation process can be best described in terms of sensitivity, selection bias, and activation threshold. Here, we show a systems neuroscience approach of the visual saccade decision circuit and the interaction between its components during decision formation.