Striatal prediction error modulates cortical coupling (original) (raw)
Related papers
Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning
2017
Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning (RL) or by using explicit reasoning. We tested if striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a surprise signal derived from a Bayesian rule-learning model. We also find that striatal feedback responses are inconsistent with RL prediction error and demonstrate that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules, rather than representing reward prediction errors.
Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning
Human Brain Mapping, 2006
A prominent theory in neuroscience suggests reward learning is driven by the discrepancy between a subject's expectation of an outcome and the actual outcome itself. Furthermore, it is postulated that midbrain dopamine neurons relay this mismatch to target regions including the ventral striatum. Using functional MRI (fMRI), we tested striatal responses to prediction errors for probabilistic classification learning with purely cognitive feedback. We used a version of the Rescorla-Wagner model to generate prediction errors for each subject and then entered these in a parametric analysis of fMRI activity. Activation in ventral striatum/nucleus-accumbens (Nacc) increased parametrically with prediction error for negative feedback. This result extends recent neuroimaging findings in reward learning by showing that learning with cognitive feedback also depends on the same circuitry and dopaminergic signaling mechanisms.
Ventral striatum encodes past and predicted value independent of motor contingencies
The Journal of neuroscience : the official journal of the Society for Neuroscience, 2012
The ventral striatum (VS) is thought to signal the predicted value of expected outcomes. However, it is still unclear whether VS can encode value independently from variables often yoked to value such as response direction and latency. Expectations of high value reward are often associated with a particular action and faster latencies. To address this issue we trained rats to perform a task in which the size of the predicted reward was signaled before the instrumental response was instructed. Instrumental directional cues were presented briefly at a variable onset to reduce accuracy and increase reaction time. Rats were more accurate and slower when a large versus small reward was at stake. We found that activity in VS was high during odors that predicted large reward even though reaction times were slower under these conditions. In addition to these effects, we found that activity before the reward predicting cue reflected past and predicted reward. These results demonstrate that V...
Cerebral Cortex, 2007
Associative theory postulates that learning the consequences of our actions in a given context is represented in the brain as stimulus-response--outcome associations that evolve according to predictionerror signals (the discrepancy between the observed and predicted outcome). We tested the theory on brain functional magnetic resonance imaging data acquired from human participants learning arbitrary visuomotor associations. We developed a novel task that systematically manipulated learning and induced highly reproducible performances. This granted the validation of the model-based results and an in-depth analysis of the brain signals in representative single trials. Consistent with the Rescorla--Wagner model, prediction-error signals are computed in the human brain and selectively engage the ventral striatum. In addition, we found evidence of computations not formally predicted by the Rescorla--Wagner model. The dorsal fronto-parietal network, the dorsal striatum, and the ventrolateral prefrontal cortex are activated both on the incorrect and first correct trials and may reflect the processing of relevant visuomotor mappings during the early phases of learning. The left dorsolateral prefrontal cortex is selectively activated on the first correct outcome. The results provide quantitative evidence of the neural computations mediating arbitrary visuomotor learning and suggest new directions for future computational models.
2021
According to the predictive coding (PC) theory, the brain is constantly engaged in predicting its upcoming states and refining these predictions through error signals. Despite extensive research investigating the neural bases of this theory, to date no previous study has systematically attempted to define the neural mechanisms of predictive coding across studies and sensory channels, focussing on functional connectivity. In this study, we employ a coordinate-based meta-analytical approach to address this issue. We first use the Activation Likelihood Estimation (ALE) algorithm to detect spatial convergence across studies, related to prediction error and encoding. Overall, our ALE results suggest the ultimate role of the left inferior frontal gyrus and left insula in both processes. Moreover, we employ a meta-analytic connectivity method (Seed-Voxel Correlations Consensus). This technique reveals a large, bilateral predictive network, which resembles large-scale networks involved in taskdriven attention and execution. In sum, we find that: (i) predictive processing seems to occur more in certain brain regions than others, when considering different sensory modalities at a time; (ii) there is no evidence, at the network level, for a distinction between error and prediction processing. According to the theory of predictive coding (PC) 1-5 , our brain constantly attempts to model the probability of its own future states, with the goal of minimizing uncertainty 4. More specifically, the brain is considered a hierarchically organized system where, at each level of processing, higher layers try to predict the latent causes of the sensory input coming from lower layers 6,7. Thus, neurons at higher levels encode predictions about the upcoming signal, which is continuously compared with the effective signal received from lower levels. Through this comparison, the brain either reinforces existing predictions or it updates them, if these do not match the incoming signal 8. When predictions are violated, a prediction error signal 5,9,10 is fed back to the neurons encoding predictions. These recursive loops of predictions and error signals ultimately allow the individual to maintain up-to-date representations about its own internal states 11 and the surrounding external stimuli. Over the past two decades, PC theory has received extensive support from a vast range of theoretical and experimental studies, both in relation to primary sensory processes 5,12-14 and higher level cognitive processes 15,16 , such as decision making and naturalistic speech comprehension 14,17,18. Moreover, evidence has been obtained with a variety of methods, mostly with functional magnetic resonance imaging (fMRI), but also electroencephalography 19-21 , computational simulations 22 , transcranial magnetic stimulation 23 , and physiological recordings of single neurons (for a review, see 24). Since 1999, when Rao and Ballard published their seminal simulation work on predictive coding in the visual cortex 5 , there has been a proliferation of attempts to implement PC in the human brain. Initially, it was argued that predictive processing occurs at the cellular level 25 , where the activity of neural populations is modulated by higher-order predictions and units signalling precision of those predictions. According to Bastos and colleagues 26 , PC is a typical property of the human cerebral neocortex because its structure suits a hierarchical signal exchange between cortical layers. In particular, error signals seem to be computed in the granular layers (especially layer IV), while predictions would be encoded in layers II and III 26. These mechanisms have been identified in a large set of brain areas, including the primary sensory and motor cortices, motor association cortices, dorsal and ventral prefrontal cortices, parietal cortex, anterior cingulate cortex, insula, hippocampus, amygdala, basal ganglia, thalamus, hypothalamus, cerebellum and the superior colliculus 27,28. However, in all these regions, neuronal
Learning predictive statistics: strategies and brain mechanisms
The Journal of neuroscience : the official journal of the Society for Neuroscience, 2017
When immersed in a new environment we are challenged to decipher initially incomprehensible streams of sensory information. Yet, quite rapidly, the brain finds structure and meaning in these incoming signals, helping us to predict and prepare ourselves for future actions. This skill relies on extracting the statistics of event streams in the environment that contain regularities of variable complexity: from simple repetitive patterns to complex probabilistic combinations. Here, we test the brain mechanisms that mediate our ability to adapt to the environment's statistics and predict upcoming events. By combining behavioral training and multi-session fMRI in human participants (male and female), we track the cortico-striatal mechanisms that mediate learning of temporal sequences as they change in structure complexity. We show that learning of predictive structures relates to individual decision strategy; that is, selecting the most probable outcome in a given context (maximizing)...
A dual role for prediction error in associative learning
Cerebral …, 2009
Confronted with a rich sensory environment, the brain must learn statistical regularities across sensory domains to construct causal models of the world. Here, we used functional magnetic resonance imaging and dynamic causal modeling (DCM) to furnish neurophysiological evidence that statistical associations are learnt, even when task-irrelevant. Subjects performed an audio-visual target-detection task while being exposed to distractor stimuli. Unknown to them, auditory distractors predicted the presence or absence of subsequent visual distractors. We modeled incidental learning of these associations using a Rescorla-Wagner (RW) model. Activity in primary visual cortex and putamen reflected learning-dependent surprise: these areas responded progressively more to unpredicted, and progressively less to predicted visual stimuli. Critically, this prediction-error response was observed even when the absence of a visual stimulus was surprising. We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error. Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity. These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity.
Response selection versus feedback analysis in conditional visuo-motor learning
NeuroImage, 2012
Conditional associative sensori-motor learning (i.e. the acquisition of specific arbitrary sensori-motor mappings) involves several processes that depend upon the integrity of the fronto-striatal system. The specific role of the different components of the fronto-striatal system in this type of learning is still unclear and was examined in the present functional Magnetic Resonance Imaging (fMRI) study in humans. The subjects had to learn by trial and error arbitrary associations between visual stimuli and motor responses in an experimental paradigm designed to dissociate between the neuronal substrates specifically involved in the selection of the appropriate response and in the analysis of the feedback obtained during the learning and post-learning periods. First, the results demonstrate that the dorsal premotor (PMd) cortex is the critical structure for the acquisition and execution of arbitrary mappings of visual stimuli to motor responses. Second, they reveal an important shift in activation from the cognitive fronto-striatal network (involving the caudate nucleus, the dorsolateral prefrontal cortex, and the PMd) to the motor fronto-striatal network (involving the putamen and the PMd) as we move from initial learning of sensori-motor relations to the post-learning selection of the responses. Finally, they show that feedback processing, but not response selection, increased activity in the anterior cingulate and orbitofrontal cortical regions, demonstrating the selective involvement of these limbic frontal regions in the processing of the consequences of a given action. Altogether our data suggest that, in conditional visuo-motor learning, the associations are critically regulated by the dorsal premotor cortex and the striatum, with additional brain areas contributing to specific aspects of the learning and performance of such associations.
Choice modulates the neural dynamics of prediction error processing during rewarded learning
Neuroimage, 2011
Our ability to selectively engage with our environment enables us to guide our learning and to take advantage of its benefits. When facing multiple possible actions, our choices are a critical aspect of learning. In the case of learning from rewarding feedback, there has been substantial theoretical and empirical progress in elucidating the associated behavioral and neural processes, predominantly in terms of a reward prediction error, a measure of the discrepancy between actual versus expected reward. Nevertheless, the distinct influence of choice on prediction error processing and its neural dynamics remains relatively unexplored. In this study we used a novel paradigm to determine how choice influences prediction error processing and to examine whether there are correspondingly distinct neural dynamics. We recorded scalp electroencephalogram while healthy adults were administered a rewarded learning task in which choice trials were intermingled with control trials involving the same stimuli, motor responses, and probabilistic rewards. We used a temporal difference learning model of subjects' trial-by-trial choices to infer subjects' image valuations and corresponding prediction errors. As expected, choices were associated with lower overall prediction error magnitudes, most notably over the course of learning the stimulus-reward contingencies. Choices also induced a higher-amplitude relative positivity in the frontocentral event-related potential about 200 ms after reward signal onset that was negatively correlated with the differential effect of choice on the prediction error. Thus choice influences the neural dynamics associated with how reward signals are processed during learning. Behavioral, computational, and neurobiological models of rewarded learning should therefore accommodate a distinct influence for choice during rewarded learning.
Learning & Memory, 2008
This fMRI study investigated the neural correlates of reward-related trial-and-error learning in association with changing degrees of stimulus-outcome predictabilities. We found that decreasing predictability was associated with increasing activation in a frontoparietal network. Only maximum predictability was associated with signal decreases across the learning process. The receipt of monetary reward revealed activation in the striatum and associated frontoparietal regions. Present data indicate that during reward-related learning, high uncertainty forces areas relevant for cognitive control to remain activated. In contrast, learning on the basis of predictable stimulus-outcome associations enables the brain to reduce resources in association with the processes of prediction.