Reinforcement learning and dopamine in schizophrenia: dimensions of symptoms or specific features of a disease group? (original) (raw)
Related papers
2006
How Do Computational Models of the Role of Dopamine as a Reward Prediction Error Map on to Current Dopamine Theories of Schizophrenia? Angela J. Thurnham* (a.j.thurnham@herts.ac.uk), D. John Done** (d.j.done@herts.ac.uk), Neil Davey* (n.davey@herts.ac.uk), Ray J. Frank* (r.j.frank@herts.ac.uk) School of Computer Science,* School of Psychology, **University of Hertfordshire, College Lane, Hatfield, Hertfordshire. AL10 9AB United Kingdom reward prediction, or TD error, including evidence that the RPE model of dopamine activity applies to humans as well as primates. The biological plausibility of existing neural network models by Cohen & Servan-Schreiber, (1992); Braver Barch & Cohen, (1999); Suri & Schultz, (1999); Rougier, Noelle, Braver, Cohen & O’Reilly, (2005) and O’Reilly & Frank, (2006) are then discussed in the light of the afore-mentioned dopamine theories of schizophrenia. Finally, we conclude with four major questions arising from recent dopamine theories of schizophrenia th...
Brain Sciences
Schizophrenia spectrum disorders (SZ) are characterized by impairments in probabilistic reinforcement learning (RL), which is associated with dopaminergic circuitry encompassing the prefrontal cortex and basal ganglia. However, there are no studies examining dopaminergic genes with respect to probabilistic RL in SZ. Thus, the aim of our study was to examine the impact of dopaminergic genes on performance assessed by the Probabilistic Selection Task (PST) in patients with SZ in comparison to healthy control (HC) subjects. In our study, we included 138 SZ patients and 188 HC participants. Genetic analysis was performed with respect to the following genetic polymorphisms: rs4680 in COMT, rs907094 in DARP-32, rs2734839, rs936461, rs1800497, and rs6277 in DRD2, rs747302 and rs1800955 in DRD4 and rs28363170 and rs2975226 in DAT1 genes. The probabilistic RL task was completed by 59 SZ patients and 95 HC subjects. SZ patients performed significantly worse in acquiring reinforcement continge...
Biological psychiatry, 2007
Background:Rewards and punishments may make distinct contributions to learning via separate striato-cortical pathways. We investigated whether fronto-striatal dysfunction in schizophrenia (SZ) is characterized by selective impairment in either reward- (Go) or punishment-driven (NoGo) learning.Methods:We administered two versions of a Probabilistic Selection task (Frank et al., 2004) to 40 SZs and 31 controls, using difficult-to-verbalize stimuli (Exp 1) and nameable objects (Exp 2). In an acquisition phase, participants learned to choose between three different stimulus pairs (AB, CD, EF) presented in random order, based on probabilistic feedback (80%, 70%, 60%). We used ANOVAs to assess the effects of group and reinforcement probability on two measures of contingency learning. To characterize the preference of subjects for choosing the most rewarded stimulus and avoiding the most punished stimulus, we subsequently tested participants with novel pairs of stimuli involving either A or B, providing no feedback.Results:Controls demonstrated superior performance during the first 40 acquisition trials in each of the 80% and 70% conditions versus the 60% condition; patients showed similarly impaired (<60%) performance in all three conditions. In novel test pairs, patients showed decreased preference for the most rewarded stimulus (A; t=2.674; p=0.01). Patients were unimpaired at avoiding the most negative stimulus (B; t=0.737).Conclusions:The results of these experiments provide additional evidence for the presence of deficits in reinforcement learning in SZ, suggesting that reward-driven (Go) learning may be more profoundly impaired than punishment-driven (NoGo) learning.
How Do Computational Models of the Role of Dopamine as a Reward Prediction
2008
A review of the current dopamine theories of schizophrenia reveals a likely imbalance between cortical and subcortical microcircuits due to an insufficient inhibitory brake, leading to a disruption of the dopamine system and the classic positive psychotic symptoms, negative symptoms and cognitive deficits associated with the disorder. Recent computational models have modelled the role of dopamine as a reward prediction error, using Temporal Difference and have successfully shown how these symptoms could arise from a disturbance to the dopamine system. We review these models in the light of dopamine theories of schizophrenia and highlight some of the major points that should be addressed by future computational models.
Neuropsychology, 2011
Objective: Patients with schizophrenia (SZ) show reinforcement learning impairments related to both the gradual/procedural acquisition of reward contingencies, and the ability to use trial-to-trial feedback to make rapid behavioral adjustments. Method: We used neurocomputational modeling to develop plausible mechanistic hypotheses explaining reinforcement learning impairments in individuals with SZ. We tested the model with a novel Go/NoGo learning task in which subjects had to learn to respond or withhold responses when presented with different stimuli associated with different probabilities of gains or losses in points. We analyzed data from 34 patients and 23 matched controls, characterizing positive-and negative-feedback-driven learning in both a training phase and a test phase. Results: Consistent with simulations from a computational model of aberrant dopamine input to the basal ganglia patients, patients with SZ showed an overall increased rate of responding in the training phase, together with reduced response-time acceleration to frequently rewarded stimuli across training blocks, and a reduced relative preference for frequently rewarded training stimuli in the test phase. Patients did not differ from controls on measures of procedural negative-feedback-driven learning, although patients with SZ exhibited deficits in trial-to-trial adjustments to negative feedback, with these measures correlating with negative symptom severity. Conclusions: These findings support the hypothesis that patients with SZ have a deficit in procedural "Go" learning, linked to abnormalities in DA transmission at D1-type receptors, despite a "Go bias" (increased response rate), potentially related to excessive tonic dopamine. Deficits in trial-to-trial reinforcement learning were limited to a subset of patients with SZ with severe negative symptoms, putatively stemming from prefrontal cortical dysfunction.
Probabilistic reinforcement learning in schizophrenia: Relationships to anhedonia and avolition
Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 2016
BACKGROUND: Anhedonia (a reduced experience of pleasure) and avolition (a reduction in goal-directed activity) are common features of patients with schizophrenia that have substantial effects on functional outcome but are poorly understood and treated. We examined whether alterations in reinforcement learning may contribute to these symptoms in patients with schizophrenia by impairing the translation of reward information into goal-directed action. METHODS: Thirty-eight stable outpatients with schizophrenia or schizoaffective disorder and 37 healthy control subjects underwent functional magnetic resonance imaging scans during a probabilistic stimulus selection reinforcement learning task with dissociated choice-and feedback-related activation, followed by a behavioral transfer task allowing separate assessment of learning from positive versus negative outcomes. A Q-learning algorithm was used to examine functional activation relating to prediction error at the time of feedback and to expected value at the time of choice. RESULTS: Behavioral results suggested a reduction in learning from positive feedback in patients; however, this reduction was unrelated to anhedonia/avolition severity. On analysis of the functional magnetic resonance imaging scans, prediction error-related activation at the time of feedback was highly similar between patients and control subjects. During early learning, patients activated regions in the cognitive control network to a lesser extent than control subjects. Correlation analyses revealed reduced responses to positive feedback in dorsolateral prefrontal cortex and caudate among those patients higher in anhedonia/avolition. CONCLUSIONS: These results suggest that anhedonia/avolition are as strongly related to cortical learning or higherlevel processes involved in goal-directed behavior, such as effort computation and planning, as to striatally mediated learning mechanisms.
Psychological Medicine, 2020
Background:Schizophrenia is a disorder characterized by pervasive deficits in cognitive functioning. However, few well-powered studies have examined the degree to which cognitive performance is impaired even among individuals with schizophrenia not currently on antipsychotic medications using a wide-range of cognitive and reinforcement learning measures derived from cognitive neuroscience. Such research is particularly needed in the domain of reinforcement learning, given the central role of dopamine in reinforcement learning, and the potential impact of antipsychotic medications on dopamine function.Methods:The present study sought to fill this gap by examining healthy controls (N=75), unmedicated (N=48) and medicated (N=148) individuals with schizophrenia. Participants were recruited across 5 sites as part of the CNTRaCS Consortium to complete tasks assessing processing speed, cognitive control, working memory, verbal learning, relational encoding and retrieval, visual integration, and reinforcement learning.Results:Individuals with schizophrenia who were not taking antipsychotic medications, as well as those taking antipsychotic medications, showed pervasive deficits across cognitive domains including reinforcement learning, processing speed, cognitive control, working memory, verbal learning, and relational encoding and retrieval. Further, we found that chlorpromazine equivalency rates were significantly related to processing speed and working memory, while there were no significant relationships between anticholinergic load and performance on other tasks.Conclusions:These findings add to a body of literature suggesting that cognitive deficits are an enduring aspect of schizophrenia, present in those off antipsychotic medications as well as those taking antipsychotic medications.
NeuroImage, 2010
In patients with schizophrenia, the ability to learn from reinforcement is known to be impaired. The present fMRI study aimed at investigating the neural correlates of reinforcement-related trial-and-error learning in 19 schizophrenia patients and 20 healthy volunteers. A modified gambling paradigm was applied where each cue indicated a subsequent number which had to be guessed. In order to vary predictability, the cuenumber associations were based on different probabilities (50%, 81%, 100%) which the participants were not informed about. Patients' ability to learn contingencies on the basis of feedback and reward was significantly impaired. While in healthy volunteers increasing predictability was associated with decreasing activation in a fronto-parietal network, this decrease was not detectable in patients. Analysis of expectancy-related reinforcement processing yielded a hypoactivation in putamen, dorsal cingulate and superior frontal cortex in patients relative to controls. Present results indicate that both reinforcement-associated processing and reinforcement learning might be impaired in the context of the disorder. They moreover suggest that the activation deficits which patients exhibit in association with the processing of reinforcement might constitute the basis for the learning deficits and their accompanying activation alterations.
Motivational Context Modulates Prediction Error Response in Schizophrenia
Schizophrenia Bulletin, 2016
Background: Recent findings demonstrate that patients with schizophrenia are worse at learning to predict rewards than losses, suggesting that motivational context modulates learning in this disease. However, these findings derive from studies in patients treated with antipsychotic medications, D2 receptor antagonists that may interfere with the neural systems that underlie motivation and learning. Thus, it remains unknown how motivational context affects learning in schizophrenia, separate from the effects of medication. Methods: To examine the impact of motivational context on learning in schizophrenia, we tested 16 unmedicated patients with schizophrenia and 23 matched controls on a probabilistic learning task while they underwent functional magnetic resonance imaging (fMRI) under 2 conditions: one in which they pursued rewards, and one in which they avoided losses. Computational models were used to derive trial-by-trial prediction error responses to feedback. Results: Patients performed worse than controls on the learning task overall, but there were no behavioral effects of condition. FMRI revealed an attenuated prediction error response in patients in the medial prefrontal cortex, striatum, and medial temporal lobe when learning to predict rewards, but not when learning to avoid losses. Conclusions: Patients with schizophrenia showed differences in learning-related brain activity when learning to predict rewards, but not when learning to avoid losses. Together with prior work, these results suggest that motivational deficits related to learning in schizophrenia are characteristic of the disease and not solely a result of antipsychotic treatment.