Michael J Frank | Brown University (original) (raw)

Papers by Michael J Frank

Research paper thumbnail of Model-based cognitive neuroscience approaches to computational psychiatry: clustering and classification

Clinical Psychological Science

Psychiatric research is in crisis. We highlight efforts to overcome current challenges, focusing ... more Psychiatric research is in crisis. We highlight efforts to overcome current challenges, focusing on the emerging field of computational psychiatry, which might enable us to move from a symptombased description of mental illness to descriptors based on objective computational multidimensional functional variables. We survey recent efforts towards this goal, and describe a set of methods that together form a toolbox to aid this research program. We identify four levels in computational psychiatry: (i) Behavioral tasks indexing various psychological processes; (ii) computational models that identify the generative psychological processes; (iii) parameter estimation methods concerned with quantitatively fitting these models to subject behavior, focusing on hierarchical Bayesian estimation as a rich framework with many desirable properties; and (iv) machine learning clustering methods which identify clinically significant conditions and subgroups of individuals. As a proof of principle we apply these methods to two different data sets. Finally, we highlight challenges for future research.

Research paper thumbnail of Neural mechanisms of acquired phasic dopamine responses in learning

What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminer... more What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminergic neurons, and how do they relate to the complex constellation of empirical findings understood as Pavlovian and instrumental conditioning?

Research paper thumbnail of A computational model of executive control in frontal cortex and basal ganglia: multiple levels of analysis

Abstract: Planning and executing volitional actions in the face of conflicting habitual responses... more Abstract: Planning and executing volitional actions in the face of conflicting habitual responses is a critical aspect of human behavior. At the core of the interplay between these two control systems lies an override mechanism that can suppress the habitual action selection process and allow executive control to take over.

Research paper thumbnail of Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder

Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline ... more Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit/hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in'Go'learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations.

Research paper thumbnail of Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning

Abstract What are the genetic and neural components that support adaptive learning from positive ... more Abstract What are the genetic and neural components that support adaptive learning from positive and negative outcomes? Here, we show with genetic analyses that three independent dopaminergic mechanisms contribute to reward and avoidance learning in humans. A polymorphism in the DARPP-32 gene, associated with striatal dopamine function, predicted relatively better probabilistic reward learning.

Research paper thumbnail of Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism

Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both fron... more Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA “overdose.” These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system.

Research paper thumbnail of Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction

BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate stri... more BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate striatal-cortical pathways. We investigated whether fronto-striatal dysfunction in schizophrenia (SZ) is characterized by selective impairment in either reward-(Go) or punishment-driven (NoGo) learning. METHODS: We administered two versions of a probabilistic selection task to 40 schizophrenia patients and 31 control subjects, using difficult to verbalize stimuli (experiment 1) and nameable objects (experiment 2).

Research paper thumbnail of By carrot or by stick: cognitive reinforcement learning in parkinsonism

Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? ... more Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error.

Research paper thumbnail of Hippocampus, cortex, and basal ganglia: Insights from computational models of complementary learning systems

We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work t... more We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning.

Research paper thumbnail of Frontal theta links prediction errors to behavioral adaptation in reinforcement learning

Investigations into action monitoring have consistently detailed a frontocentral voltage deflecti... more Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested.

Research paper thumbnail of Task-related dissociation in ERN amplitude as a function of obsessive-compulsive symptoms

Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have... more Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have been implicated to underlie obtrusive thoughts and repetitive behaviors in Obsessive-Compulsive Disorder (OCD). Larger Error-Related Negativities (ERNs) in OCD patients during simple flanker tasks have been proposed to reflect an amplified error signal in these hyperactive circuits.

Research paper thumbnail of Computational models of motivated action selection in corticostriatal circuits

Computational models of the basal ganglia have matured and received increasing attention over the... more Computational models of the basal ganglia have matured and received increasing attention over the last decade. This article reviews some of the theoretical advances offered by these models, focusing on motor and cognitive action selection, learning, and the interaction between multiple corticostriatal circuits in selection and learning.

Research paper thumbnail of Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: Computational analysis

Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, ... more Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically.

Research paper thumbnail of From reinforcement learning models to psychiatric and neurological disorders

Over the last decade and a half, reinforcement learning models have fostered an increasingly soph... more Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits.

Research paper thumbnail of The neurogenetics of exploration and exploitation Supplementary Material

As noted in the main text, the CEVR condition was included to compare with CEV as a measure of pr... more As noted in the main text, the CEVR condition was included to compare with CEV as a measure of probability-magnitude bias (PM-bias= CEVR-CEV). We found PM-bias to be positive across all groups (p<. 001; Figure S1), as participants avoided responding early in CEVR due to the low reward probability at that time, consistent with loss-aversion (1).

Research paper thumbnail of Social stress reactivity alters reward and punishment learning

Abstract To examine how stress affects cognitive functioning, individual differences in trait vul... more Abstract To examine how stress affects cognitive functioning, individual differences in trait vulnerability (punishment sensitivity) and state reactivity (negative affect) to social evaluative threat were examined during concurrent reinforcement learning. Lower trait-level punishment sensitivity predicted better reward learning and poorer punishment learning; the opposite pattern was found in more punishment sensitive individuals.

Research paper thumbnail of Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation

Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning,... more Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning, previous theoretical and empirical accounts have suggested that increased uncertainty may precede the decision to explore an alternative option. To date, the neural mechanisms that support the strategic application of uncertainty-driven exploration remain underspecified. In this study, electroencephalography (EEG) was used to assess trial-to-trial dynamics relevant to exploration and exploitation.

Research paper thumbnail of Patients with schizophrenia demonstrate inconsistent preference judgments for affective and nonaffective stimuli

Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report le... more Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report levels of emotional experience that are similar to controls (CN) when asked to view a single evocative stimulus and make an absolute judgment of stimulus “value.” However, value is rarely assigned in absolute terms in real-life situations, where one alternative or experience is often evaluated alongside others, and value judgments are made in relative terms.

Research paper thumbnail of CNTRICS final task selection: long-term memory

Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of in... more Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia.

Research paper thumbnail of A Role for Dopamine-Mediated Learning in the Pathophysiology and Treatment of Parkinson’s Disease

Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervati... more Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade.

Research paper thumbnail of Model-based cognitive neuroscience approaches to computational psychiatry: clustering and classification

Clinical Psychological Science

Psychiatric research is in crisis. We highlight efforts to overcome current challenges, focusing ... more Psychiatric research is in crisis. We highlight efforts to overcome current challenges, focusing on the emerging field of computational psychiatry, which might enable us to move from a symptombased description of mental illness to descriptors based on objective computational multidimensional functional variables. We survey recent efforts towards this goal, and describe a set of methods that together form a toolbox to aid this research program. We identify four levels in computational psychiatry: (i) Behavioral tasks indexing various psychological processes; (ii) computational models that identify the generative psychological processes; (iii) parameter estimation methods concerned with quantitatively fitting these models to subject behavior, focusing on hierarchical Bayesian estimation as a rich framework with many desirable properties; and (iv) machine learning clustering methods which identify clinically significant conditions and subgroups of individuals. As a proof of principle we apply these methods to two different data sets. Finally, we highlight challenges for future research.

Research paper thumbnail of Neural mechanisms of acquired phasic dopamine responses in learning

What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminer... more What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminergic neurons, and how do they relate to the complex constellation of empirical findings understood as Pavlovian and instrumental conditioning?

Research paper thumbnail of A computational model of executive control in frontal cortex and basal ganglia: multiple levels of analysis

Abstract: Planning and executing volitional actions in the face of conflicting habitual responses... more Abstract: Planning and executing volitional actions in the face of conflicting habitual responses is a critical aspect of human behavior. At the core of the interplay between these two control systems lies an override mechanism that can suppress the habitual action selection process and allow executive control to take over.

Research paper thumbnail of Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder

Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline ... more Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit/hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in'Go'learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations.

Research paper thumbnail of Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning

Abstract What are the genetic and neural components that support adaptive learning from positive ... more Abstract What are the genetic and neural components that support adaptive learning from positive and negative outcomes? Here, we show with genetic analyses that three independent dopaminergic mechanisms contribute to reward and avoidance learning in humans. A polymorphism in the DARPP-32 gene, associated with striatal dopamine function, predicted relatively better probabilistic reward learning.

Research paper thumbnail of Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism

Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both fron... more Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA “overdose.” These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system.

Research paper thumbnail of Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction

BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate stri... more BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate striatal-cortical pathways. We investigated whether fronto-striatal dysfunction in schizophrenia (SZ) is characterized by selective impairment in either reward-(Go) or punishment-driven (NoGo) learning. METHODS: We administered two versions of a probabilistic selection task to 40 schizophrenia patients and 31 control subjects, using difficult to verbalize stimuli (experiment 1) and nameable objects (experiment 2).

Research paper thumbnail of By carrot or by stick: cognitive reinforcement learning in parkinsonism

Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? ... more Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error.

Research paper thumbnail of Hippocampus, cortex, and basal ganglia: Insights from computational models of complementary learning systems

We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work t... more We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning.

Research paper thumbnail of Frontal theta links prediction errors to behavioral adaptation in reinforcement learning

Investigations into action monitoring have consistently detailed a frontocentral voltage deflecti... more Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested.

Research paper thumbnail of Task-related dissociation in ERN amplitude as a function of obsessive-compulsive symptoms

Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have... more Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have been implicated to underlie obtrusive thoughts and repetitive behaviors in Obsessive-Compulsive Disorder (OCD). Larger Error-Related Negativities (ERNs) in OCD patients during simple flanker tasks have been proposed to reflect an amplified error signal in these hyperactive circuits.

Research paper thumbnail of Computational models of motivated action selection in corticostriatal circuits

Computational models of the basal ganglia have matured and received increasing attention over the... more Computational models of the basal ganglia have matured and received increasing attention over the last decade. This article reviews some of the theoretical advances offered by these models, focusing on motor and cognitive action selection, learning, and the interaction between multiple corticostriatal circuits in selection and learning.

Research paper thumbnail of Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: Computational analysis

Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, ... more Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically.

Research paper thumbnail of From reinforcement learning models to psychiatric and neurological disorders

Over the last decade and a half, reinforcement learning models have fostered an increasingly soph... more Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits.

Research paper thumbnail of The neurogenetics of exploration and exploitation Supplementary Material

As noted in the main text, the CEVR condition was included to compare with CEV as a measure of pr... more As noted in the main text, the CEVR condition was included to compare with CEV as a measure of probability-magnitude bias (PM-bias= CEVR-CEV). We found PM-bias to be positive across all groups (p<. 001; Figure S1), as participants avoided responding early in CEVR due to the low reward probability at that time, consistent with loss-aversion (1).

Research paper thumbnail of Social stress reactivity alters reward and punishment learning

Abstract To examine how stress affects cognitive functioning, individual differences in trait vul... more Abstract To examine how stress affects cognitive functioning, individual differences in trait vulnerability (punishment sensitivity) and state reactivity (negative affect) to social evaluative threat were examined during concurrent reinforcement learning. Lower trait-level punishment sensitivity predicted better reward learning and poorer punishment learning; the opposite pattern was found in more punishment sensitive individuals.

Research paper thumbnail of Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation

Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning,... more Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning, previous theoretical and empirical accounts have suggested that increased uncertainty may precede the decision to explore an alternative option. To date, the neural mechanisms that support the strategic application of uncertainty-driven exploration remain underspecified. In this study, electroencephalography (EEG) was used to assess trial-to-trial dynamics relevant to exploration and exploitation.

Research paper thumbnail of Patients with schizophrenia demonstrate inconsistent preference judgments for affective and nonaffective stimuli

Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report le... more Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report levels of emotional experience that are similar to controls (CN) when asked to view a single evocative stimulus and make an absolute judgment of stimulus “value.” However, value is rarely assigned in absolute terms in real-life situations, where one alternative or experience is often evaluated alongside others, and value judgments are made in relative terms.

Research paper thumbnail of CNTRICS final task selection: long-term memory

Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of in... more Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia.

Research paper thumbnail of A Role for Dopamine-Mediated Learning in the Pathophysiology and Treatment of Parkinson’s Disease

Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervati... more Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade.