David Mathar - Academia.edu (original) (raw)

Papers by David Mathar

Research paper thumbnail of The catecholamine precursor Tyrosine reduces autonomic arousal and decreases decision thresholds in reinforcement learning and temporal discounting

PLOS Computational Biology

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, ... more Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two catecholamine-dependent trans-diagnostic traits: model-based control during reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n = 28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants’ choices and response times (RTS) in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants’ RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuate...

Research paper thumbnail of Exploring temporal aspects of choice behavior in a reinforcement learning task: A reanalysis of Gillan et al. (2016) with a drift diffusion model extension

Reward-based instrumental learning processes are typically investigated in reinforcement learning... more Reward-based instrumental learning processes are typically investigated in reinforcement learning (RL) paradigms (e.g., Daw et al., 2011). Within those, choice behavior that merely relies on the history of experienced action-outcome associations is referred to as model-free RL. In contrast, choice behavior that is based on an internal model of the actual task structure is called model-based RL. The balance between these two RL strategies seems to constitute a transdiagnostic feature in a range of psychopathologies (Gillan et al., 2016). To infer participants' individual RL strategies, model-free and model-based values of the options in each trial are modeled with the help of standard Q-learning algorithms (Daw et al., 2011), and choice probabilities are modeled with a simple soft-max action selection function (Luce, 1959). In a large-scale online study, Gillan and colleagues (2016) found an association between attenuated model-based learning strategies and several psychiatric ab...

Research paper thumbnail of Obesity-related effects on response inhibition after the presentation of food stimuli

Research paper thumbnail of Weight status and eating behavior affect how the brain regulates food craving

Research paper thumbnail of Dopamine regulates decision thresholds in human reinforcement learning

Dopamine fundamentally contributes to reinforcement learning by encoding prediction errors, devia... more Dopamine fundamentally contributes to reinforcement learning by encoding prediction errors, deviations of an outcome from expectation. Prediction error coding in dopaminergic regions in human functional neuroimaging studies is well replicated. In contrast, replications of behavioral and neural effects of pharmacological modulations of the dopamine system in human reinforcement learning are scarce. Additionally, dopamine contributes to action selection, but direct evidence and process-specific accounts in human reinforcement learning are lacking. Here we examined dopaminergic mechanisms underlying human reinforcement learning in a within-subjects pharmacological approach in male human volunteers (n=31, within-subjects design; Placebo, 150mg L-dopa, 2mg Haloperidol) in combination with functional magnetic resonance imaging and a stationary reinforcement learning task. We had two aims. First, we aimed to replicate previously reported beneficial effects of L-dopa vs. Haloperidol on rein...

Research paper thumbnail of A single dose of the catecholamine precursor Tyrosine reduces physiological arousal and decreases decision thresholds in reinforcement learning and temporal discounting

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, ... more Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two key aspects of catecholamine-dependent decision-making: model-based reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n=28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants’ choices and response times in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye-blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants’ RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuated decision-thres...

Research paper thumbnail of Erotic cue exposure increases physiological arousal, biases choices towards immediate rewards and attenuates model-based reinforcement learning

Computational psychiatry focuses on identifying core cognitive processes that appear altered acro... more Computational psychiatry focuses on identifying core cognitive processes that appear altered across a broad range of psychiatric disorders. Temporal discounting of future rewards and model-based control during reinforcement learning have proven as two promising candidates. Despite its trait-like stability, temporal discounting has been suggested to be at least partly under contextual control. For example, highly arousing cues such as erotic pictures were shown to increase discounting, although overall evidence to date remains somewhat mixed. Whether model-based reinforcement learning is similarly affected by arousing cues is unclear. Here we tested cue-reactivity effects (erotic pictures) on subsequent temporal discounting and model-based reinforcement learning in a within-subjects design in n=39 healthy male participants. Self-reported and physiological arousal (cardiac activity and pupil dilation) were assessed before and during cue exposure. Arousal was increased during exposure ...

Research paper thumbnail of Obesity is associated with decreased behavioural sensitivity to reward devaluation

30. Jahrestagung der Deutschen Adipositas Gesellschaft, Nov 1, 2014

Research paper thumbnail of Obesity is associated with insufficientbehavioral adaptation

Schematic representation of the main signaling pathways ivolved in homeostatic food intake contro... more Schematic representation of the main signaling pathways ivolved in homeostatic food intake control. Within the central nervous system (CNS) hypothalamic nuclei and the nucleus of the solitary tract (NTS) process peripheral orexigenic (ghrelin) and anorexigenic (leptin, insulin, peptide Y (PYY), glucagon-like peptide (GLP-1), cholecystokinin (CCK)) signals. The hypothalamic arcuate nucleus (ARC) can be separated into anorexigenic Pro-opiomelanocortin expressing neurons (POMC) and neurons expressing agouti-related protein (AgRP)and neuropeptide Y (NPY). In response to peripheral stimulation, they either release neuropeptides and neurotransmitters into downstream neurons of the paraventricular nucleus (PVN) and the lateral hypothalamic area (LHA) to induce hunger and initiate energy intake or satiety and energy expenditure. Besides anorexigenic / orexigenic signals from PVN and LHA, the NTS within the brainstem also receives direct anorexigenic signals from the intestines.. .. .. .. .. .. 3 Schematic representation of the (1) nigrostriatal, (2) mesolimbic and (3) mesocortical DAergic pathways

Research paper thumbnail of Slave to habit

Research paper thumbnail of shared data of "Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

Research paper thumbnail of The impact of obesity, gender and impulsivity on fronto-striatal system function

Research paper thumbnail of The impact of obesity and gender on performance and strategy use in a probabilistic stimulus-response learning task

Research paper thumbnail of The impact of obesity and gender on probabilistic stimulus response learning

Research paper thumbnail of Do obesity and gender bias implicit multi-cue learning performance?

Research paper thumbnail of Do obesity and gender bias performance on implicit multi-cue learning?

Research paper thumbnail of Parameter and model recovery of reinforcement learning models for restless bandit problems

Multi-armed restless bandit tasks are regularly applied in psychology and cognitive neuroscience ... more Multi-armed restless bandit tasks are regularly applied in psychology and cognitive neuroscience to assess exploration and exploitation behavior in structured environments. These models are also readily applied to examine effects of (virtual) brain lesions on performance, and to infer neurocomputational mechanisms using neuroimaging or pharmacological approaches. However, to infer individual, psychologically meaningful parameters from such data, computational cognitive modeling is typically applied. Recent studies indicate that softmax (SM) decision rule models that include a representation of environmental dynamics (e.g. the Kalman Filter) and additional parameters for modeling exploration and perseveration (Kalman SMEP) fit human bandit task data better than competing models. Parameter and model recovery are two central requirements for computational models: parameter recovery refers to the ability to recover true data-generating parameters; model recovery refers to the ability to...

Research paper thumbnail of Dopamine release, diffusion and uptake: A computational model for synaptic and volume transmission

PLOS Computational Biology, 2020

Computational modeling of dopamine transmission is challenged by complex underlying mechanisms. H... more Computational modeling of dopamine transmission is challenged by complex underlying mechanisms. Here we present a new computational model that (I) simultaneously regards release, diffusion and uptake of dopamine, (II) considers multiple terminal release events and (III) comprises both synaptic and volume transmission by incorporating the geometry of the synaptic cleft. We were able to validate our model in that it simulates concentration values comparable to physiological values observed in empirical studies. Further, although synaptic dopamine diffuses into extra-synaptic space, our model reflects a very localized signal occurring on the synaptic level, i.e. synaptic dopamine release is negligibly recognized by neighboring synapses. Moreover, increasing evidence suggests that cognitive performance can be predicted by signal variability of neuroimaging data (e.g. BOLD). Signal variability in target areas of dopaminergic neurons (striatum, cortex) may arise from dopamine concentratio...

Research paper thumbnail of Gambling environment exposure increases temporal discounting but improves model-based control in regular slot-machine gamblers

Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, rela... more Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, relationships and mental health. In this pre-registered study (https://osf.io/5ptz9/?view_only=62956a2afcd1495984db4be18c54b50a) we investigated the impact of real-life gambling environments on two computational markers of addiction, temporal discounting and model-based reinforcement learning. Gambling disorder is associated with increased temporal discounting and reduced model-based learning. Regular gamblers (n = 30, DSM-5 score range 3-9) performed both tasks in a neutral (café) and a gambling-related environment (slot-machine venue) in counterbalanced order. Data were modeled using drift diffusion models for temporal discounting and reinforcement learning via hierarchical Bayesian estimation. Replicating previous findings, gamblers discounted rewards more steeply in the gambling-related context. This effect was positively correlated with gambling related cognitive distortions (pre-regist...

Research paper thumbnail of Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments

High-performance virtual reality (VR) technology has opened new possibilities for the examination... more High-performance virtual reality (VR) technology has opened new possibilities for the examination of the reactivity towards addiction-related cues (cue-reactivity) in addiction. In this preregistered study (https://osf.io/4mrta), we investigated the subjective, physiological, and behavioral effects of gambling-related VR environment exposure in participants reporting frequent or pathological gambling (n=31) as well as non-gambling controls (n=29). On two separate days, participants explored two rich and navigable VR-environments (neutral: café vs. gambling-related: casino/sports-betting facility), while electrodermal activity and heart rate were continuously measured using remote sensors. Within VR, participants performed a temporal discounting task and a sequential decision-making task designed to assess model-based and model-free contributions to behavior. Replicating previous findings, we found strong evidence for increased temporal discounting and reduced model-based control in ...

Research paper thumbnail of The catecholamine precursor Tyrosine reduces autonomic arousal and decreases decision thresholds in reinforcement learning and temporal discounting

PLOS Computational Biology

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, ... more Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two catecholamine-dependent trans-diagnostic traits: model-based control during reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n = 28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants’ choices and response times (RTS) in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants’ RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuate...

Research paper thumbnail of Exploring temporal aspects of choice behavior in a reinforcement learning task: A reanalysis of Gillan et al. (2016) with a drift diffusion model extension

Reward-based instrumental learning processes are typically investigated in reinforcement learning... more Reward-based instrumental learning processes are typically investigated in reinforcement learning (RL) paradigms (e.g., Daw et al., 2011). Within those, choice behavior that merely relies on the history of experienced action-outcome associations is referred to as model-free RL. In contrast, choice behavior that is based on an internal model of the actual task structure is called model-based RL. The balance between these two RL strategies seems to constitute a transdiagnostic feature in a range of psychopathologies (Gillan et al., 2016). To infer participants' individual RL strategies, model-free and model-based values of the options in each trial are modeled with the help of standard Q-learning algorithms (Daw et al., 2011), and choice probabilities are modeled with a simple soft-max action selection function (Luce, 1959). In a large-scale online study, Gillan and colleagues (2016) found an association between attenuated model-based learning strategies and several psychiatric ab...

Research paper thumbnail of Obesity-related effects on response inhibition after the presentation of food stimuli

Research paper thumbnail of Weight status and eating behavior affect how the brain regulates food craving

Research paper thumbnail of Dopamine regulates decision thresholds in human reinforcement learning

Dopamine fundamentally contributes to reinforcement learning by encoding prediction errors, devia... more Dopamine fundamentally contributes to reinforcement learning by encoding prediction errors, deviations of an outcome from expectation. Prediction error coding in dopaminergic regions in human functional neuroimaging studies is well replicated. In contrast, replications of behavioral and neural effects of pharmacological modulations of the dopamine system in human reinforcement learning are scarce. Additionally, dopamine contributes to action selection, but direct evidence and process-specific accounts in human reinforcement learning are lacking. Here we examined dopaminergic mechanisms underlying human reinforcement learning in a within-subjects pharmacological approach in male human volunteers (n=31, within-subjects design; Placebo, 150mg L-dopa, 2mg Haloperidol) in combination with functional magnetic resonance imaging and a stationary reinforcement learning task. We had two aims. First, we aimed to replicate previously reported beneficial effects of L-dopa vs. Haloperidol on rein...

Research paper thumbnail of A single dose of the catecholamine precursor Tyrosine reduces physiological arousal and decreases decision thresholds in reinforcement learning and temporal discounting

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, ... more Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two key aspects of catecholamine-dependent decision-making: model-based reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n=28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants’ choices and response times in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye-blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants’ RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuated decision-thres...

Research paper thumbnail of Erotic cue exposure increases physiological arousal, biases choices towards immediate rewards and attenuates model-based reinforcement learning

Computational psychiatry focuses on identifying core cognitive processes that appear altered acro... more Computational psychiatry focuses on identifying core cognitive processes that appear altered across a broad range of psychiatric disorders. Temporal discounting of future rewards and model-based control during reinforcement learning have proven as two promising candidates. Despite its trait-like stability, temporal discounting has been suggested to be at least partly under contextual control. For example, highly arousing cues such as erotic pictures were shown to increase discounting, although overall evidence to date remains somewhat mixed. Whether model-based reinforcement learning is similarly affected by arousing cues is unclear. Here we tested cue-reactivity effects (erotic pictures) on subsequent temporal discounting and model-based reinforcement learning in a within-subjects design in n=39 healthy male participants. Self-reported and physiological arousal (cardiac activity and pupil dilation) were assessed before and during cue exposure. Arousal was increased during exposure ...

Research paper thumbnail of Obesity is associated with decreased behavioural sensitivity to reward devaluation

30. Jahrestagung der Deutschen Adipositas Gesellschaft, Nov 1, 2014

Research paper thumbnail of Obesity is associated with insufficientbehavioral adaptation

Schematic representation of the main signaling pathways ivolved in homeostatic food intake contro... more Schematic representation of the main signaling pathways ivolved in homeostatic food intake control. Within the central nervous system (CNS) hypothalamic nuclei and the nucleus of the solitary tract (NTS) process peripheral orexigenic (ghrelin) and anorexigenic (leptin, insulin, peptide Y (PYY), glucagon-like peptide (GLP-1), cholecystokinin (CCK)) signals. The hypothalamic arcuate nucleus (ARC) can be separated into anorexigenic Pro-opiomelanocortin expressing neurons (POMC) and neurons expressing agouti-related protein (AgRP)and neuropeptide Y (NPY). In response to peripheral stimulation, they either release neuropeptides and neurotransmitters into downstream neurons of the paraventricular nucleus (PVN) and the lateral hypothalamic area (LHA) to induce hunger and initiate energy intake or satiety and energy expenditure. Besides anorexigenic / orexigenic signals from PVN and LHA, the NTS within the brainstem also receives direct anorexigenic signals from the intestines.. .. .. .. .. .. 3 Schematic representation of the (1) nigrostriatal, (2) mesolimbic and (3) mesocortical DAergic pathways

Research paper thumbnail of Slave to habit

Research paper thumbnail of shared data of "Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

Research paper thumbnail of The impact of obesity, gender and impulsivity on fronto-striatal system function

Research paper thumbnail of The impact of obesity and gender on performance and strategy use in a probabilistic stimulus-response learning task

Research paper thumbnail of The impact of obesity and gender on probabilistic stimulus response learning

Research paper thumbnail of Do obesity and gender bias implicit multi-cue learning performance?

Research paper thumbnail of Do obesity and gender bias performance on implicit multi-cue learning?

Research paper thumbnail of Parameter and model recovery of reinforcement learning models for restless bandit problems

Multi-armed restless bandit tasks are regularly applied in psychology and cognitive neuroscience ... more Multi-armed restless bandit tasks are regularly applied in psychology and cognitive neuroscience to assess exploration and exploitation behavior in structured environments. These models are also readily applied to examine effects of (virtual) brain lesions on performance, and to infer neurocomputational mechanisms using neuroimaging or pharmacological approaches. However, to infer individual, psychologically meaningful parameters from such data, computational cognitive modeling is typically applied. Recent studies indicate that softmax (SM) decision rule models that include a representation of environmental dynamics (e.g. the Kalman Filter) and additional parameters for modeling exploration and perseveration (Kalman SMEP) fit human bandit task data better than competing models. Parameter and model recovery are two central requirements for computational models: parameter recovery refers to the ability to recover true data-generating parameters; model recovery refers to the ability to...

Research paper thumbnail of Dopamine release, diffusion and uptake: A computational model for synaptic and volume transmission

PLOS Computational Biology, 2020

Computational modeling of dopamine transmission is challenged by complex underlying mechanisms. H... more Computational modeling of dopamine transmission is challenged by complex underlying mechanisms. Here we present a new computational model that (I) simultaneously regards release, diffusion and uptake of dopamine, (II) considers multiple terminal release events and (III) comprises both synaptic and volume transmission by incorporating the geometry of the synaptic cleft. We were able to validate our model in that it simulates concentration values comparable to physiological values observed in empirical studies. Further, although synaptic dopamine diffuses into extra-synaptic space, our model reflects a very localized signal occurring on the synaptic level, i.e. synaptic dopamine release is negligibly recognized by neighboring synapses. Moreover, increasing evidence suggests that cognitive performance can be predicted by signal variability of neuroimaging data (e.g. BOLD). Signal variability in target areas of dopaminergic neurons (striatum, cortex) may arise from dopamine concentratio...

Research paper thumbnail of Gambling environment exposure increases temporal discounting but improves model-based control in regular slot-machine gamblers

Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, rela... more Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, relationships and mental health. In this pre-registered study (https://osf.io/5ptz9/?view_only=62956a2afcd1495984db4be18c54b50a) we investigated the impact of real-life gambling environments on two computational markers of addiction, temporal discounting and model-based reinforcement learning. Gambling disorder is associated with increased temporal discounting and reduced model-based learning. Regular gamblers (n = 30, DSM-5 score range 3-9) performed both tasks in a neutral (café) and a gambling-related environment (slot-machine venue) in counterbalanced order. Data were modeled using drift diffusion models for temporal discounting and reinforcement learning via hierarchical Bayesian estimation. Replicating previous findings, gamblers discounted rewards more steeply in the gambling-related context. This effect was positively correlated with gambling related cognitive distortions (pre-regist...

Research paper thumbnail of Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments

High-performance virtual reality (VR) technology has opened new possibilities for the examination... more High-performance virtual reality (VR) technology has opened new possibilities for the examination of the reactivity towards addiction-related cues (cue-reactivity) in addiction. In this preregistered study (https://osf.io/4mrta), we investigated the subjective, physiological, and behavioral effects of gambling-related VR environment exposure in participants reporting frequent or pathological gambling (n=31) as well as non-gambling controls (n=29). On two separate days, participants explored two rich and navigable VR-environments (neutral: café vs. gambling-related: casino/sports-betting facility), while electrodermal activity and heart rate were continuously measured using remote sensors. Within VR, participants performed a temporal discounting task and a sequential decision-making task designed to assess model-based and model-free contributions to behavior. Replicating previous findings, we found strong evidence for increased temporal discounting and reduced model-based control in ...