Neuroethology of decision-making (original) (raw)
Related papers
Adaptive Behavior, 2002
Reinforcement learning is a fundamental process by which organisms learn to achieve goals from their interactions with the environment. Using evolutionary computation techniques we evolve (near-)optimal neuronal learning rules in a simple neural network model of reinforcement learning in bumblebees foraging for nectar. The resulting neural networks exhibit efficient reinforcement learning, allowing the bees to respond rapidly to changes in reward contingencies. The evolved synaptic plasticity dynamics give rise to varying exploration/exploitation levels and to the well-documented choice strategies of risk aversion and probability matching. Additionally, risk aversion is shown to emerge even when bees are evolved in a completely risk-less environment. In contrast to existing theories in economics and game theory, risk-averse behavior is shown to be a direct consequence of (near-)optimal reinforcement learning, without requiring additional assumptions such as the existence of a nonlinear subjective utility function for rewards. Our results are corroborated by a rigorous mathematical analysis, and their robustness in real-world situations is supported by experiments in a mobile robot. Thus we provide a biologically founded, parsimonious, and novel explanation for risk aversion and probability matching. Downloaded from W i (t ) = η[A · V i (t)P (t) + B · V i (t ) + C · P (t) + D] dependencies met 0 otherwise
Neuronal basis of sequential foraging decisions in a patchy environment
2011
Deciding when to leave a depleting resource to exploit another is a fundamental problem for all decision makers. The neuronal mechanisms mediating patch-leaving decisions remain unknown. We found that neurons in primate (Macaca mulatta) dorsal anterior cingulate cortex, an area that is linked to reward monitoring and executive control, encode a decision variable signaling the relative value of leaving a depleting resource for a new one.
Humans and Insects Decide in Similar Ways
PLOS One, 2010
Behavioral ecologists assume that animals use a motivational mechanism for decisions such as action selection and time allocation, allowing the maximization of their fitness. They consider both the proximate and ultimate causes of behavior in order to understand this type of decision-making in animals. Experimental psychologists and neuroeconomists also study how agents make decisions but they consider the proximate causes of the behavior. In the case of patch-leaving, motivation-based decision-making remains simple speculation. In contrast to other animals, human beings can assess and evaluate their own motivation by an introspection process. It is then possible to study the declared motivation of humans during decision-making and discuss the mechanism used as well as its evolutionary significance. In this study, we combine both the proximate and ultimate causes of behavior for a better understanding of the human decision-making process. We show for the first time ever that human subjects use a motivational mechanism similar to small insects such as parasitoids [1] and bumblebees [2] to decide when to leave a patch. This result is relevant for behavioral ecologists as it supports the biological realism of this mechanism. Humans seem to use a motivational mechanism of decision making known to be adaptive to a heterogeneously distributed resource. As hypothesized by Hutchinson et al. [3] and Wilke and Todd [4], our results are consistent with the evolutionary shaping of decision making because hominoids were hunters and gatherers on food patches for more than two million years. We discuss the plausibility of a neural basis for the motivation mechanism highlighted here, bridging the gap between behavioral ecology and neuroeconomy. Thus, both the motivational mechanism observed here and the neuroeconomy findings are most likely adaptations that were selected for during ancestral times.
2013
Selective attention in real-world environments enables subjects to direct information gathering systems towards relevant stimuli in the presence of distractors. In a dynamic visual environment that lacks cues identifying the relevance of stimuli for reaching goals, attention requires internal mechanisms to track valuable targets. Reinforcement learning (RL) provides a theoretical framework that is typically applied to describe the mechanisms underlying optimal action control (Lee et al., 2012; Schultz et al., 1997; Sutton and Barto, 1998). Here we propose a RL approach to resolve the credit assignment problem for the deployment of covert attentional selection (Balcarras et al., 2013; Kaping et al., 2011). We compared two types of RL models to explain behavior of two macaques performing a selective attention task during foraging in the space of stimulus features: we show that despite being extensively reinforced on the relevance of stimulus color for reward, monkey behavior does not ...
Decision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition
Frontiers in Ecology and Evolution, 2019
Organisms have evolved to trade priorities across various needs, such as growth, survival, and reproduction. In naturally complex environments this incurs high computational costs. Models exist for several types of decisions, e.g., optimal foraging or life history theory. However, most models ignore proximate complexities and infer simple rules specific to each context. They try to deduce what the organism must do, but do not provide a mechanistic explanation of how it implements decisions. We posit that the underlying cognitive machinery cannot be ignored. From the point of view of the animal, the fundamental problems are what are the best contexts to choose and which stimuli require a response to achieve a specific goal (e.g., homeostasis, survival, reproduction). This requires a cognitive machinery enabling the organism to make predictions about the future and behave autonomously. Our simulation framework includes three essential aspects: (a) the focus on the autonomous individual, (b) the need to limit and integrate information from the environment, and (c) the importance of goal-directed rather than purely stimulus-driven cognitive and behavioral control. The resulting models integrate cognition, decision-making, and behavior in the whole phenotype that may include the genome, physiology, hormonal system, perception, emotions, motivation, and cognition. We conclude that the fundamental state is the global organismic state that includes both physiology and the animal's subjective "mind". The approach provides an avenue for evolutionary understanding of subjective phenomena and self-awareness as evolved mechanisms for adaptive decision-making in natural environments.
Decision Making: The Neuroethological Turn
Neuron, 2014
Neuroeconomics applies models from economics and psychology to inform neurobiological studies of choice. This approach has revealed neural signatures of concepts like value, risk, and ambiguity, which are known to influence decision making. Such observations have led theorists to hypothesize a single, unified decision process that mediates choice behavior via a common neural currency for outcomes like food, money, or social praise. In parallel, recent neuroethological studies of decision making have focused on natural behaviors like foraging, mate choice, and social interactions. These decisions strongly impact evolutionary fitness and thus are likely to have played a key role in shaping the neural circuits that mediate decision making. This approach has revealed a suite of computational motifs that appear to be shared across a wide variety of organisms. We argue that the existence of deep homologies in the neural circuits mediating choice may have profound implications for understanding human decision making in health and disease.
Ecological expected utility and the mythical neural code
Cognitive Neurodynamics, 2010
Neural spikes are an evolutionarily ancient innovation that remains nature’s unique mechanism for rapid, long distance information transfer. It is now known that neural spikes sub serve a wide variety of functions and essentially all of the basic questions about the communication role of spikes have been answered. Current efforts focus on the neural communication of probabilities and utility values
The evolutionary biology of decision making
Evolutionary and psychological approaches to decision making remain largely separate endeavors. Each offers necessary techniques and perspectives which, when integrated, will aid the study of decision making in both humans and nonhuman animals. The evolutionary focus on selection pressures highlights the goals of decisions and the conditions under which different selection processes likely influence decision making. An evolutionary view also suggests that fully rational decision processes do not likely exist in nature. The psychological view proposes that cognition is hierarchically built on lower-level processes. Evolutionary approaches to decision making have not considered the cognitive building blocks necessary to implement decision strategies, thereby making most evolutionary models of behavior psychologically implausible. The synthesis of evolutionary and psychological constraints will generate more plausible models of decision making.
Revisiting foraging approaches in neuroscience
Many complex real-world decisions, such as deciding which house to buy or whether to switch jobs, involve trying to maximise reward across a sequence of choices. Optimal Foraging Theory is well suited to study these kinds of choices because it provides formal models for reward-maximisation in sequential situations. In this article, we review recent insights from foraging neuroscience, behavioural ecology and computational modelling. We find that a commonly used approach in foraging neuroscience, in which choice items are encountered at random, does not reflect the way animals direct their foraging efforts in real-world settings, nor does it reflect efficient reward-maximising behaviour. Based on this, we propose that task designs allowing subjects to encounter choice items strategically will further improve the ecological validity of foraging approaches used in neuroscience, as well as give rise to new behavioural and neural predictions that deepen our understanding of sequential, v...
The ecology of action selection: insights from artificial life
Philosophical Transactions of the Royal Society B: Biological Sciences, 2007
The problem of action selection has two components: what is selected and how is it selected? To understand what is selected, it is necessary to distinguish between behavioural and mechanistic levels of description. Animals do not choose between behaviours per se ; rather, behaviour reflects interactions among brains, bodies and environments. To understand what guides selection, it is useful to take a normative perspective that evaluates behaviour in terms of a fitness metric. This perspective, rooted in behavioural ecology, can be especially useful for understanding apparently irrational choice behaviour. This paper describes a series of models that use artificial life (AL) techniques to address the above issues. We show that successful action selection can arise from the joint activity of parallel, loosely coupled sensorimotor processes. We define a class of AL models that help to bridge the ecological approaches of normative modelling and agent- or individual-based modelling (IBM)...