Human self-control and the density of reinforcement (original) (raw)
Related papers
Self-control in adult humans: variation in positive reinforcer amount and delay
Journal of The Experimental Analysis of Behavior, 1986
In five experiments, choice responding of female human adults was examined, as a function of variations in reinforcer amount and reinforcer delay. Experiment 1 used a discrete-trials procedure, and Experiments 2, 3, 4, and 5 used a concurrent variable-interval variable-interval schedule. Reinforcer amount and reinforcer delay were varied both separately and together. In contrast to results previously reported with pigeons, the subjects in the present experiments usually chose the larger reinforcers even when those reinforcers were delayed. Together, the results from all the experiments suggest that the subjects followed a maximization strategy in choosing reinforcers. Such behavior makes it easy to observe self-control and difficult to observe impulsiveness in traditional laboratory experiments that use adult human subjects.
Self-Control and Responding during Reinforcement Delay
Annals of The New York Academy of Sciences, 1984
Self-control has been defined as the choice of a delayed large reinforcer over a less-delayed small reinforcer.'** Mazur and Logue' increased self-control in pigeons by first giving the pigeons a choice between large or small equally delayed reinforcers. Then they slowly removed the delay to the small reinforcer. The pigeons continued to choose the large, delayed reinforcer more often than did a control group not exposed to this fading procedure. Grosch and Neuringer4 showed that pecking a different key while pigeons were waiting could also improve self-control. In many real-life situations, individuals can make various responses during the large reinforcer delay, including changing their choice. However, many previous experiments that have attempted to train self-control have involved irreversible choices of the large reinfor~er.~*'*~ Therefore, it would be helpful in understanding self-control to design a procedure in which self-control is trained while subjects have the opportunity to change their choice during the large-reinforcer delay or to make other responses.
Self-control and impulsiveness with asynchronous presentation of reinforcement schedules
Behavioural Processes, 2002
In discrete trials, pigeons were presented with two alternatives: to wait for a larger reinforcer, or to respond and obtain a smaller reinforcer immediately. The choice of the former was defined as self-control, and the choice of the latter as impulsiveness. The stimulus that set the opportunity for an impulsive choice was presented after a set interval from the onset of the stimulus that signaled the waiting period. That interval increased or decreased from session to session so that the opportunity for an impulsive choice became available either more removed from or closer in time to the presentation of the larger reinforcer. In three separate conditions, the larger reinforcer was delivered according to either a fixed interval (FI) schedule, a fixed time (FT) schedule, or a differential reinforcement of other behavior (DRO) schedule. The results showed that impulsive choices increased as the opportunity for such a choice was more distant in time from presentation of the larger reinforcer. Although the schedule of the larger reinforcer affected the rate of response in the waiting period, the responses themselves had no effect on choice unless the responses postponed presentation of the larger reinforcer.
Enhancing “self-control”: The paradoxical effect of delay of reinforcement
Learning & Behavior
Delay of reinforcement is generally thought to be inversely correlated with speed of acquisition. However, in the case of simultaneous discrimination learning, in which choice results in immediate reinforcement, delay of reinforcement can improve acquisition. For example, in the ephemeral reward task, animals are given a choice between two alternatives, A and B. Choice of A provides reinforcement, and the trial is over. Choice of B provides reinforcement and access to alternative A (thus, two reinforcements). Many animals appear unable to learn to choose B consistently, but inserting a 20-s delay between choice and outcome has been shown to facilitate optimal choice. Similarly, pigeons given a choice between a signal for one pellet and a signal for two pellets (each occurring without a delay) have difficulty learning to choose the two-pellet alternative, unless the reinforcement is delayed. In a version of object permanence, food is placed in one of two containers, and the pigeon must choose the container with the food. Pigeons have difficulty reliably choosing the correct container unless a brief delay is inserted between baiting and choice. Finally, pigeons have been shown to prefer a suboptimal alternative (a 20% chance of getting a cue for reinforcement) over an optimal alternative (a 100% chance of getting a cue for 50% reinforcement). However, if pigeons are forced to wait 20 s following their choice to receive the cues, no preference for the suboptimal alternative is found. Thus, impulsive choice may be reduced by delaying the consequence of that choice.
THE EFFECT OF RESPONSE RATE ON REWARD VALUE IN A SELF-CONTROL TASK
To understand how effort, defined by number of responses required to obtain a reward, affects reward value, five pigeons were exposed to a self-control task. They chose between two alternatives, 2 s of access to food after a delay of 10 s, and 6 s of access to food after an adjusting delay. The adjusting delay increased or decreased depending on the pigeons' choices. The delay at which the two alternatives were equally chosen defined the indifference point. To determine whether requiring responses during the delay led to more impulsive (smaller-sooner rewards) or self-controlled (larger-later rewards) choices, we varied the number of required pecks during the 10-s delay to the 2-s reinforcer, and assessed how the requirement affected the indifference points. In the High Rate Phase, they had to peck at least 10 times during the delay; in the Low Rate Phase, they could peck at most 5 times during the delay. For four pigeons the indifference point increased with the response requirement; for one pigeon it decreased. The results suggest that, in general, reward value varies inversely with effort.
BMC neuroscience, 2006
Impulsivity is defined as intolerance/aversion to waiting for reward. In intolerance-to-delay (ID) protocols, animals must choose between small/soon (SS) versus large/late (LL) rewards. In the probabilistic discount (PD) protocols, animals are faced with choice between small/sure (SS) versus large/luck-linked (LLL) rewards. It has been suggested that PD protocols also measure impulsivity, however, a clear dissociation has been reported between delay and probability discounting. Wistar adolescent rats (30- to 46-day-old) were tested using either protocol in drug-free state. In the ID protocol, animals showed a marked shift from LL to SS reward when delay increased, and this despite adverse consequences on the total amount of food obtained. In the PD protocol, animals developed a stable preference for LLL reward, and maintained it even when SS and LLL options were predicted and demonstrated to become indifferent. We demonstrate a clear dissociation between these two protocols. In the ...
Is it patience or motivation? On motivational confounds in intertemporal choice tasks
Journal of the Experimental Analysis of Behavior, 2014
Intertemporal choices create a tension between amount maximization, which would favor the larger and later option (LL), and delay minimization, which would promote the smaller and sooner reward (SS). Two common interpretations of intertemporal choice behavior are discussed: looking at LL responses as indicative of self-control, and using intertemporal choices to assess delay aversion. We argue that both interpretations need to take into account motivational confounds, in order to be warranted by data. In intertemporal choices with prepotent, salient stimuli (e.g., food amounts, typically used with nonhuman primates), LL responses could also be indicative of failed inhibition of a "go for more" impulsive response-the opposite of self-control. Similarly, intertemporal choices can be used to measure delay aversion only with respect to the subject's baseline motivation to maximize the reinforcer in question, and this baseline is not always assessed in current experimental protocols. This concern is especially crucial in comparing intertemporal choices across different groups or manipulation. We focus in particular on the effects of reward types on intertemporal choices, presenting two experimental studies where the difference in behavior with monetary versus food rewards is the product of different baseline motivation, rather than variations in delay aversion. We conclude discussing the implications of these and other similar recent findings, which are far-reaching.
Responding during reinforcement delay in a self-control paradigm
Journal of The Experimental Analysis of Behavior, 1984
Eight pigeons chose between a small, immediate reinforcer and a large, increasingly delayed reinforcer. Responding during the large-reinforcer delays was examined. During large-reinforcer delays, pecks on one key produced the small, immediate reinforcer; pecks on the other key had no effect. Thus, a pigeon could reverse its initial choice of the large, delayed reinforcer, or it could maintain its original choice. Pigeons that made a relatively high number of initial large-reinforcer choices tended to maintain these choices. Those pigeons that made a relatively high number of initial large-reinforcer choices, and those pigeons that actually received a relatively high number of large reinforcers, tended to respond more frequently on the ineffective key during the delay periods. The findings suggest that some previous studies of self-control training in pigeons may have resulted in increased self-control partially due to a lack of opportunity for the pigeons to change their choices.
Human Risky Choice: Delay Sensitivity Depends on Reinforcer Type
The present study was designed to help bridge the methodological gap between human and nonhuman animal research in delay-based risky choice. In Part 1, 4 adult human subjects made repeated choices between variable-time and fixed-time schedules of 30-s video clips. Both alternatives had equal mean delays of 15 s, 30 s, or 60 s. Three of 4 subjects strongly preferred the variable-delay alternative across all conditions. In Part 2, these 3 subjects were then provided pairwise choices between 2 variable-time schedules with different delay distributions. Subjects generally preferred the variabledelay distributions with a higher probability of short-reinforcer delays, consistent with accounts based on nonlinear discounting of delayed reinforcement. There was only weak correspondence between experimental results and verbal reports. The overall pattern of results is inconsistent with prior risky choice research with human subjects but is consistent with prior results with nonhuman subjects, suggesting that procedural differences may be a critical factor determining risk-sensitivity across species.