A Hierarchical Unequal-Variance Signal Detection Model for Binary Data (original) (raw)
Related papers
A direct test of the unequal-variance signal detection model of recognition memory
Psychonomic Bulletin & Review, 2007
Analyses of the receiver operating characteristic (ROC) almost invariably suggest that, on a recognition memory test, the standard deviation of memory strengths associated with the lures (σ lure ) is smaller than that of the targets (σ target ). Often, σ lure /σ target < 0.80. However, that conclusion is based on a model that assumes that the memory strength distributions are Gaussian in form. In two experiments, we investigated this issue in a more direct way by asking subjects to simply rate the memory strengths of targets and lures using a 20-point or a 99point strength scale. The results showed that the standard deviation of the ratings made to the targets (s target ) was, indeed, larger than the standard deviation of the ratings made to the lures (s lure ). Moreover, across subjects, the ratio s lure /s target correlated highly with the estimate of σ lure /σ target obtained from ROC analysis, and both estimates were, on average, approximately equal to 0.80.
A common signal detection model describes threshold and supra-threshold performance
2012
Psychophysical experiments are typically based on analyzing observer choices as a function of some stimulus dimension in order to make inferences about the underlying sensory and/or decision processes that account for the observer’s choices. Modern psychophysical theory derives from Signal Detection Theory (SDT) [1, 2] in which the observer’s performance depends on a noise contaminated decision variable that in association with a criterion that determines the rates of both successful classifications and errors. When the decision rule can be characterized as a linear predictor, the framework can be formalized as a Generalized Linear Model (GLM) with a binomial family, facilitating estimating of model parameters by maximum likelihood. The Gaussian, equal-variance model is the most commonly employed which leads naturally to the use of a probit link. The largest body of psychophysical work is based on discrimination of small (threshold) stimulus differences, yielding measures of percept...
Signal Detection Models with Random Participant and Item Effects
Psychometrika, 2007
The theory of signal detection is convenient for measuring mnemonic ability in recognition memory paradigms. In these paradigms, randomly selected participants are asked to study randomly selected items. In practice, researchers aggregate data across items or participants or both. The signal detection model is nonlinear; consequently, analysis with aggregated data is not consistent. In fact, mnemonic ability is underestimated, even in the large-sample limit. We present two hierarchical Bayesian models that simultaneously account for participant and item variability. We show how these models provide for accurate estimation of participants' mnemonic ability as well as the memorability of items. The model is benchmarked with a simulation study and applied to a novel data set.
A dynamic stimulus-driven model of signal detection
Psychological Review, 2011
Signal detection theory forms the core of many current models of cognition, including memory, choice, and categorization. However, the classic signal detection model presumes the a priori existence of fixed stimulus representations-usually Gaussian distributions-even when the observer has no experience with the task. Furthermore, the classic signal detection model requires the observer to place a response criterion along the axis of stimulus strength, and without theoretical elaboration, this criterion is fixed and independent of the observer's experience. We present a dynamic, adaptive model that addresses these 2 long-standing issues. Our model describes how the stimulus representation can develop from a rough subjective prior and thereby explains changes in signal detection performance over time. The model structure also provides a basis for the signal detection decision that does not require the placement of a criterion along the axis of stimulus strength. We present simulations of the model to examine its behavior and several experiments that provide data to test the model. We also fit the model to recognition memory data and discuss the role that feedback plays in establishing stimulus representations.
Testing signal-detection models of yes/no and two-alternative forced-choice recognition memory
Journal of experimental psychology. General, 2009
The current study compared 3 models of recognition memory in their ability to generalize across yes/no and 2-alternative forced-choice (2AFC) testing. The unequal-variance signal-detection model assumes a continuous memory strength process. The dual-process signal-detection model adds a thresholdlike recollection process to a continuous familiarity process. The mixture signal-detection model assumes a continuous memory strength process, but the old item distribution consists of a mixture of 2 distributions with different means. Prior efforts comparing the ability of the models to characterize data from both test formats did not consider the role of parameter reliability, which can be critical when comparing models that differ in flexibility. Parametric bootstrap simulations revealed that parameter regressions based on separate fits of each test type only served to identify the least flexible model. However, simultaneous fits of receiver-operating characteristic data from both test t...
Psychonomic Bulletin & Review, 2011
We tested whether the unequal-variance signaldetection (UVSD) and dual-process signal-detection (DPSD) models of recognition memory mimic the behavior of each other when applied to individual data. Replicating previous results, there was no mimicry for an analysis that fit each individual, summed the goodness-of-fit values over individuals, and compared the two sums (i.e., a single model selection). However, when the models were compared separately for each individual (i.e., multiple model selections), mimicry was substantial. To quantify the diagnosticity of the individual data, we used mimicry to calculate the probability of making a model selection error for each individual. For nondiagnostic data (high model selection error), the results were compatible with equalvariance signal-detection theory. Although neither model was justified in this situation, a forced-choice between the UVSD and DPSD models favored the DPSD model for being less flexible. For diagnostic data (low model selection error), the UVSD model was selected more often.
Calculation of signal detection theory measures
Behavior Research Methods, Instruments, & Computers, 1999
Proper application of SOT requires an understanding of the theory and the measures it prescribes. We present an overview of SOT here; for more extensive discussions, see Green and Swets (1966) or Macmillan and Creelman (1991). Readers who are already familiar with SDT may wish to skip this section. SOT can be applied whenever two possible stimulus types must be discriminated. Psychologists first applied the theory in studies of perception, where subjects discriminated between signals (stimuli) and noise (no stimuli). The signal and noise labels remain, but SOT has since been applied in many other areas. Examples (and their corresponding signal and noise stimuli) include recognition memory (old and new items), lie detection (lies and truths), personnel selection (desirable and undesirable applicants), jury decision making (guilty and innocent defendants), medical diagnosis (diseased and well patients), industrial inspection (unacceptable and acceptable items), and information retrieval (relevant and irrelevant information; see also Hutchinson, 1981; Swets, 1973; and the extensive bibliographies compiled by Swets, 1988b, pp. 685-742). Performance in each of these areas may be studied with a variety of tasks. We deal here with three of the most popular: yes/no tasks, rating tasks, and forced-choice tasks. YeslNo Tasks A yes/no task involves signal trials, which present one or more signals, and noise trials, which present one or more noise stimuli. For example, yes/no tasks in auditory perception may present a tone during signal trials and nothing at all during noise trials, whereas yes/no tasks for memory may present old (previously studied) words during signal trials and new (distractor) words during noise trials. After each trial, the subjects indicate whether a sig-137
Decision noise: An explanation for observed violations of signal detection theory
Psychonomic Bulletin & Review, 2008
In signal detection theory (SDT), responses are governed by perceptual noise and a flexible decision criterion. Recent criticisms of SDT (see, e.g., Balakrishnan, 1999) have identified violations of its assumptions, and researchers have suggested that SDT fundamentally misrepresents perceptual and decision processes. We hypothesize that, instead, these violations of SDT stem from decision noise: the inability to use deterministic response criteria. In order to investigate this hypothesis, we present a simple extension of SDT—the decision noise model—with which we demonstrate that shifts in a decision criterion can be masked by decision noise. In addition, we propose a new statistic that can help identify whether the violations of SDT stem from perceptual or from decision processes. The results of a stimulus classification experiment—together with model fits to past experiments—show that decision noise substantially affects performance. These findings suggest that decision noise is important across a wide range of tasks and needs to be better understood in order to accurately measure perceptual processes.