Separating mnemonic process from participant and item effects in the assessment of ROC asymmetries (original) (raw)
Related papers
Receiver operating characteristics (ROCs) in recognition memory: A review
Psychological Bulletin, 2007
Receiver operating characteristic (ROC) analysis is being used increasingly to examine the memory processes underlying recognition memory. The authors discuss the methodological issues involved in conducting and analyzing ROC results, describe the various models that have been developed to account for these results, review the behavioral empirical literature, and assess the models in light of those results. The empirical literature includes studies of item recognition, relational recognition (e.g., source and associative tests), as well as exclusion and remember-know tasks. Nine empirical regularities are described, and a number of unresolved empirical issues are identified. The results indicate that several common classes of recognition models, such as pure threshold and pure signal detection models, are inadequate to account for recognition memory, whereas several hybrid models that incorporate a signal detection-based process and a threshold recollection or attention process are in better agreement with the results. The results indicate that there are at least 2 functionally distinct component/processes underlying recognition memory. In addition, the ROC results have various implications for how recognition memory performance should be measured.
Item recognition memory and the ROC.
Journal of Experimental Psychology: Learning, Memory and Cognition, 2003
Four experiments were conducted to investigate the effects of study time, study repetition, semantic and orthographic similarity, and category length on item recognition memory receiver operating characteristics (ROCs). Analyses of ROC shape rejected A. P. Yonelinas's (1994) dual-process model. The normal unequal variance signal-detection model provided a better account of the data, except for a small but consistent excess of high-confidence errors. It was found that z-transformed ROC slope was increased by similarity, category length, and study item repetition, rejecting R. "constancy-of-slopes" generalization for these variables, but slope was relatively unaffected by massed study time.
Binary ROCs in perception and recognition memory are curved. (JEP:LMC)
2011
In recognition memory, a classic finding is that receiver operating characteristics (ROCs) are curvilinear. This has been taken to support the fundamental assumptions of signal detection theory (SDT) over discrete-state models such as the double high-threshold model (2HTM), which predicts linear ROCs. Recently, however, Bröder and Schütz (2009) challenged this argument by noting that most of the data on which support for SDT is based have involved confidence ratings. The authors argued that certain types of rating scale usage may result in curved ROCs even if the generating process is thresholded in nature. From this point of view, only ROCs constructed via experimental bias manipulations are useful for discriminating between the models. Bröder and Schütz conducted a meta-analysis and new experiments that compared SDT and the 2HTM using binary (yes-no) ROCs and found that many of these functions were linear, supporting 2HTM over SDT. We examine all the data reported by Bröder and Schütz, noting important limitations in their methodology, analyses, and conclusions. We report a new meta-analysis and 2 new experiments to examine the issue more closely while avoiding the limitations of Bröder and Schütz's study. These new data indicate that binary ROCs are curved in recognition, consistent with previous findings in perception and reasoning. Our results support classic arguments in favor of SDT and indicate that curvature in ratings ROCs is not task specific. We recommend the ratings procedure and suggest that analyses based on threshold models be treated with caution.
Recollection and familiarity in recognition memory: Evidence from ROC curves
Journal of Memory and Language, 2006
Does recognition memory rely on discrete recollection, continuous evidence, or both? Is continuous evidence sensitive to only the recency and duration of study (familiarity), or is it also sensitive to details of the study episode? Dual process theories assume recognition is based on recollection and familiarity, with only recollection providing knowledge about study details. Single process theories assume a single continuous evidence dimension that can provide information about familiarity and details. We replicated list and plural discrimination experiments requiring knowledge of details to discriminate targets from similar non-targets. We also ran modified versions of these experiments aiming to increase recollection by removing non-targets that could be discriminated by familiarity alone. Single process models provided the best trade-off between goodness-of-fit and model complexity and dual process models were able to account for the data only when they incorporated continuous evidence sensitive to details.
Recall-to-Reject in Recognition: Evidence from ROC Curves
Journal of Memory and Language, 2000
Dual-process models of recognition assume that recognition judgments are based on a fast-acting familiarity-based process and a slower, more accurate, recall-based process. Often, the recall process is assumed to operate as a recall-to-reject process in which mismatching information that is retrieved from memory is used to reject test foils that are similar to studied items. In four experiments, we use receiver operating characteristic curves to evaluate the evidence for a recall-to-reject process in recognition judgments. Recall-to-reject emerged in item recognition and was especially apparent when participants were explicitly told that it was an appropriate strategy. Recall-to-reject also emerged in associative-recognition tasks in which sufficient time was allowed for a recall process to contribute to the recognition judgments.
Binary ROCs in perception and recognition memory are curved
Journal of Experimental Psychology: Learning, Memory, and Cognition, 2012
In recognition memory, a classic finding is that receiver operating characteristics (ROCs) are curvilinear. This has been taken to support the fundamental assumptions of signal detection theory (SDT) over discrete-state models such as the double high-threshold model (2HTM), which predicts linear ROCs. Recently, however, Bröder and Schütz (2009) challenged this argument by noting that most of the data on which support for SDT is based have involved confidence ratings. The authors argued that certain types of rating scale usage may result in curved ROCs even if the generating process is thresholded in nature. From this point of view, only ROCs constructed via experimental bias manipulations are useful for discriminating between the models. Bröder and Schütz conducted a meta-analysis and new experiments that compared SDT and the 2HTM using binary (yes-no) ROCs and found that many of these functions were linear, supporting 2HTM over SDT. We examine all the data reported by Bröder and Schütz, noting important limitations in their methodology, analyses, and conclusions. We report a new meta-analysis and 2 new experiments to examine the issue more closely while avoiding the limitations of Bröder and Schütz's study. These new data indicate that binary ROCs are curved in recognition, consistent with previous findings in perception and reasoning. Our results support classic arguments in favor of SDT and indicate that curvature in ratings ROCs is not task specific. We recommend the ratings procedure and suggest that analyses based on threshold models be treated with caution.
Consciousness and …, 1996
Threshold-and signal-detection-based models have dominated theorizing about recognition memory. Building upon these theoretical frameworks, we have argued for a dualprocess model in which conscious recollection (a threshold process) and familiarity (a signal-detection process) contribute to memory performance. In the current paper we assessed several memory models by examining the effects of levels of processing and the number of presentations on recognition memory receiver operating characteristics (ROCs). In general, when the ROCs were plotted in probability space they exhibited an inverted U shape; however, when they were plotted in z space they exhibited a U shape. An examination of the ROCs showed that the dual-process model could account for the observed ROCs, but that models based solely on either threshold or signal-detection processes failed to provide a sufficient account of the data. Furthermore, an examination of subjects' introspective reports using the remember/know procedure showed that subjects were aware of recollection and familiarity and were able to consistently report on their occurrence. The remember/know data were used to accurately predict the shapes of the ROCs, and estimates of recollection and familiarity derived from the ROC data mirrored the subjective reports of these processes.
Psychonomic bulletin & review, 2001
Signal detection accounts of recognition assume that all item endorsements arise from the assessment of a single continuous indication of memory strength, even when subjects claim to categorically separate items accompanied by contextual recollection from those that are not (viz., remembering vs. knowing). Dissociations of these response types are held to occur because the former require a higher response criterion for item strength than does the latter. Meta-analytic and individual subject data suggest that when the A' metric is used, accuracy for remembering can systematically deviate from that of overall responding for individual subjects. This occurs because, unlike the symmetric and rigid receiver-operating characteristic (ROC) implied under A', empirical ROCs are asymmetric and plastic. A dual-process model predicted that the magnitude of the deviation would vary as a systematic function of the proportion of overall recognition accompanied by subjective remember report...
A direct test of the unequal-variance signal detection model of recognition memory
Psychonomic Bulletin & Review, 2007
Analyses of the receiver operating characteristic (ROC) almost invariably suggest that, on a recognition memory test, the standard deviation of memory strengths associated with the lures (σ lure ) is smaller than that of the targets (σ target ). Often, σ lure /σ target < 0.80. However, that conclusion is based on a model that assumes that the memory strength distributions are Gaussian in form. In two experiments, we investigated this issue in a more direct way by asking subjects to simply rate the memory strengths of targets and lures using a 20-point or a 99point strength scale. The results showed that the standard deviation of the ratings made to the targets (s target ) was, indeed, larger than the standard deviation of the ratings made to the lures (s lure ). Moreover, across subjects, the ratio s lure /s target correlated highly with the estimate of σ lure /σ target obtained from ROC analysis, and both estimates were, on average, approximately equal to 0.80.
Comparison of RK and confidence judgement ROCs in recognition memory
Journal of Cognitive …, 2011
CITATION 1 READS 50 7 authors, including: Some of the authors of this publication are also working on these related projects: Face recognition in schizophrenia: Do individual and average ROCs tell the same story? View project Clara D Martin Basque Center on Cognition, Brain and La…