Comparison of RK and confidence judgement ROCs in recognition memory (original) (raw)
Related papers
Binary ROCs in perception and recognition memory are curved. (JEP:LMC)
2011
In recognition memory, a classic finding is that receiver operating characteristics (ROCs) are curvilinear. This has been taken to support the fundamental assumptions of signal detection theory (SDT) over discrete-state models such as the double high-threshold model (2HTM), which predicts linear ROCs. Recently, however, Bröder and Schütz (2009) challenged this argument by noting that most of the data on which support for SDT is based have involved confidence ratings. The authors argued that certain types of rating scale usage may result in curved ROCs even if the generating process is thresholded in nature. From this point of view, only ROCs constructed via experimental bias manipulations are useful for discriminating between the models. Bröder and Schütz conducted a meta-analysis and new experiments that compared SDT and the 2HTM using binary (yes-no) ROCs and found that many of these functions were linear, supporting 2HTM over SDT. We examine all the data reported by Bröder and Schütz, noting important limitations in their methodology, analyses, and conclusions. We report a new meta-analysis and 2 new experiments to examine the issue more closely while avoiding the limitations of Bröder and Schütz's study. These new data indicate that binary ROCs are curved in recognition, consistent with previous findings in perception and reasoning. Our results support classic arguments in favor of SDT and indicate that curvature in ratings ROCs is not task specific. We recommend the ratings procedure and suggest that analyses based on threshold models be treated with caution.
Binary ROCs in perception and recognition memory are curved
Journal of Experimental Psychology: Learning, Memory, and Cognition, 2012
In recognition memory, a classic finding is that receiver operating characteristics (ROCs) are curvilinear. This has been taken to support the fundamental assumptions of signal detection theory (SDT) over discrete-state models such as the double high-threshold model (2HTM), which predicts linear ROCs. Recently, however, Bröder and Schütz (2009) challenged this argument by noting that most of the data on which support for SDT is based have involved confidence ratings. The authors argued that certain types of rating scale usage may result in curved ROCs even if the generating process is thresholded in nature. From this point of view, only ROCs constructed via experimental bias manipulations are useful for discriminating between the models. Bröder and Schütz conducted a meta-analysis and new experiments that compared SDT and the 2HTM using binary (yes-no) ROCs and found that many of these functions were linear, supporting 2HTM over SDT. We examine all the data reported by Bröder and Schütz, noting important limitations in their methodology, analyses, and conclusions. We report a new meta-analysis and 2 new experiments to examine the issue more closely while avoiding the limitations of Bröder and Schütz's study. These new data indicate that binary ROCs are curved in recognition, consistent with previous findings in perception and reasoning. Our results support classic arguments in favor of SDT and indicate that curvature in ratings ROCs is not task specific. We recommend the ratings procedure and suggest that analyses based on threshold models be treated with caution.
Recall-to-Reject in Recognition: Evidence from ROC Curves
Journal of Memory and Language, 2000
Dual-process models of recognition assume that recognition judgments are based on a fast-acting familiarity-based process and a slower, more accurate, recall-based process. Often, the recall process is assumed to operate as a recall-to-reject process in which mismatching information that is retrieved from memory is used to reject test foils that are similar to studied items. In four experiments, we use receiver operating characteristic curves to evaluate the evidence for a recall-to-reject process in recognition judgments. Recall-to-reject emerged in item recognition and was especially apparent when participants were explicitly told that it was an appropriate strategy. Recall-to-reject also emerged in associative-recognition tasks in which sufficient time was allowed for a recall process to contribute to the recognition judgments.
Recollection and familiarity in recognition memory: Evidence from ROC curves
Journal of Memory and Language, 2006
Does recognition memory rely on discrete recollection, continuous evidence, or both? Is continuous evidence sensitive to only the recency and duration of study (familiarity), or is it also sensitive to details of the study episode? Dual process theories assume recognition is based on recollection and familiarity, with only recollection providing knowledge about study details. Single process theories assume a single continuous evidence dimension that can provide information about familiarity and details. We replicated list and plural discrimination experiments requiring knowledge of details to discriminate targets from similar non-targets. We also ran modified versions of these experiments aiming to increase recollection by removing non-targets that could be discriminated by familiarity alone. Single process models provided the best trade-off between goodness-of-fit and model complexity and dual process models were able to account for the data only when they incorporated continuous evidence sensitive to details.
ROC (Tiberghien et al, 2014).pdf
2014): Face recognition in schizophrenia: do individual and average ROCs tell the same story?, Cognitive Neuropsychiatry, Introduction.
Cognitive Research: Principles and Implications
Researchers use a wide range of confidence scales when measuring the relationship between confidence and accuracy in reports from memory, with the highest number usually representing the greatest confidence (e.g., 4-point, 20-point, and 100-point scales). The assumption seems to be that the range of the scale has little bearing on the confidence-accuracy relationship. In two old/new recognition experiments, we directly investigated this assumption using word lists (Experiment 1) and faces (Experiment 2) by employing 4-, 5-, 20-, and 100-point scales. Using confidence-accuracy characteristic (CAC) plots, we asked whether confidence ratings would yield similar CAC plots, indicating comparability in use of the scales. For the comparisons, we divided 100-point and 20-point scales into bins of either four or five and asked, for example, whether confidence ratings of 4, 16-20, and 76-100 would yield similar values. The results show that, for both types of material, the different scales yield similar CAC plots. Notably, when subjects express high confidence, regardless of which scale they use, they are likely to be very accurate (even though they studied 100 words and 50 faces in each list in 2 experiments). The scales seem convertible from one to the other, and choice of scale range probably does not affect research into the relationship between confidence and accuracy. High confidence indicates high accuracy in recognition in the present experiments.
Receiver operating characteristics (ROCs) in recognition memory: A review
Psychological Bulletin, 2007
Receiver operating characteristic (ROC) analysis is being used increasingly to examine the memory processes underlying recognition memory. The authors discuss the methodological issues involved in conducting and analyzing ROC results, describe the various models that have been developed to account for these results, review the behavioral empirical literature, and assess the models in light of those results. The empirical literature includes studies of item recognition, relational recognition (e.g., source and associative tests), as well as exclusion and remember-know tasks. Nine empirical regularities are described, and a number of unresolved empirical issues are identified. The results indicate that several common classes of recognition models, such as pure threshold and pure signal detection models, are inadequate to account for recognition memory, whereas several hybrid models that incorporate a signal detection-based process and a threshold recollection or attention process are in better agreement with the results. The results indicate that there are at least 2 functionally distinct component/processes underlying recognition memory. In addition, the ROC results have various implications for how recognition memory performance should be measured.