Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond - PubMed (original) (raw)
. 2008 Jan 30;27(2):157-72; discussion 207-12.
doi: 10.1002/sim.2929.
Affiliations
- PMID: 17569110
- DOI: 10.1002/sim.2929
Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond
Michael J Pencina et al. Stat Med. 2008.
Abstract
Identification of key factors associated with the risk of developing cardiovascular disease and quantification of this risk using multivariable prediction algorithms are among the major advances made in preventive cardiology and cardiovascular epidemiology in the 20th century. The ongoing discovery of new risk markers by scientists presents opportunities and challenges for statisticians and clinicians to evaluate these biomarkers and to develop new risk formulations that incorporate them. One of the key questions is how best to assess and quantify the improvement in risk prediction offered by these new models. Demonstration of a statistically significant association of a new biomarker with cardiovascular risk is not enough. Some researchers have advanced that the improvement in the area under the receiver-operating-characteristic curve (AUC) should be the main criterion, whereas others argue that better measures of performance of prediction models are needed. In this paper, we address this question by introducing two new measures, one based on integrated sensitivity and specificity and the other on reclassification tables. These new measures offer incremental information over the AUC. We discuss the properties of these new measures and contrast them with the AUC. We also develop simple asymptotic tests of significance. We illustrate the use of these measures with an example from the Framingham Heart Study. We propose that scientists consider these types of measures in addition to the AUC when assessing the performance of newer biomarkers.
Comment in
- Net reclassification improvement and decision theory.
Vickers AJ, Elkin EB, Steyerberg E. Vickers AJ, et al. Stat Med. 2009 Feb 1;28(3):525-6; author reply 526-8. doi: 10.1002/sim.3087. Stat Med. 2009. PMID: 17907248 No abstract available. - Integrated discrimination improvement and probability-sensitive AUC variants.
Van Calster B, Van Huffel S. Van Calster B, et al. Stat Med. 2010 Jan 30;29(2):318-9. doi: 10.1002/sim.3761. Stat Med. 2010. PMID: 20027623 No abstract available.
Similar articles
Cited by
- Predictive value of contrast-enhanced MRI for the regrowth of residual uterine fibroids after high-intensity focused ultrasound treatment.
Liu Y, Xiao Z, Luo Y, Qiu X, Wang L, Deng J, Yang M, Lv F. Liu Y, et al. Insights Imaging. 2024 Nov 15;15(1):274. doi: 10.1186/s13244-024-01839-w. Insights Imaging. 2024. PMID: 39546185 Free PMC article. - Diagnostic value comparison of the combination of prostate-specific membrane antigen-body PET/MR and the prostate health index with each alone in early diagnosis of prostate cancer.
Liu J, Lin X, Huang D, Zhang M, Liu A, Ruan X, Jiang J, Huang H, Chen L, Xu D. Liu J, et al. Asian J Urol. 2024 Oct;11(4):555-562. doi: 10.1016/j.ajur.2023.10.003. Epub 2024 Feb 29. Asian J Urol. 2024. PMID: 39534010 Free PMC article. - Comprehensive analysis of psychological symptoms and quality of life in early patients with IBD: a multicenter study from China.
Liu Y, Hu J, Tian S, Zhang J, An P, Wu Y, Liu Z, Jiang C, Shi J, Wu K, Dong W. Liu Y, et al. BMC Psychiatry. 2024 Nov 12;24(1):792. doi: 10.1186/s12888-024-06247-4. BMC Psychiatry. 2024. PMID: 39533191 Free PMC article. - Assessment of risk factors associated with post-molar gestational trophoblastic neoplasia: a retrospective cohort.
Piazzetta SR, Hoch KA, Benetti-Pinto CL, Yela DA. Piazzetta SR, et al. Rev Bras Ginecol Obstet. 2024 Oct 23;46:e-rbgo83. doi: 10.61622/rbgo/2024rbgo83. eCollection 2024. Rev Bras Ginecol Obstet. 2024. PMID: 39530069 Free PMC article. - Prognostic value of serum complement cleavage factor Bb in idiopathic membranous nephropathy and establishment of nomogram model.
Wen L, Li Q, Cheng G, Zhang Y, Zhao Z. Wen L, et al. Sci Rep. 2024 Nov 8;14(1):27266. doi: 10.1038/s41598-024-78325-2. Sci Rep. 2024. PMID: 39516318 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Other Literature Sources