New Metrics for Assessing Diagnostic Potential of Candidate ... : Clinical Journal of the American Society of Nephrology (original) (raw)
Special Features
New Metrics for Assessing Diagnostic Potential of Candidate Biomarkers
*Christchurch Kidney Research Group, Department of Medicine, University of Otago, Christchurch, New Zealand; and
†Department of Nephrology, Prince of Wales Clinical School, University of New South Wales, Sydney, Australia
Correspondence: Dr. John Pickering, Christchurch Kidney Research Group, Department of Medicine, University of Otago, PO Box 4245, Christchurch 8140, New Zealand. Email: [email protected]
Abstract
Summary
New tests should improve the diagnostic performance of available tests. The area under the receiver operator characteristic curve has been the “metric of choice” to quantify new biomarker performance. Two new metrics, the integrated discrimination improvement (IDI) and net reclassification improvement (NRI), have been rapidly adopted to quantify the added value of a biomarker to an existing test. These metrics require the development of risk prediction models that calculate the probability of an event for each individual. This study demonstrates the application of these metrics in 528 critically ill patients with risk models of AKI, sepsis, and 30-day mortality to which the biomarker urinary cystatin C was added. Analogous to the receiver operator characteristic curve, we present a new risk assessment plot for visualizing these metrics. The results showed that the NRI was sensitive to the choice of risk threshold. The risk assessment plot identified that the addition of urinary cystatin C to the model decreased the calculated risk for some who did not have sepsis but increased it for others. The category-free NRI for each outcome indicated that most of those without the event had reduced calculated risk. This was driven by very small changes in calculated risk in the AKI and death models. The IDI reflected those small changes. Of the new metrics, the IDI, reported separately for those with and without the events, best represents the value of a new test. The risk assessment plot identified differences in the models not apparent in any of the metrics.
Copyright © 2012 The Authors. Published by Wolters Kluwer Health, Inc. All rights reserved.
Full Text Access for Subscribers:
