Statistical methods to assess the prognostic value of risk prediction rules in clinical research (original) (raw)

Abstract

Prognosis aims at estimating the future course of a given disease in probabilistic terms. As in diagnosis, where clinicians are interested in knowing the accuracy of a new test to identify patients affected by a given disease, in prognosis they wish to accurately identify patients at risk of a future event conditional to one or more prognostic factors. Thus, accurate risk predictions play a primary role in all fields of clinical medicine and in geriatrics as well because they can help clinicians to tailor the intensity of a treatment and to schedule clinical surveillance according to the risk of the concerned patient. Statistical methods able to evaluate the prognostic accuracy of a risk score demand the assessment of discrimination (the Harrell's C-index), calibration (Hosmer-May test) and risk reclassification abilities (IDI, an index of risk reclassification) of the same risk prediction rule whereas, in spite of the popular belief that traditional statistical techniques providing relative measures of effect (such as the hazard ratio derived by Cox regression analysis or the odds ratio obtained by logistic regression analysis) could be per se enough to assess the prognostic value of a biomarker or of a risk score. In this paper we provide a brief theoretical background of each statistical test and a practical approach to the issue. For didactic purposes, in the paper we also provide a dataset (n = 40) to allow the reader to train in the application of the proposed statistical methods.

FAQs

sparkles

AI

What are key statistical methods for assessing prognostic value in clinical research?add

The study emphasizes using Harrell's C-index for discrimination, Hosmer-May test for calibration, and integrated discrimination improvements (IDI) for risk reclassification.

How does the PREDICT model compare to the Ash classification for mortality predictions?add

PREDICT demonstrates a hazard ratio of 3.8 for mortality, while the Ash classification shows a ratio of 3.1, indicating comparable predictive abilities but necessitating further statistical criteria for assessment.

What is the significance of the C-index in evaluating the PREDICT model?add

The PREDICT model achieved a Harrell's C-index of 84.3%, indicating strong discrimination between patients who died and those who survived.

How did sedentary lifestyle impact the prognostic accuracy of PREDICT?add

Inclusion of sedentary lifestyle marginally improved PREDICT’s discrimination from 84.3% to 86.1%, but the improvement was not statistically significant (P = 0.22).

Why is external validation crucial for risk prediction rules?add

External validation ensures the predictive rule is reliable across different patient cohorts, which is essential for its practical application in clinical settings.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (7)

  1. Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognos- tic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361-387
  2. Crowson CS, Atkinson EJ, Therneau TM (2016) Assessing calibration of prognostic risk scores. Stat Methods Med Res 25:1692-1706
  3. Pencina MJ, D'Agostino RB Jr, D'Agostino RB Jr et al (2008) Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 27:157-172
  4. Kerkmeijer LG, Monninkhof EM, van Oort IM et al (2016) PRE- DICT: model for prediction of survival in localized prostate can- cer. World J Urol 34:789-795
  5. Schmoor C, Sauerbrei W, Schumacher M (2000) Sample size considerations for the evaluation of prognostic factors in survival analysis. Stat Med 19:441-452
  6. Ash D, Flynn A, Battermann J et al (2000) ESTRO/EAU/EORTC recommendations on permanent seed implantation for localized prostate cancer. Radiother Oncol 57:315-321
  7. Tripepi G, Heinze G, Jager KJ et al (2013) Risk prediction models. Nephrol Dial Transplant 28:1975-1980