Fracture Risk Assessment: State of the Art, Methodologically Unsound, or Poorly Reported? (original) (raw)

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. Johnell O, Kanis JA. An estimate of the worldwide prevalence and disability associated with osteoporotic fractures. Osteoporos Int. 2006;17(12):1726–33.
    Article PubMed CAS Google Scholar
  2. Kanis JA, Oden A, McCloskey EV, Johansson H, Wahl DA, Cooper C. A systematic review of hip fracture incidence and probability of fracture worldwide. Osteoporos Int. 2012;in press.
  3. Cummings SR, Melton LJ. Epidemiology and outcomes of osteoporotic fractures. Lancet. 2002;359:1761–67.
    Article PubMed Google Scholar
  4. Kanis JA, Johnell O. Requirements for DXA for the management of osteoporosis in Europe. Osteoporos Int. 2005;16:229–38.
    Article PubMed CAS Google Scholar
  5. Feinstein AR. Clinical judgment revisited: the distraction of quantitative models. Ann Intern Med. 1994;120:799–805.
    PubMed CAS Google Scholar
  6. Meads C, Ahmed I, Riley RD. A systematic review of breast cancer incidence risk prediction models with meta-analysis of their performance. Breast Canc Res Treat. 2012;132(2):365–77.
    Article Google Scholar
  7. Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat. 1998;52(1–3):289–303.
    Article PubMed CAS Google Scholar
  8. Mallett S, Royston P, Dutton S, Waters R, Altman DG. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med. 2010;8:20.
    Article PubMed Google Scholar
  9. • Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9:103. Highlights the general poor level of methodological conduct and reporting in studies developing clinical prediction models.
  10. Perel P, Edwards P, Wentz R, Roberts I. Systematic review of prognostic models in traumatic brain injury. BMC Med Inform Decis Mak. 2006;6:38.
    Article PubMed Google Scholar
  11. Steurer J, Haller C, Hauselmann H, Brunner F, Bachmann LM. Clinical value of prognostic instruments to identify patients with an increased risk for osteoporotic fractures: systematic review. PLoS One. 2011;6(5):e19994.
    Article PubMed CAS Google Scholar
  12. Kanis JA, Johnell O, Oden A, Johansson H, McCloskey E. FRAX and the assessment of fracture probability in men and women from the UK. Osteoporos Int. 2008;19(4):385–97.
    Article PubMed CAS Google Scholar
  13. •• Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: Validating a prognostic model. BMJ. 2009;338:b605. Provides an excellent overview of the general issues in evaluating the performance of clinical prediction models.
  14. • Moons KGM, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009;338:b606. Describes the issues in evaluating impact of clinical prediciton models on clinician behavior and patient outcomes.
  15. Moons KGM, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009;338(feb23 1):b375-b75.
    Google Scholar
  16. Royston P, Moons KGM, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. BMJ 2009;338(mar31 1):b604-b04.
    Google Scholar
  17. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–9.
    PubMed Google Scholar
  18. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2008.
    Google Scholar
  19. Vickers AJ. Prediction models in cancer care. CA: a cancer journal for clinicians. 2011;61(5):315–26.
    Google Scholar
  20. • Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol_._ 2010;37(1):31-8. Describes a new method to evalaute the clinical usefulness of a clinical prediction model without imposing one single treatment threshold.
  21. Wallace E, Smith SM, Perera-Salazar R, Vaucher P, McCowan C, Collins G, et al. Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs). BMC Med Inform Decis Mak. 2011;11:62.
    Article PubMed Google Scholar
  22. Toll DB, Janssen KJM, Vergouwe Y, Moons KGM. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085–94.
    Article PubMed CAS Google Scholar
  23. Moons KG, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 2012;Mar 7([Epub ahead of print]).
  24. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453–73.
    Article PubMed CAS Google Scholar
  25. Bleeker SE, Moll HA, Steyerberg EW, Donders ART, Derksen-Lubsen G, Grobbee DE, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56(9):826–32.
    Article PubMed CAS Google Scholar
  26. Peek N, Arts DG, Bosman RJ, van der Voort PH, de Keizer NF. External validation of prognostic models for critically ill patients required substantial sample sizes. J Clin Epidemiol. 2007;60(5):491–501.
    Article PubMed CAS Google Scholar
  27. Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012.
  28. •• Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-38. Methodological experts in clinical prediction give an overview of traditional and new methods to evaluate clinical prediction models.
  29. Vickers AJ, Cronin AM. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology. 2010;76(6):1298–301.
    Article PubMed Google Scholar
  30. Collins GS, Altman DG. An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ. 2010;340:c2442.
    Article PubMed Google Scholar
  31. Mallett S, Royston P, Waters R, Dutton S, Altman DG. Reporting performance of prognostic models in cancer: a review. BMC Med. 2010;8:21.
    Article PubMed Google Scholar
  32. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74.
    Article PubMed Google Scholar
  33. Gelman A. Scaling regression inputs by dividing by two standard deviations. Stat Med. 2008;27(15):2865–73.
    Article PubMed Google Scholar
  34. Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64(12):1463–64.
    Article Google Scholar
  35. Peduzzi P, Concato J, Feinsten AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis.2. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995;48(12):1503–12.
    Article PubMed CAS Google Scholar
  36. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–79.
    Article PubMed CAS Google Scholar
  37. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165(6):710–8.
    Article PubMed Google Scholar
  38. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58(5):475–83.
    Article PubMed Google Scholar
  39. Ettema RG, Peelen LM, Schuurmans MJ, Nierich AP, Kalkman CJ, Moons KG. Prediction models for prolonged intensive care unit stay after cardiac surgery: systematic review and validation study. Circulation. 2010;122(7):682–89.
    Article PubMed Google Scholar
  40. Burton A, Altman DG. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. Br J Cancer. 2004;91(1):4–8.
    Article PubMed CAS Google Scholar
  41. Janssen KJ, Donders AR, Harrell Jr FE, Vergouwe Y, Chen Q, Grobbee DE, et al. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol. 2010;63(7):721–7.
    Article PubMed Google Scholar
  42. Janssen KJM, Vergouwe Y, Donders ART, Harrell Jr FE, Chen Q, Grobbee DE, et al. Dealing with missing predictor values when applying clinical prediction models. Clin Chem. 2009;55(5):994–1001.
    Article PubMed CAS Google Scholar
  43. Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Meth. 2009;9:57.
    Article Google Scholar
  44. Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data withing prognostic modelling studies: a simulation study. BMC Med Res Meth. 2010;10:7.
    Article Google Scholar
  45. Moons KG, Donders RA, Stijnen T, Harrell Jr FE. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59(10):1092–101.
    Article PubMed Google Scholar
  46. Vergouwe Y, Royston P, Moons KGM, Altman DG. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol. 2010;63(2):205–14.
    Article PubMed Google Scholar
  47. White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011;30(4):377–99.
    Article PubMed Google Scholar
  48. Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338(jun29 1):b2393-b93.
    Google Scholar
  49. •• Kanis JA, Oden A, Johnell O, Johansson H, De Laet C, Brown J, et al. The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int. 2007;18(8):1033-46. This article describes the development of FRAX and the indivdual studies used in the development and validation of FRAX.
  50. National Osteoporosis Foundation. Clinician's guide to prevention and treatment of osteoporosis (available at http://www.nof.org/sites/default/files/pdfs/NOF_ClinicianGuide2009_v7.pdf). 2010.
  51. National Osteoporosis Guideline Group (NOGG). Osteoporosis: clinical guidelines for prevention and treatment. Executive Summary. 2010.
  52. Papaioannou A, Morin S, Cheung AM, Atkinson S, Brown JP, Feldman S, et al. 2010 clinical practice guidelines for the diagnosis and management of osteoporosis in Canada: summary. CMAJ. 2010;182(17):1864–73.
    Article PubMed Google Scholar
  53. • Laine C, Goodman SN, Griswold ME, Sox HC. Reproducible research: moving toward research the public can really trust. Ann Intern Med. 2007;146(6):450-53. A discussion on the importance of reproducible research.
  54. Peng RD. Reproducible research and biostatistics. Biostatistics. 2009;10(3):405–08.
    Article PubMed Google Scholar
  55. Mayor S. Ethics code for professional medical writers emphasises transparency and completeness of research reporting. BMJ. 2010;341:c7025.
    Article PubMed Google Scholar
  56. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24.
    Article PubMed Google Scholar
  57. Simera I, Altman DG. Writing a research article that is “fit for purpose”: EQUATOR Network and reporting guidelines. Evid Based Med. 2009;14(5):132–34.
    Article PubMed Google Scholar
  58. •• Steurer J, Haller C, Häuselmann H, Brunner F, Bachmann LM. Clinical value of prognostic instruments to identify patients with an increased risk for osteoporotic fractures: systematic review. PLoS One 2011;6(5):e19994. Systematic review of clinical prediction model for osteoporotic fracture, highlighting weaknesses in the development and validation of prediction models.
  59. Kanis JA, Oden A, Johansson H, McCloskey E. Pitfalls in the external validation of FRAX. Osteoporos Int. 2012;23(2):423–31.
    Article PubMed CAS Google Scholar
  60. Collins GS, Mallett S, Altman DG. Predicting risk of osteoporotic and hip fracture in the United Kingdom: prospective independent and external validation of QFractureScores. BMJ. 2011;342:d3651.
    Article PubMed Google Scholar
  61. Siontis GC, Tzoulaki I, Ioannidis JP. Predicting death: an empirical evaluation of predictive tools for mortality. Arch Intern Med. 2011;171(19):1721–6.
    Article PubMed Google Scholar
  62. Silverman SL, Calderon AD. The utility and limitations of FRAX: a US perspective. Curr Osteoporos Rep. 2010;8(4):192–7.
    Article PubMed Google Scholar
  63. van Geel TA, van den Bergh JP, Dinant GJ, Geusens PP. Individualizing fracture risk prediction. Maturitas. 2010;65(2):143–8.
    Article PubMed Google Scholar
  64. Sennerby U, Melhus H, Gedeborg R, Byberg L, Garmo H, Ahlbom A, et al. Cardiovascular diseases and risk of hip fracture. JAMA. 2009;302(15):1666–73.
    Article PubMed CAS Google Scholar
  65. Leslie WD, Lix LM, Johansson H, Oden A, McCloskey E, Kanis JA. Independent clinical validation of a Canadian FRAX tool: fracture prediction and model calibration. J Bone Miner Res. 2010;25(11):2350–8.
    Article PubMed Google Scholar
  66. • Hippisley-Cox J, Coupland C. Predicting risk of osteoporotic fracture in men and women in England and Wales: prospective derivation and validation of QFractureScores. BMJ. 2009;339:b4229. Describes the development and internal validation of QFracture in a very large cohort of general practice patients in the United Kingdom.
  67. Cummins NM, Poku EK, Towler MR, O'Driscoll OM, Ralston SH. clinical risk factors for osteoporosis in Ireland and the UK: a comparison of FRAX and QFractureScores. Calcif Tissue Int. 2011;89(2):172–77.
    Article PubMed CAS Google Scholar
  68. Hippisley-Cox J, Coupland C. Derivation and validation of updated QFracture algorithm to predict risk of osteoporotic fracture in primary care in the United Kingdom: prospective cohort study. BMJ. 2012;344:e3427.
    Google Scholar
  69. • Nguyen ND, Frost SA, Center JR, Eisman JA, Nguyen TV. Development of a nomogram for individualizing hip fracture risk in men and women. Osteoporos Int. 2007;18(8):1109-17. Describes the development of the Garvan prediction model.
  70. Nguyen ND, Frost SA, Center JR, Eisman JA, Nguyen TV. Development of prognostic nomograms for individualizing 5-year and 10-year fracture risks. Osteoporos Int. 2008;19(10):1431–44.
    Article PubMed CAS Google Scholar
  71. Langsetmo L, Nguyen TV, Nguyen ND, Kovacs CS, Prior JC, Center JR, et al. Independent external validation of nomograms for predicting risk of low-trauma fracture and hip fracture. CMAJ. 2011;183(2):E107–E14.
    Article PubMed Google Scholar
  72. Pluskiewicz W, Adamczyk P, Franek E, Leszczynski P, Sewerynek E, Wichrowska H, et al. Ten-year probability of osteoporotic fracture in 2012 Polish women assessed by FRAX and nomogram by Nguyen et al.-Conformity between methods and their clinical utility. Bone. 2010;46(6):1661–67.
    Article PubMed CAS Google Scholar
  73. Sandhu SK, Nguyen ND, Center JR, Pocock NA, Eisman JA, Nguyen TV. Prognosis of fracture: evaluation of predictive accuracy of the FRAX algorithm and Garvan nomogram. Osteoporos Int. 2010;21(5):863–71.
    Article PubMed CAS Google Scholar
  74. van Geel TA, Nguyen ND, Geusens PP, Center JR, Nguyen TV, Dinant GJ, et al. Development of a simple prognostic nomogram for individualising 5-year and 10-year absolute risks of fracture: a population-based prospective study among postmenopausal women. Ann Rheum Dis. 2011;70(1):97–7.
    Article Google Scholar
  75. Watts NB. The Fracture Risk Assessment Tool (FRAX(R)): applications in clinical practice. J Womens Health (Larchmt). 2011;20(4):525–31.
    Article Google Scholar
  76. Shepstone L, Fordham R, Lenaghan E, Harvey I, Cooper C, Gittoes N, et al. A pragmatic randomised controlled trial of the effectiveness and cost-effectiveness of screening older women for the prevention of fractures: rationale, design and methods for the SCOOP study. Osteoporos Int. 2012.
  77. Wells G, Cranney A, Peterson J, Boucher M, Shea B, Robinson V, et al. Risedronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane Database Syst Rev. 2008;23(1):CD004523.
    Google Scholar
  78. Wells GA, Cranney A, Peterson J, Boucher M, Shea B, Robinson V, et al. Alendronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane Database Syst Rev. 2008;23(1):CD001155.
    Google Scholar
  79. Wells GA, Cranney A, Peterson J, Boucher M, Shea B, Robinson V, et al. Etidronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane Database Syst Rev. 2008;23(1):CD003376.
    Google Scholar
  80. McClung M, Boonen S, Törring O, Roux C, Rizzoli R, Bone H, et al. Effect of denosumab treatment on the risk of fractures in subgroups of women with postmenopausal osteoporosis. J Bone Miner Res. 2011;Epub ahead of print.
  81. Bolland MJ, Grey A. Disparate outcomes from applying U.K. and U.S. osteoporosis treatment guidelines. J Clin Endocrinol Metab. 2010;95(4):1856–60.
    Article PubMed CAS Google Scholar
  82. Compston J, Cooper A, Cooper C, Francis R, Kanis JA, Marsh D, et al. Guidelines for the diagnosis and management of osteoporosis in postmenopausal women and men from the age of 50 years in the UK. Maturitas. 2009;62(2):105–08.
    Article PubMed CAS Google Scholar
  83. Dawson-Hughes B. National Osteoporosis Foundation Guide C. A revised clinician's guide to the prevention and treatment of osteoporosis. J Clin Endocrinol Metab. 2008;93(7):2463–65.
    Article PubMed CAS Google Scholar
  84. Compston J. Pathophysiology of atypical femoral fractures and osteonecrosis of the jaw. Osteoporos Int. 2011;22(12):2951–61.
    Article PubMed CAS Google Scholar
  85. Michaëlsson K, Schilcher J, Aspenberg P. Comment on Compston: Pathophysiology of atypical femoral fractures and osteonecrosis of the jaw. Osteoporos Int 2011;Jan 31. [Epub ahead of print].
  86. Schilcher J, Michaelsson K, Aspenberg P. Bisphosphonate use and atypical fractures of the femoral shaft. N Engl J Med. 2011;364(18):1728–37.
    Article PubMed CAS Google Scholar
  87. Siris ES, Chen YT, Abbott TA, Barrett-Connor E, Miller PD, Wehren LE, et al. Bone mineral density thresholds for pharmacological intervention to prevent fractures. Arch Intern Med. 2004;164(10):1108–12.
    Article PubMed Google Scholar
  88. Stone KL, Seeley DG, Lui LY, Cauley JA, Ensrud K, Browner WS, et al. BMD at multiple sites and risk of fracture of multiple types: long-term results from the Study of Osteoporotic Fractures. J Bone Miner Res. 2003;18(11):1947–54.
    Article PubMed Google Scholar
  89. Ankerst DP, Koniarski T, Liang Y, Leach RJ, Feng Z, Sanda MG, et al. Updating risk prediction tools: a case study in prostate cancer. Biom J. 2012;54(1):127–42.
    Article PubMed Google Scholar
  90. Janssen KJM, Moons KGM, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61(1):76–86.
    Article PubMed CAS Google Scholar
  91. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004;23(16):2567–86.
    Article PubMed Google Scholar
  92. van Houwelingen HC, Thorogood J. Construction, validation and updating of a prognostic model for kidney graft-survival. Stat Med. 1995;14(18):1999–2008.
    Article PubMed Google Scholar
  93. Calltorp J, Adami HO, Aström H, Fryklund L, Rossner S, Trolle Y, et al. Country profile: Sweden. Lancet. 1996;347(9001):587–94.
    Article PubMed CAS Google Scholar
  94. Gedeborg R, Engquist H, Berglund L, Michaelsson K. Identification of incident injuries in hospital discharge registers. Epidemiology. 2008;19(6):860–67.
    Article PubMed Google Scholar
  95. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.
    Article PubMed Google Scholar
  96. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326:41–4.
    Article PubMed Google Scholar
  97. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
    Article PubMed Google Scholar
  98. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–57.
    Article Google Scholar
  99. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM. REporting recommendations for tumour MARKer prognostic studies (REMARK). Br J Cancer. 2005;93(4):387–91.
    Article PubMed CAS Google Scholar
  100. Collins GS. Opening up multivariable prediction models: Consensus-based guidelines for transparent reporting: BMJ Blogs. 2011.

Download references