Empirical evidence of design-related bias in studies of diagnostic tests - PubMed (original) (raw)
Empirical evidence of design-related bias in studies of diagnostic tests
J G Lijmer et al. JAMA. 1999.
Erratum in
- JAMA 2000 Apr 19;283(15):1963
Abstract
Context: The literature contains a large number of potential biases in the evaluation of diagnostic tests. Strict application of appropriate methodological criteria would invalidate the clinical application of most study results.
Objective: To empirically determine the quantitative effect of study design shortcomings on estimates of diagnostic accuracy.
Design and setting: Observational study of the methodological features of 184 original studies evaluating 218 diagnostic tests. Meta-analyses on diagnostic tests were identified through a systematic search of the literature using MEDLINE, EMBASE, and DARE databases and the Cochrane Library (1996-1997). Associations between study characteristics and estimates of diagnostic accuracy were evaluated with a regression model.
Main outcome measures: Relative diagnostic odds ratio (RDOR), which compared the diagnostic odds ratios of studies of a given test that lacked a particular methodological feature with those without the corresponding shortcomings in design.
Results: Fifteen (6.8%) of 218 evaluations met all 8 criteria; 64 (30%) met 6 or more. Studies evaluating tests in a diseased population and a separate control group overestimated the diagnostic performance compared with studies that used a clinical population (RDOR, 3.0; 95% confidence interval [CI], 2.0-4.5). Studies in which different reference tests were used for positive and negative results of the test under study overestimated the diagnostic performance compared with studies using a single reference test for all patients (RDOR, 2.2; 95% CI, 1.5-3.3). Diagnostic performance was also overestimated when the reference test was interpreted with knowledge of the test result (RDOR, 1.3; 95% CI, 1.0-1.9), when no criteria for the test were described (RDOR, 1.7; 95% CI, 1.1-2.5), and when no description of the population under study was provided (RDOR, 1.4; 95% CI, 1.1-1.7).
Conclusion: These data provide empirical evidence that diagnostic studies with methodological shortcomings may overestimate the accuracy of a diagnostic test, particularly those including nonrepresentative patients or applying different reference standards.
Comment in
- Measuring the quality of trials: the quality of quality scales.
Berlin JA, Rennie D. Berlin JA, et al. JAMA. 1999 Sep 15;282(11):1083-5. doi: 10.1001/jama.282.11.1083. JAMA. 1999. PMID: 10493209 No abstract available. - Using meta-analysis to answer clinical questions.
Laheij RJ. Laheij RJ. JAMA. 2001 Dec 5;286(21):2669; author reply 2669-70. JAMA. 2001. PMID: 11730436 No abstract available. - Improving reports of studies of diagnostic tests: the STARD initiative.
Rennie D. Rennie D. JAMA. 2003 Jan 1;289(1):89-90. doi: 10.1001/jama.289.1.89. JAMA. 2003. PMID: 12503983 No abstract available.
Similar articles
- Evidence of bias and variation in diagnostic accuracy studies.
Rutjes AW, Reitsma JB, Di Nisio M, Smidt N, van Rijn JC, Bossuyt PM. Rutjes AW, et al. CMAJ. 2006 Feb 14;174(4):469-76. doi: 10.1503/cmaj.050090. CMAJ. 2006. PMID: 16477057 Free PMC article. - [Methodological bias and variation of systematic reviews on diagnostic test accuracy].
Li ZX, Yang ZR, Xiang X, Gao P, Shu Z, Huang YS, Cao Y, Sun F, Zhan SY. Li ZX, et al. Zhonghua Liu Xing Bing Xue Za Zhi. 2016 Feb;37(2):286-90. doi: 10.3760/cma.j.issn.0254-6450.2016.02.027. Zhonghua Liu Xing Bing Xue Za Zhi. 2016. PMID: 26917532 Chinese. - Evidence of reference standard related bias in studies of plain radiograph reading performance: a meta-regression.
Brealey SD, Scally AJ, Hahn S, Godfrey C. Brealey SD, et al. Br J Radiol. 2007 Jun;80(954):406-13. doi: 10.1259/bjr/41006673. Epub 2006 Dec 6. Br J Radiol. 2007. PMID: 17151064 Review. - Diagnostic test accuracy of nutritional tools used to identify undernutrition in patients with colorectal cancer: a systematic review.
Håkonsen SJ, Pedersen PU, Bath-Hextall F, Kirkpatrick P. Håkonsen SJ, et al. JBI Database System Rev Implement Rep. 2015 May 15;13(4):141-87. doi: 10.11124/jbisrir-2015-1673. JBI Database System Rev Implement Rep. 2015. PMID: 26447079 Review. - Regional cerebral blood flow single photon emission computed tomography for detection of Frontotemporal dementia in people with suspected dementia.
Archer HA, Smailagic N, John C, Holmes RB, Takwoingi Y, Coulthard EJ, Cullum S. Archer HA, et al. Cochrane Database Syst Rev. 2015 Jun 23;2015(6):CD010896. doi: 10.1002/14651858.CD010896.pub2. Cochrane Database Syst Rev. 2015. PMID: 26102272 Free PMC article. Review.
Cited by
- A systematic review of the screening accuracy of the HIV Dementia Scale and International HIV Dementia Scale.
Haddow LJ, Floyd S, Copas A, Gilson RJ. Haddow LJ, et al. PLoS One. 2013 Apr 16;8(4):e61826. doi: 10.1371/journal.pone.0061826. Print 2013. PLoS One. 2013. PMID: 23613945 Free PMC article. Review. - Proteomic approaches and identification of novel therapeutic targets for alcoholism.
Gorini G, Harris RA, Mayfield RD. Gorini G, et al. Neuropsychopharmacology. 2014 Jan;39(1):104-30. doi: 10.1038/npp.2013.182. Epub 2013 Jul 31. Neuropsychopharmacology. 2014. PMID: 23900301 Free PMC article. Review. - The quality of reporting of diagnostic accuracy studies published in ophthalmic journals.
Siddiqui MA, Azuara-Blanco A, Burr J. Siddiqui MA, et al. Br J Ophthalmol. 2005 Mar;89(3):261-5. doi: 10.1136/bjo.2004.051862. Br J Ophthalmol. 2005. PMID: 15722299 Free PMC article. - Sources of bias in diagnostic accuracy studies and the diagnostic process.
Furukawa TA, Guyatt GH. Furukawa TA, et al. CMAJ. 2006 Feb 14;174(4):481-2. doi: 10.1503/cmaj.060014. CMAJ. 2006. PMID: 16477059 Free PMC article. No abstract available. - How should diagnostic tests be evaluated in glaucoma?
Medeiros FA. Medeiros FA. Br J Ophthalmol. 2007 Mar;91(3):273-4. doi: 10.1136/bjo.2006.107409. Br J Ophthalmol. 2007. PMID: 17322462 Free PMC article. No abstract available.
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Medical
Miscellaneous