Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis - PubMed (original) (raw)
Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis
Matthew J Page et al. J Clin Epidemiol. 2018 Mar.
Abstract
Objectives: The objective of the study was to investigate the application and interpretation of statistical analyses in a cross-section of systematic reviews (SRs) of therapeutic interventions, without restriction by journal, clinical condition, or specialty.
Study design and setting: We evaluated a random sample of SRs assembled previously, which were indexed in MEDLINE® during February 2014, focused on a treatment or prevention question, and reported at least one meta-analysis. The reported statistical methods used in each SR were extracted from articles and online appendices by one author, with a 20% random sample extracted in duplicate.
Results: We evaluated 110 SRs; 78/110 (71%) were non-Cochrane SRs and 55/110 (50%) investigated a pharmacological intervention. The SRs presented a median of 13 (interquartile range: 5-27) meta-analytic effects. When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs; however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR.
Conclusions: There is scope for improvement in the application and interpretation of statistical analyses in SRs of therapeutic interventions. The involvement of statisticians on the SR team and establishment of partnerships between researchers with specialist expertise in SR methods and journal editors may help overcome these shortcomings.
Keywords: Meta-analysis; Meta-research; Methodology; Quality; Reporting; Systematic reviews.
Copyright © 2017 Elsevier Inc. All rights reserved.
Similar articles
- Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas.
Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article. - Reproducible research practices are underused in systematic reviews of biomedical interventions.
Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, Wolfe D, Yazdi F, Catalá-López F, Tricco AC, Moher D. Page MJ, et al. J Clin Epidemiol. 2018 Feb;94:8-18. doi: 10.1016/j.jclinepi.2017.10.017. Epub 2017 Nov 4. J Clin Epidemiol. 2018. PMID: 29113936 Review. - Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study.
Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D. Page MJ, et al. PLoS Med. 2016 May 24;13(5):e1002028. doi: 10.1371/journal.pmed.1002028. eCollection 2016 May. PLoS Med. 2016. PMID: 27218655 Free PMC article. - The future of Cochrane Neonatal.
Soll RF, Ovelman C, McGuire W. Soll RF, et al. Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
Cited by
- Comparison of statistical methods used to meta-analyse results from interrupted time series studies: an empirical study.
Korevaar E, Turner SL, Forbes AB, Karahalios A, Taljaard M, McKenzie JE. Korevaar E, et al. BMC Med Res Methodol. 2024 Feb 10;24(1):31. doi: 10.1186/s12874-024-02147-z. BMC Med Res Methodol. 2024. PMID: 38341540 Free PMC article. - Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: A simulation study.
Korevaar E, Turner SL, Forbes AB, Karahalios A, Taljaard M, McKenzie JE. Korevaar E, et al. Res Synth Methods. 2023 Nov;14(6):882-902. doi: 10.1002/jrsm.1669. Epub 2023 Sep 20. Res Synth Methods. 2023. PMID: 37731166 Free PMC article. - Neutrophil-Enriched Biomarkers and Long-Term Prognosis in Acute Coronary Syndrome: a Systematic Review and Meta-analysis.
Yiu JYT, Hally KE, Larsen PD, Holley AS. Yiu JYT, et al. J Cardiovasc Transl Res. 2024 Apr;17(2):426-447. doi: 10.1007/s12265-023-10425-2. Epub 2023 Aug 18. J Cardiovasc Transl Res. 2024. PMID: 37594719 Free PMC article. - [The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsDeclaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas].
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. Page MJ, et al. Rev Panam Salud Publica. 2022 Dec 30;46:e112. doi: 10.26633/RPSP.2022.112. eCollection 2022. Rev Panam Salud Publica. 2022. PMID: 36601438 Free PMC article. Portuguese. - Associations of working conditions and chronic low-grade inflammation among employees: a systematic review and meta-analysis.
Kaltenegger HC, Becker L, Rohleder N, Nowak D, Weigl M. Kaltenegger HC, et al. Scand J Work Environ Health. 2021 Nov 1;47(8):565-581. doi: 10.5271/sjweh.3982. Epub 2021 Sep 15. Scand J Work Environ Health. 2021. PMID: 34523689 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Research Materials