Reproducible research practices are underused in systematic reviews of biomedical interventions - PubMed (original) (raw)
Review
Reproducible research practices are underused in systematic reviews of biomedical interventions
Matthew J Page et al. J Clin Epidemiol. 2018 Feb.
Abstract
Objectives: To evaluate how often reproducible research practices, which allow others to recreate the findings of studies, given the original data, are used in systematic reviews (SRs) of biomedical research.
Study design and setting: We evaluated a random sample of SRs indexed in MEDLINE during February 2014, which focused on a therapeutic intervention and reported at least one meta-analysis. Data on reproducible research practices in each SR were extracted using a 26-item form by one author, with a 20% random sample extracted in duplicate. We explored whether the use of reproducible research practices was associated with an SR being a Cochrane review, as well as with the reported use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.
Results: We evaluated 110 SRs of therapeutic interventions, 78 (71%) of which were non-Cochrane SRs. Across the SRs, there were 2,139 meta-analytic effects (including subgroup meta-analytic effects and sensitivity analyses), 1,551 (73%) of which were reported in sufficient detail to recreate them. Systematic reviewers reported the data needed to recreate all meta-analytic effects in 72 (65%) SRs only. This percentage was higher in Cochrane than in non-Cochrane SRs (30/32 [94%] vs. 42/78 [54%]; risk ratio 1.74, 95% confidence interval 1.39-2.18). Systematic reviewers who reported imputing, algebraically manipulating, or obtaining some data from the study author/sponsor infrequently stated which specific data were handled in this way. Only 33 (30%) SRs mentioned access to data sets and statistical code used to perform analyses.
Conclusion: Reproducible research practices are underused in SRs of biomedical interventions. Adoption of such practices facilitates identification of errors and allows the SR data to be reanalyzed.
Keywords: Data sharing; Methodology; Quality; Reporting; Reproducibility; Systematic reviews.
Copyright © 2017 Elsevier Inc. All rights reserved.
Similar articles
- Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis.
Page MJ, Altman DG, McKenzie JE, Shamseer L, Ahmadzai N, Wolfe D, Yazdi F, Catalá-López F, Tricco AC, Moher D. Page MJ, et al. J Clin Epidemiol. 2018 Mar;95:7-18. doi: 10.1016/j.jclinepi.2017.11.022. Epub 2017 Dec 2. J Clin Epidemiol. 2018. PMID: 29203419 - Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas.
Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article. - Evaluation of Reproducible Research Practices in Oncology Systematic Reviews With Meta-analyses Referenced by National Comprehensive Cancer Network Guidelines.
Wayant C, Page MJ, Vassar M. Wayant C, et al. JAMA Oncol. 2019 Nov 1;5(11):1550-1555. doi: 10.1001/jamaoncol.2019.2564. JAMA Oncol. 2019. PMID: 31486837 Free PMC article. - Completeness of reporting in systematic reviews and meta-analyses in vascular surgery.
Javidan A, Alaichi J, Nassar Y, Li A, Balta KY, Naji F. Javidan A, et al. J Vasc Surg. 2023 Dec;78(6):1550-1558.e2. doi: 10.1016/j.jvs.2023.04.009. Epub 2023 Apr 15. J Vasc Surg. 2023. PMID: 37068527 Review.
Cited by
- Impact of harm minimization interventions on reducing blood-borne infection transmission and some injecting behaviors among people who inject drugs: an overview and evidence gap mapping.
Tonin FS, Alves da Costa F, Fernandez-Llimos F. Tonin FS, et al. Addict Sci Clin Pract. 2024 Feb 4;19(1):9. doi: 10.1186/s13722-024-00439-9. Addict Sci Clin Pract. 2024. PMID: 38310293 Free PMC article. - Diagnostic Approach and Differences between Spinal Infections and Tumors.
Compagnone D, Cecchinato R, Pezzi A, Langella F, Damilano M, Redaelli A, Vanni D, Lamartina C, Berjano P, Boriani S. Compagnone D, et al. Diagnostics (Basel). 2023 Aug 23;13(17):2737. doi: 10.3390/diagnostics13172737. Diagnostics (Basel). 2023. PMID: 37685273 Free PMC article. Review. - Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data.
Hamilton DG, Hong K, Fraser H, Rowhani-Farid A, Fidler F, Page MJ. Hamilton DG, et al. BMJ. 2023 Jul 11;382:e075767. doi: 10.1136/bmj-2023-075767. BMJ. 2023. PMID: 37433624 Free PMC article. - Guidance to best tools and practices for systematic reviews1.
Kolaski K, Romeiser Logan L, Ioannidis JPA. Kolaski K, et al. J Pediatr Rehabil Med. 2023;16(2):241-273. doi: 10.3233/PRM-230019. J Pediatr Rehabil Med. 2023. PMID: 37302044 Free PMC article. - Guidance to best tools and practices for systematic reviews.
Kolaski K, Logan LR, Ioannidis JPA. Kolaski K, et al. Syst Rev. 2023 Jun 8;12(1):96. doi: 10.1186/s13643-023-02255-9. Syst Rev. 2023. PMID: 37291658 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Research Materials