From pre-registration to publication: a nontechnical primer for conducting a meta-analysis to synthesize correlational data (original) (raw)
Related papers
Examining the Reproducibility of Meta-Analyses in Psychology: A Preliminary Report
Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Research in medicine has revealed meta-analyses often cannot be reproduced. In this project, we examined the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. Reproducing published meta-analyses was surprisingly difficult. 96% of meta-analyses published in 2013-2014 did not adhere to reporting guidelines. A third of these meta-analyses did not contain a table specifying all individual effect sizes. Five of the 20 randomly selected meta-analyses we attempted to reproduce could not be reproduced at all due to lack of access to raw data, no details about the effect sizes extrac...
How to Perform a Meta-Analysis: A Practical Step-By-Step Guide Using R Software and Rstudio
Acta Ortopédica Brasileira
Meta-analysis is an adequate statistical technique to combine results from different studies, and its use has been growing in the medical field. Thus, not only knowing how to interpret meta-analysis, but also knowing how to perform one, is fundamental today. Therefore, the objective of this article is to present the basic concepts and serve as a guide for conducting a meta-analysis using R and RStudio software. For this, the reader has access to the basic commands in the R and RStudio software, necessary for conducting a meta-analysis. The advantage of R is that it is a free software. For a better understanding of the commands, two examples were presented in a practical way, in addition to revising some basic concepts of this statistical technique. It is assumed that the data necessary for the meta-analysis has already been collected, that is, the description of methodologies for systematic review is not a discussed subject. Finally, it is worth remembering that there are many other...
A Practical Tutorial on Conducting Meta-Analysis in R
Meta-analysis is a set of statistical procedures used for providing transparent, objective, and replicable summaries of research findings. This tutorial demonstrates the most common procedures on conducting a meta-analysis using the R statistical software program. It begins with an introduction to meta-analysis along with detailing the preliminary steps involved in completing a research synthesis. Then, a step-by-step tutorial for each of the quantitative components involved for meta-analysis is provided using a fictional set of psychotherapy treatment-control studies as a running example.
Meta-Analysis in a Nutshell: Techniques and General Findings
Economics: The Open-Access, Open-Assessment E-Journal, 2015
The purpose of this article is to introduce the technique and main findings of meta-analysis to the reader, who is unfamiliar with the field and has the usual objections. A meta-analysis is a quantitative survey of a literature reporting estimates of the same parameter. The funnel showing the distribution of the estimates is normally amazingly wide given their t-ratios. Little of the variation can be explained by the quality of the journal (as measured by its impact factor) or by the estimator used. The funnel has often asymmetries consistent with the most likely priors of the researchers, giving a publication bias.
A Guide to Conducting a Meta-Analysis
Meta-analysis is widely accepted as the preferred method to synthesize research findings in various disciplines. This paper provides an introduction to when and how to conduct a meta-analysis. Several practical questions, such as advantages of meta-analysis over conventional narrative review and the number of studies required for a meta-analysis, are addressed. Common meta-analytic models are then introduced. An artificial dataset is used to illustrate how a meta-analysis is conducted in several software packages. The paper concludes with some common pitfalls of meta-analysis and their solutions. The primary goal of this paper is to provide a summary background to readers who would like to conduct heir first meta-analytic study.
metaSEM: An R Package for Meta-Analysis using Structural Equation Modeling
The metaSEM package provides functions to conducting univariate and multivariate meta-analysis using a structural equation modeling (SEM) approach via the OpenMx package. It also implemented the two-stage SEM approach to conducting fixed-and random-effects meta-analytic SEM on correlation or covariance matrices. This paper outlines the basic theories and its implementations. Several examples are used to illustrate the procedures.
A Statistical Method for Synthesizing Meta-Analyses
Computational and Mathematical Methods in Medicine, 2013
Multiple meta-analyses may use similar search criteria and focus on the same topic of interest, but they may yield different or sometimes discordant results. The lack of statistical methods for synthesizing these findings makes it challenging to properly interpret the results from multiple meta-analyses, especially when their results are conflicting. In this paper, we first introduce a method to synthesize the meta-analytic results when multiple meta-analyses use the same type of summary effect estimates. When meta-analyses use different types of effect sizes, the meta-analysis results cannot be directly combined. We propose a two-step frequentist procedure to first convert the effect size estimates to the same metric and then summarize them with a weighted mean estimate. Our proposed method offers several advantages over existing methods by Hemming et al. (2012). First, different types of summary effect sizes are considered. Second, our method provides the same overall effect size ...
Psychological Methods, 2020
Psychology researchers are rapidly adopting open science practices, yet clear guidelines on how to apply these practices to meta-analysis remain lacking. In this tutorial, we describe why open science is important in the context of meta-analysis in psychology, and suggest how to adopt the 3 main components of open science: preregistration, open materials, and open data. We first describe how to make the preregistration as thorough as possible-and how to handle deviations from the plan. We then focus on creating easy-to-read materials (e.g., search syntax, R scripts) to facilitate reproducibility and bolster the impact of a meta-analysis. Finally, we suggest how to organize data (e.g., literature search results, data extracted from studies) that are easy to share, interpret, and update as new studies emerge. For each step of the meta-analysis, we provide example templates, accompanied by brief video tutorials, and show how to integrate these practices into the Open Science Framework (https://osf.io/q8stz/). Translational Abstract Open science practices are gaining traction in the psychological research ecosystem. A number of contributions have been published recently with the goal to facilitate open science practices in a number of research designs; however, clear guidelines about how to integrate these practices in the context of meta-analyses have been lacking. Here, we describe why and how open science practices can be applied in the context of meta-analysis. After highlighting the benefits of the three main components of open science (preregistration, open materials, and open data), we propose a series of nine templates and video tutorials to help researchers with all aspects of an open meta-analysis. These materials are designed to facilitate the evaluation of empirical findings and help increase the impact of a meta-analysis-both important steps toward robust and valid inferences in psychological science. Meta-analyses play a central role in synthesizing research findings. With a high value-to-cost ratio, especially in fields where studies are typically onerous or expensive (e.g., clinical trials, interventions, fMRI studies), meta-analyses can maximize information and reduce the influence of biased studies, often providing the basis for decisions or policies. Yet, despite the recent rise of open science across fields of investigation (Federer et al., 2018; Nosek & Lindsay, 2018; Popkin, 2019), clear and well-defined guidelines about how to integrate meta-analyses within the open science ecosystem remain lacking in psychology. This gap in the literature has important consequences, as it suggests that some-if not all-of these practices are not crucial in the context of meta-analyses, or that the specificities of meta-analyses do not warrant customized suggestions. In this tutorial, we argue that open science practices are critical to ensure robust and reliable meta-analyses, and propose specific practical recommendations to facilitate preregistration, open materials and open data. For each component, we provide templates that can help standardization, while allowing the necessary flexibility for researchers to adapt them to a range of designs. With the many small decisions that go into a meta-analysis, careful planning and clear documentation of these decisions is critical for transparency. The templates provided help to facilitate this. For instance, registering the analysis script ensures consideration of statistical decisions in advance of the literature search; registering the data extraction file ensures early decision-making about which variables to extract from included studies. All templates-together with tips on how to best utilize them-are also illustrated with
A Review of Meta-Analysis Packages in R
Journal of Educational and Behavioral Statistics
Meta-analysis is a statistical technique that allows an analyst to synthesize effect sizes from multiple primary studies. To estimate meta-analysis models, the open-source statistical environment R is quickly becoming a popular choice. The meta-analytic community has contributed to this growth by developing numerous packages specific to meta-analysis. The purpose of this study is to locate all publicly available meta-analytic R packages. We located 63 packages via a comprehensive online search. To help elucidate these functionalities to the field, we describe each of the packages, recommend applications for researchers interested in using R for meta-analyses, provide a brief tutorial of two meta-analysis packages, and make suggestions for future meta-analytic R package creators.