Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis (original) (raw)

Journal Article

,

Department of Preventive Medicine and Biometrics, University of Colorado Health Sciences Center

,

Denver, Colorado 80262

,

U.S.A.

Search for other works by this author on:

Division of Biostatistics, School of Public Health, University of Minnesota

,

Minneapolis, Minnesota 55455

,

U.S.A.

Search for other works by this author on:

Received:

01 October 1998

Revision received:

01 June 1999

• PDF
• Split View
• • Article contents
  • Figures & tables
  • Video
  • Audio
  • Supplementary Data

Navbar Search Filter Mobile Enter search term Search

Summary

We study recently developed nonparametric methods for estimating the number of missing studies that might exist in a meta-analysis and the effect that these studies might have had on its outcome. These are simple rank-based data augmentation techniques, which formalize the use of funnel plots. We show that they provide effective and relatively powerful tests for evaluating the existence of such publication bias. After adjusting for missing studies, we find that the point estimate of the overall effect size is approximately correct and coverage of the effect size confidence intervals is substantially improved, in many cases recovering the nominal confidence levels entirely. We illustrate the trim and fill method on existing meta-analyses of studies in clinical trials and psychometrics.

References

Begg

,

C. B.

(

1994

). Publication bias. In

The Handbook of Research Synthesis

,

H.

Cooper

and

L. V.

Hedges

(eds),

399

–

409

.

New York

:

Russell Sage Foundation

.

Begg

,

C. B.

and

Berlin

,

J. A.

(

1988

).

Publication bias: A problem in interpreting medical data

.

Journal of the Royal Statistical Society, Series A

151

,

419

–

463

.

Begg

,

C. B.

and

Mazumdar

,

M.

(

1994

).

Operating characteristics of a rank correlation test for publication bias

.

Biometrics

50

,

1088

–

1101

.

Biggerstaff

,

B. J.

and

Tweedie

,

R. L.

(

1997

).

Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis

.

Statistics in Medicine

16

,

753

–

768

.

British Medical Journal Editorial Staff

. (

1983

).

The editor regrets … (editorial)

.

British Medical Journal

280

,

508

.

Cooper

,

H.

(

1998

).

Synthesizing Research

.

Thousand Oaks , California

:

Sage Publications

.

Cooper

,

H.

and

Hedges

,

L. V.

(eds). (

1994

).

The Handbook of Research Synthesis

.

New York

:

Russell Sage Foundation

.

Dear

,

K.

and

Begg

,

C.

(

1992

).

An approach for assessing publication bias prior t o performing a meta-analysis

.

Statistical Science

7

,

237

–

245

.

Dear

,

K.

and

Dobson

,

A.

(

1997

). Comment on Givens, G. H., Smith, D. D., and Tweedie, R. L. (1997).

Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate (with discussion)

.

Statistical Science

12

,

245

–

246

.

DerSimonian

,

R.

and

Laird

,

N. M.

(

1986

).

Meta-analysis in clinical trials

.

Controlled Clinical Trials

7

,

177

–

188

.

Dickersin

,

K.

,

Min

,

Y.

, and

Meinert

,

C.

(

1992

).

Factors influencing publication of research results

.

Journal of the American Medical Association

267

,

374

–

378

.

DuMouchel

,

W.

and

Harris

,

J.

(

1997

). Comment on Givens, G. H., Smith, D. D., and Tweedie, R. L. (1997).

Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate (with discussion)

.

Statistical Science

12

,

244

–

245

.

Duval

,

S. J.

(

1999

).

Effects of publication bias in meta-analysis

.

Ph.D. dissertation, University of Colorado Health Sciences Center, Department of Preventive Medicine and Biometrics

.

Duval

,

S. J.

and

Tweedie

,

R. L.

(

2000

).

A non-parametric ‘trim and fill' method of assessing publication bias in meta-analysis

.

Journal of the American Statistical Association

95

, (in press).

Easterbrook

,

P.

,

Berlin

,

J.

,

Gopalan

,

R.

, and

Matthews

,

D.

(

1991

).

Publication bias in clinical research

.

Lancet

337

,

867

–

872

.

Egger

,

M.

,

Smith

,

G. D.

,

Schneider

,

M.

, and

Minder

,

C.

(

1997

).

Bias in meta-analysis detected by a simple, graphical test

.

British Medical Journal

315

,

629

–

634

.

Givens

,

G. H.

,

Smith

,

D. D.

, and

Tweedie

,

R. L.

(

1997

).

Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate (with discussion)

.

Statistical Science

12

,

221

–

250

.

Glass

,

G. V.

(

1976

).

Primary, secondary, and meta-analysis of research

.

Educational Researcher

5

,

3

–

8

.

Gleser

,

L. J.

and

Olkin

,

I.

(

1996

).

Models for estimating the number of unpublished studies

.

Statistics i n Medicine

15

,

2493

–

2507

.

Hedges

,

L.

(

1992

).

Modeling publication selection effects in meta-analysis

.

Statistical Science

7

,

227

–

236

.

Iyengar

,

S.

and

Greenhouse

,

J. B.

(

1988

).

Selection models and the file drawer problem (with discussion)

.

Statistical Science

3

,

109

–

135

.

Light

,

R.

and

Pillemer

,

D.

(

1984

).

Summing Up: The Science of Reviewing Research

.

Cambridge

:

Harvard University Press

.

McIntosh

,

H. M.

and

Olliaro

,

P.

(

1998

). Artemisinin derivatives in the treatment of severe malaria (Cochrane Review). In

The Cochrane Library, Issue 3

.

Oxford

:

Update Software

.

Mengersen

,

K.

,

Tweedie

,

R.

, and

Biggerstaff

,

B.

(

1995

).

The impact of method choice in meta-analysis

.

Australian Journal of Statistics

37

,

19

–

44

.

Misakian

,

A. L.

and

Bero

,

L. A.

(

1998

).

Publication bias and research on passive smoking: Comparison of published and unpublished studies

.

Journal of the American Medical Association

280

,

250

–

253

.

NRC Committee on Applied and Theoretical Statistics

. (

1992

).

Combining Information: Statistical Issues and Opportunities for Research

.

Washington , D.C.

:

National Academy Press

.

Raudenbush

,

S. W.

(

1984

).

Magnitude of teacher expectancy effects of pupil IQ as a function of the credibility of expectancy induction: A synthesis of findings from 18 experiments

.

Journal of Educational Psychology

76

,

85

–

97

.

Rosenthal

,

R.

(

1979

).

The “file-drawer problem” and tolerance for null results

.

Psychological Bulletin

86

,

85

–

97

.

Thompson

,

S.

and

Pocock

,

S.

(

1991

).

Can meta-analyses be trusted?

Lancet

338

,

1127

–

1130

.

This content is only available as a PDF.

© 2000 The International Biometric Society.

Citations

Views

Altmetric

Metrics

Total Views 1,302

0 Pageviews

1,302 PDF Downloads

Since 1/1/2024

×

Email alerts

Email alerts

Citing articles via

• Latest

• Most Read

• Most Cited

More from Oxford Academic