The Risk of False-Positive Results in Orthopaedic Surgical... : Clinical Orthopaedics and Related Research® (original) (raw)

SECTION I SYMPOSIUM: Issues in the Design, Analysis, and Critical Appraisal of Orthopaedic Clinical Research: Part II: Statistical Issues in the Design of Orthopaedic Studies

Bhandari, Mohit MD, MSc*; Whang, William MD**; Kuo, Jonathann C. MD**; Devereaux, P. J. MD*; Sprague, Sheila BS Cand*; Tornetta , Paul III MD**

From the *Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada; and the **Department of Orthopaedic Surgery, Boston Medical Center, Boston, MA.

Reprint request to Dr. Mohit Bhandari, McMaster University Medical Center, Department of Clinical Epidemiology and Biostatistics, 1200 Main Street West, Room 2C3, Hamilton, Ontario, L8N 3Z5, Canada. Phone: 905–525-9140 ext. 22825; Fax: 905–524-3841; E-mail: [email protected].

Clinical Orthopaedics and Related Research 413():p 63-69, August 2003. | DOI: 10.1097/01.blo.0000079320.41006.c9

• Buy

Abstract

The risk of concluding that the results of a particular study are true, when, in fact, they really are attributable to chance (or random sampling error) is underappreciated by investigators. This erroneous false-positive conclusion is designated as a Type I or alpha error. The extent to which randomized trials in surgery risk Type I errors is unclear. The current authors hand-searched four orthopaedic journals, six general surgery journals, and five medical journals to identify recently published randomized trials (within the past 2 years). Information on outcomes and statistical adjustment for multiple outcomes was recorded for each study. The risk of a Type I error was calculated for each study that did not explicitly state a primary outcome measure for the main statistical comparison. One hundred fifty-nine studies met the inclusion criteria for the study: 60 studies from orthopaedic journals, 49 studies from nonorthopaedic surgical journals, and 50 studies from medical journals. Of the trials that did not state a primary outcome measure, the risk of Type I errors (false-positive results) in orthopaedic and nonorthopaedic surgery journals (mean 37.3% ± 13.3% and 37.6% ± 10.5%, respectively) were significantly greater than medical journals (10.1% ± 1.9%). In the current review of randomized trials in surgery and medicine, the following is reported: (1) reporting of primary outcomes in trials was inadequate; (2) one in three trials in surgery and one in 10 trials in medicine risked false-positive results; and (3) few trials in surgery and medicine considered adjustment for multiple comparisons.

© 2003 Lippincott Williams & Wilkins, Inc.