Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature Analysis (original) (raw)

Open Access

Peer-reviewed

Research Article

Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature Analysis

PLOS

x

Figures

Abstract

Background

The United States (US) Food and Drug Administration (FDA) approves new drugs based on sponsor-submitted clinical trials. The publication status of these trials in the medical literature and factors associated with publication have not been evaluated. We sought to determine the proportion of trials submitted to the FDA in support of newly approved drugs that are published in biomedical journals that a typical clinician, consumer, or policy maker living in the US would reasonably search.

Methods and Findings

We conducted a cohort study of trials supporting new drugs approved between 1998 and 2000, as described in FDA medical and statistical review documents and the FDA approved drug label. We determined publication status and time from approval to full publication in the medical literature at 2 and 5 y by searching PubMed and other databases through 01 August 2006. We then evaluated trial characteristics associated with publication. We identified 909 trials supporting 90 approved drugs in the FDA reviews, of which 43% (394/909) were published. Among the subset of trials described in the FDA-approved drug label and classified as “pivotal trials” for our analysis, 76% (257/340) were published. In multivariable logistic regression for all trials 5 y postapproval, likelihood of publication correlated with statistically significant results (odds ratio [OR] 3.03, 95% confidence interval [CI] 1.78–5.17); larger sample sizes (OR 1.33 per 2-fold increase in sample size, 95% CI 1.17–1.52); and pivotal status (OR 5.31, 95% CI 3.30–8.55). In multivariable logistic regression for only the pivotal trials 5 y postapproval, likelihood of publication correlated with statistically significant results (OR 2.96, 95% CI 1.24–7.06) and larger sample sizes (OR 1.47 per 2-fold increase in sample size, 95% CI 1.15–1.88). Statistically significant results and larger sample sizes were also predictive of publication at 2 y postapproval and in multivariable Cox proportional models for all trials and the subset of pivotal trials.

Conclusions

Over half of all supporting trials for FDA-approved drugs remained unpublished ≥ 5 y after approval. Pivotal trials and trials with statistically significant results and larger sample sizes are more likely to be published. Selective reporting of trial results exists for commonly marketed drugs. Our data provide a baseline for evaluating publication bias as the new FDA Amendments Act comes into force mandating basic results reporting of clinical trials.

Citation: Lee K, Bacchetti P, Sim I (2008) Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature Analysis. PLoS Med 5(9): e191. https://doi.org/10.1371/journal.pmed.0050191

Academic Editor: Mike Clarke, UK Cochrane Centre, United Kingdom

Received: February 19, 2008; Accepted: August 8, 2008; Published: September 23, 2008

Copyright: © 2008 Lee et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: The project described was supported in part by funds from the United States Presidential Early Career Award for Scientists and Engineers awarded to IS and administered under grant LM-06780, and by National Institutes of Health/National Center for Research Resources (NIH/NCRR) University of California San Francisco–Clinical and Translational Science Institute (UCSF-CTSI). Grant number UL1 RR024131 and grant number KL2RR024130 to KL. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. The granting agencies had no role in the design, conduct, review, or publication approval of this study.

Competing interests: The authors have declared that no competing interests exist.

**Abbreviations:**CI, confidence interval; CINAHL, Cumulative Index for Nursing and Allied Health Literature; FDA, Food and Drug Administration; NDA, new drug application; OR, odds ratio; US, United States

Editors' Summary

Background.

Before a new drug becomes available for the treatment of a specific human disease, its benefits and harms are carefully studied, first in the laboratory and in animals, and then in several types of clinical trials. In the most important of these trials—so-called “pivotal” clinical trials—the efficacy and safety of the new drug and of a standard treatment are compared by giving groups of patients the different treatments and measuring several predefined “outcomes.” These outcomes indicate whether the new drug is more effective than the standard treatment and whether it has any other effects on the patients' health and daily life. All this information is then submitted by the sponsor of the new drug (usually a pharmaceutical company) to the government body responsible for drug approval—in the US, this is the Food and Drug Administration (FDA).

Why Was This Study Done?

After a drug receives FDA approval, information about the clinical trials supporting the FDA's decision are included in the FDA “Summary Basis of Approval” and/or on the drug label. In addition, some clinical trials are described in medical journals. Ideally, all the clinical information that leads to a drug's approval should be publicly available to help clinicians make informed decisions about how to treat their patients. A full-length publication in a medical journal is the primary way that clinical trial results are communicated to the scientific community and the public. Unfortunately, drug sponsors sometimes publish the results only of trials where their drug performed well; as a consequence, trials where the drug did no better than the standard treatment or where it had unwanted side effects remain unpublished. Publication bias like this provides an inaccurate picture of a drug's efficacy and safety relative to other therapies and may lead to excessive prescribing of newer, more expensive (but not necessarily more effective) treatments. In this study, the researchers investigate whether selective trial reporting is common by evaluating the publication status of trials submitted to the FDA for a wide variety of approved drugs. They also ask which factors affect a trial's chances of publication.

What Did the Researchers Do and Find?

The researchers identified 90 drugs approved by the FDA between 1998 and 2000 by searching the FDA's Center for Drug Evaluation and Research Web site. From the Summary Basis of Approval for each drug, they identified 909 clinical trials undertaken to support these approvals. They then searched the published medical literature up to mid-2006 to determine if and when the results of each trial were published. Although 76% of the pivotal trials had appeared in medical journals, usually within 3 years of FDA approval, only 43% of all of the submitted trials had been published. Among all the trials, those with statistically significant results were nearly twice as likely to have been published as those without statistically significant results, and pivotal trials were three times more likely to have been published as nonpivotal trials, 5 years postapproval. In addition, a larger sample size increased the likelihood of publication. Having statistically significant results and larger sample sizes also increased the likelihood of publication of the pivotal trials.

What Do These Findings Mean?

Although the search methods used in this study may have missed some publications, these findings suggest that more than half the clinical trials undertaken to support drug approval remain unpublished 5 years or more after FDA approval. They also reveal selective reporting of results. For example, they show that a pivotal trial in which the new drug does no better than an old drug is less likely to be published than one where the new drug is more effective, a publication bias that could establish an inappropriately favorable record for the new drug in the medical literature. Importantly, these findings provide a baseline for monitoring the effects of the FDA Amendments Act 2007, which was introduced to improve the accuracy and completeness of drug trial reporting. Under this Act, all trials supporting FDA-approved drugs must be registered when they start, and the summary results of all the outcomes declared at trial registration as well as specific details about the trial protocol must be publicly posted within a year of drug approval on the US National Institutes of Health clinical trials site.

Introduction

In the United States, the Food and Drug Administration (FDA) approves new drug products for sale and marketing based on results from clinical investigations that demonstrate the safety and efficacy of a drug for a proposed indication. Sponsors of a drug (e.g., companies, research institutions, or government) seek approval by submitting a new drug application (NDA) [1] to the FDA, which must include documentation and analyses of all animal and human trial data, as well as information about the ingredients, clinical pharmacology, manufacturing, processing, and packaging of the drug. The FDA relies on sponsors to submit all data, including complete protocols, protocol revisions, and data from failed trials in the NDA. The NDA is then reviewed by clinicians, statisticians, chemists, clinical pharmacologists, and other relevant scientific and regulatory disciplines within the FDA to confirm and validate the sponsor's conclusion that a drug is safe and effective.

For drugs that receive FDA approval, public disclosure of trial results may occur through a variety of sources. The FDA discloses a Summary Basis of Approval document that contains summaries and evaluations of clinical data and statistical analyses performed by FDA medical officers during the approval process [2]. However, these summaries contain only selected results from the clinical trials [1], and data deemed confidential or information considered commercial under Exemption 4 of the Freedom of Information Act may be redacted [3]. The drug label or package insert also provides a summary of clinical studies but often in less detail than the Summary Basis of Approval. Publication in the peer-reviewed medical literature is the main channel by which trial results are publicly disclosed and communicated to clinicians. The complete and accurate reporting of clinical trial results is crucial to ensuring an unbiased evidence base for advancing science and facilitating informed clinical decision-making [4], and has been considered an ethical obligation [5]. However, there was no requirement until very recently that trial results be published or otherwise made public for FDA-approved and marketed drugs.

A string of recent controversies concerning the suppression of safety risks of rosiglitazone [6], paroxetine [7], and rofecoxib [8,9] has drawn public attention to the limited and incomplete public access to clinical trial results on FDA-approved drugs [10] and has resulted in a concerted effort to achieve improved compliance with trial registration and greater disclosure of trial results [11–13].

In response to these concerns, the US recently mandated in the FDA Amendments Act 2007 (Public Law 110–85) that all trials supporting FDA-approved drugs and devices must be registered at inception and have their “basic results” publicly posted by the National Institutes of Health. The basic results to be disclosed include the demographics of the study participants, the number of participants who dropped out or were excluded from analysis, and the numeric and statistical test results of all primary and secondary outcomes declared at initial trial registration.

For the foreseeable future, however, the detailed information needed for full appraisal of a trial's evidence is likely to be available only in journal publications. This information includes protocol, protocol deviation, and conflicts of interest information, as well as additional analyses beyond the primary and secondary outcomes. The availability of basic results on ClinicalTrials.gov (http://www.clinicaltrials.gov/) will therefore complement, but not supplant, the medical literature's continuing role as the dominant channel of communication to clinicians and the public, even after the imposition of mandatory basic results reporting.

Previous research has documented the problem of publication bias and incomplete or selective reporting of trials submitted to licensing authorities in Sweden [14,15], Finland [14], and the US [10,16]. For example, among trials of antidepressants submitted to the FDA [16] or the Swedish drug regulatory authority [15], efficacy trials reporting positive results and larger effect sizes were more likely to be published. These analyses were limited to one drug class, specifically, antidepressants. Therefore, we evaluated the publication status of trials submitted to the FDA for a wide variety of approved drugs and identified factors associated with publication.

Methods

Identification of Clinical Trials

We identified all drugs approved by the FDA between January 1998 and December 2000 at the Center for Drug Evaluation and Research Web site, available at http://www.fda.gov/cder/da/da.htm. We included only new drugs classified as “new molecular entities,” which are drug products that have never been previously approved by the FDA for any indication, hereafter referred to as “new drug.” For each new drug, we retrieved the FDA Summary Basis for Approval and evaluated the medical and statistical review documents to identify clinical trials submitted by the sponsor. These review documents are available at http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm for all new drugs approved since 1998.

Classification of Clinical Trials

Phase I trials are often small studies designed to provide supporting information about a drug's pharmacokinetic parameters, dosing schedule, common side effects, tolerability, and toxicity, but are limited by design or other factors in their ability to demonstrate efficacy. Phase II and III trials are often larger studies designed to provide evidence on the overall risks and benefits of a drug. The phase of a trial was often not reported in the FDA documents. Sponsors and the FDA frequently categorize certain trials as “pivotal.” These are trials that demonstrate the efficacy and safety of a drug for its proposed indication and provide the most useful information for clinical decision-making. Pivotal trials are typically Phase II or III trials, but there is no formal definition of a pivotal trial. In practice, trials that are reported in the “clinical studies” or “clinical efficacy” section of the FDA-approved drug label are considered pivotal. We used this scheme to categorize trials as “pivotal” or “nonpivotal.” We obtained the product label at the time of FDA approval for each new drug, or the next available product label if the initial product label was not available, at http://www.fda.gov/cder/approval/index.htm. Trials described in the summary documents for each drug approval that were also described in the “clinical studies” section of the corresponding drug label were categorized as pivotal. All other trials were categorized as nonpivotal.

Data Extraction

For each submitted trial, we recorded the following characteristics when available in the FDA documents: drug name (generic and trade), the number and location of study sites, the name of the principal investigator, the number of study participants, dosage and evaluation schedules, sample size, statistical significance of the primary outcome (p < 0.05 or confidence interval [CI] excluding no difference; or if the study was an equivalency study, _p_ > 0.05 or CI including no difference or a CI excluding the prespecified difference described in the trial). Nonsignificant or null results are defined as p > 0.05 or CI including no difference; or if the study was an equivalency study,p < 0.05 or CI excluding no difference or a CI including the prespecified differences described in the trial. We also recorded whether the trial was randomized or double blinded as reported by the sponsor in the FDA documents.

Search Strategy and Publication Matching

We systematically searched common databases of biomedical journals that a typical clinician, consumer, or policy maker living in the US would reasonably search. These databases were PubMed, Cochrane Library, and the Cumulative Index for Nursing and Allied Health Literature (CINAHL). First, we electronically searched PubMed to match each trial identified in the FDA review documents to publications in the medical literature. We initially used the new drug's generic name and limited the search to Publication Type: Clinical Trial. All English-language retrievals were reviewed in abstract or full-text form. Trials identified in the FDA reviews were matched to a publication based on the following characteristics: drug name, sample size, dosing schedules, number and location of study centers, primary outcome measures, and statistical significance or estimated effect of the primary outcome results.

Only original research reports in full journal articles were counted as matching publications; abstracts or review articles were not considered matches, as these types of articles by definition contain incomplete descriptions of a trial's methods and results. For remaining trials that were not matched to a publication, we searched PubMed again without publication type limits using a variety of keywords (e.g., generic drug name; names of other drugs in the trial; disease/condition studied; outcomes measured; and trial characteristics such as “cross over”, “randomize”, “blind”, “washout”, “placebo”, “pharmacokinetics”, and “bioavailability”). If trials remained unmatched to a publication in PubMed after this more comprehensive search, we searched the entire Cochrane Library and CINAHL databases without limits using the similar keyword strategy described above. We also reviewed The Medical Letter (http://www.medletter.com/) for additional trial publications. For each trial, we verified statistical significance of primary results, randomization, double blinding, and sample size by reviewing the publication. We completed our literature search on 01 August 2006, which yields a follow-up period ranging between 5.5 to 8.5 y from the time of a new drug's approval (January 1998 to December 2000).

Statistical Analysis

The main outcome measures were time from FDA approval to publication of a full report, and whether a report was published by 2 or 5 y after approval. We analyzed publication at 2 y because pending Congressional legislation is considering mandating results reporting by 2 y after drug approval. Trials that were not published were censored as of 01 August 2006.

To control for multiple variables simultaneously, we carried out multivariate mixed effects logistic regression analysis and calculated odds ratios at 2 and 5 y after approval. All models were adjusted for clustering by drug (treated as a random effect). Predictors assessed in both univariate and multivariable analyses included statistical significance of the primary results, double blinding, randomization, sample size (dichotomized at the median size of ≤ 135 or > 135, or log-transformed to each 2-fold increase in sample size), study type (pivotal or not), and company size. Companies with annual revenues greater than 3billion,and/orannualresearchanddevelopmentexpendituresgreaterthan3 billion, and/or annual research and development expenditures greater than 3billion,and/orannualresearchanddevelopmentexpendituresgreaterthan500 million in 2004, were classified as large companies, and generally represented the top 30 pharmaceutical and biotechnology companies in the world [17]. Month zero was defined as the month of FDA approval as stated in the FDA documents. The publication month was the month of the journal issue in which the trial appeared. Trials published before their FDA approval date were analyzed as published at time zero. In cases of duplicate publication (those reporting the same findings and results from the same trial, study population, intervention, and measured outcomes), we included only the earliest publication in all analyses. We chose variables for inclusion in multivariable models using forward stepwise selection with p < 0.05 required for entry and retention.

Our primary analysis was logistic regression analyses on all supporting trials (n = 909 trials). Our secondary analysis was on the subset of trials classified as pivotal (n = 340 trials). Data were analyzed with SAS software (version 9.1, SAS Institute).

Results

All Supporting Trials

We identified 90 FDA-approved new drugs between January 1998 and December 2000. Eighty-nine (99%) of the applications were submitted by a pharmaceutical company; one application was submitted by the US Army Medical Research and Material Command. Eighty-eight drugs were available by prescription only and two had over-the-counter marketing status. Seven prescription drug products were discontinued after initial FDA approval. We were able to identify a total of 909 trials with sufficient description in the FDA review documents supporting these 90 new drugs. Table 1 describes the trials' characteristics. We matched 394 of these trials (43%) to publications in the medical literature (Figure 1): 393 to publications in PubMed, the Cochrane Library, or the CINAHL database, and one to a publication cited by The Medical Letter but not indexed by the searched databases. The remaining 515 trials (57%) could not be matched to any publication. The proportion of trials published per new drug ranged from 0% to 100%, with an average of 55% of supporting trials published per new drug (Table S1). One of the 90 new drugs, an antibiotic, had none of its supporting trials published. Duplicate publications were seen in six trials: five trials had results published twice and one trial had results published three times, to total 401 matched publications.

thumbnail

Figure 1. Flowchart of Publications by Type of Trial Supporting Applications for New Drug Approvals in 1998–2000

1Clinical trials that are adequately designed to demonstrate efficacy of the drug for a proposed indication and reported in the “clinical studies” or “clinical efficacy” section of the FDA approved drug label.

2Time to publication in years counting from the month of FDA approval.

https://doi.org/10.1371/journal.pmed.0050191.g001

In univariate analyses of all supporting trials, trials with statistically significant results, larger sample sizes, double blinding, randomization, and trials that were pivotal were more likely to be published by 2 and 5 y after FDA approval (Table 2). Company size did not appear to be associated with publication. When controlling for all of these factors simultaneously in multivariable analyses, statistically significant results, larger sample sizes, and pivotal status continued to be strong predictors of publication at 2 and 5 y after FDA approval (Table 3). Adding an interaction of statistically significant results and sample size estimated the effect of sample size to be smaller for studies with statistically significant results by a factor of 0.81 (p = 0.19) at 2 y and 0.79 (p = 0.14) at 5 y. Because statistical significance was missing for many studies, we also fit models like those in Tables 2 and 3, but with “unknown” statistical significance counted as a third possible category. This permitted inclusion of 883 trials, but produced no qualitative changes in the results. Trials with unknown statistical significance were estimated to be less likely to be published than trials with nonsignificant results at 2 y (OR 0.71,p = 0.28) and 5 y (OR 0.59, p = 0.067) with a nearly unchanged estimate of the effect of statistical significance (OR 2.53, p = 0.001 at 2 y, OR 3.06, p < 0.001 at 5 y). Results from Cox proportional hazards modeling with a shared gamma frailty were qualitatively similar to the random effects logistic regression results and so are not shown.

Figure 2 shows the yearly number and cumulative proportion of trials published relative to the time of FDA approval. Thirty-two percent (128/394) of the publications occurred prior to the relevant new drug's FDA approval and 92% (364/394) were published within 3 y of FDA approval. Among published trials reporting the statistical significance of their primary outcome (n = 337), the median time to publication from FDA approval for trials with statistically significant results was 0.77 y (range 0–4.41 y,n = 285) and 0.73 y for trials without statistically significant results (range 0–3.84 y, n = 52).

thumbnail

Figure 2. Yearly Publications of Trials Supporting Approval of New Drugs: Publication of Supporting Trials (n = 394/909)

Trials from less than half of the cohort (43%) were published. Of the trials that were published, 92% were published within 3 y of FDA approval. Trials could be published prior to or following submission of data to the FDA.

https://doi.org/10.1371/journal.pmed.0050191.g002

Pivotal Trials

Of the 909 trials, 340 (37%) were identified as pivotal, of which 257 (76%) were published (Figure 1). The predictors of publication for pivotal trials were similar to those for all supporting trials in univariate (Table 2) and multivariate analyses (Table 3). Interaction of statistically significant results and sample size was similar to that for all trials, with the effect of sample size estimated to be smaller for studies with statistically significant results by a factor of 0.82 (p = 0.54) at 2 y and 0.83 (p = 0.54) at 5 y. Like the analysis of all trials, counting unknown statistical significance as a valid third category permitted inclusion of more trials (n = 339) but produced no qualitative changes in results. Figure 3 shows the yearly number and cumulative proportion of pivotal trials published relative to the time of FDA approval. Thirty-two percent (82/257) of the publications occurred prior to the relevant drug's FDA approval and 95% (245/257) were published within 3 y after FDA approval.

thumbnail

Figure 3. Yearly Publications of Trials Supporting Approval of New Drugs: Publication of Pivotal Trials (n = 257/340)

Of the pivotal trials that were published, 95% were published within 3 y of FDA approval. Trials could be published prior to or following submission of data to the FDA.

https://doi.org/10.1371/journal.pmed.0050191.g003

Discussion

Our study evaluated the publication of 909 clinical trials identified in FDA medical and statistical review documents in support of 90 new drug products approved between 1998 and 2000. We found that after a minimum of 5.5 y of follow-up after FDA approval, we identified publications from 43% of the trials in the medical literature. For pivotal trials, which are more clinically informative than nonpivotal trials, we found publications from 76% of the trials. For one of the 90 approved new drugs, we could not find any published supporting trial. We also found strong evidence of publication bias: trials with statistically significant results were more likely to be published than trials with nonsignificant results, as were trials with larger sample sizes. There was a weak suggestion that the effect of sample size might be less among trials with statistically significant findings, but _p_-values for such interactions did not reach statistical significance. Our study therefore shows that previous findings of publication bias of trials supporting the regulatory applications of selected drug classes (e.g., antidepressants) [10,14–16] are broadly true across a diverse group of drug classes. Publication bias may lead to an inappropriately favorable record in the medical literature of a drug's true risk/benefit profile relative to other standard therapies, and may thus lead to preferential prescribing of newer and more-expensive treatments. We could not test whether similar publication bias exists for trials supporting unsuccessful new drug applications because adequate information about these applications was unavailable from the FDA or other government or commercial sources.

We also found the reporting of clinical trials in the FDA review documents and drug labels to be variable in detail and content, and not an adequate substitute for full publication in the medical literature. For example, reporting ranged from detailed descriptions of a trial's study design, intervention, patient population, statistical analyses, adverse events, primary outcomes, and other results, to brief statements that only summarized a trial's primary outcome. We also noted sections of redacted information in the FDA review documents. Neither the FDA review documents nor the drug labels followed a standard format for reporting a trial's methodology and results. Use of guidelines such as the revised CONSORT (Consolidated Standards of Reporting Trials) [18] may help to improve the quality and completeness of trial reporting in FDA review documents as others have proposed [19].

Our study has several limitations. First, we may have misclassified some published trials as being unpublished because of difficulties in matching publications to incomplete trial descriptions in the FDA documents. Also, we did not search other databases such as the European EMBASE, nor did we contact investigators or sponsors to determine publication status or verify that a trial was not published or in press. Thus, we are likely to have underestimated the overall publication rate of these trials. However, we believe that for clinicians and policy makers, the most relevant publication rate is not the overall rate but the publication rate in journals that a typical clinician, consumer, or policy maker would have access to through a reasonable literature search. We believe our searches of PubMed, the Cochrane Library, and CINAHL reflect such a reasonable search. It would not be reasonable to expect a clinician, consumer, or policy maker to contact investigators or sponsors to determine a trial's publication status.

A second limitation of our study is our follow-up time of 5.5 to 8.5 y after new drug approval may be inadequate. However, we found that publications occurred almost exclusively within the first 3 y after approval, making it unlikely that longer follow-up would yield many additional publications. Third, time-to-publication is ideally counted from the date of trial completion, but we were unable to obtain these dates reliably. Moreover, we believe the month of approval is the most relevant time point when trial results should be available to the public. Fourth, our study focused on publications in the medical literature, but some companies have started making their trial results publicly available directly on their own Web sites. For example, the pharmaceutical industry's Clinical Study Results Database contains summaries of “hypothesis-testing” trials completed since October 2002 for many pharmaceutical products [20]. We searched this database for the 515 unpublished trials and found summaries for 22 (4%) of them. The effect of this and other related Web sites on public disclosure of trial data submitted to the FDA requires further research as the information reported in these databases may not be peer reviewed and there is no guarantee that the reporting is complete for all relevant data. Fifth, we could not determine the statistical significance of the findings of a substantial proportion of the studies. We did, however, obtain qualitatively similar results when we performed a sensitivity analysis by counting unknown statistical significance as a valid third category. Finally, our findings cannot be generalized to any specific product, company, institution, organization, or investigator.

Despite these limitations, our study provides ample evidence that in the years immediately following FDA approval that are most relevant to public health, there exists incomplete and selective publication of trials supporting approved new drugs. Potential reasons for this publication bias may include the tendency of investigators and sponsors to delay or not submit trial reports [21,22], or the motivation of commercial sponsors to publish positive trials in prestigious journals to obtain article reprints for marketing [23]. Bias in editorial decisions toward publishing positive results is also possible, although there is evidence suggesting that this is not the case [24,25]. Regardless of the cause, publication bias harms the public good by impairing the ability of clinicians and patients to make informed clinical decisions, and the ability of scientists to design safer and more efficient trials based on past findings. Publication bias can thus be considered a form of scientific misconduct [5].

Potential Effects of Mandatory Results Reporting on Publication Bias

As discussed above, the FDA Amendments Act of 2007 mandates basic public results reporting for all trials supporting FDA-approved drugs and devices. Our study shows that this legislation was necessary because current reporting is marked by pervasive publication bias of positive over negative trials. Moreover, because published trial reports are often incomplete [26] and have been shown to selectively report favorable outcome results [27], the published evidence supporting FDA-approved drugs may be even more skewed than our results suggest. By ensuring the reporting of all predeclared primary and secondary outcomes regardless of their direction of benefit, the new law should go a long way toward correcting this skew.

We anticipate that the new law will also speed the dissemination of trial information. Currently, according to our data, 40% of the trials that were eventually published were published more than 1 y postapproval (34% of pivotal trials). Under the new law, basic results for all trials must be posted by 1 y after trial completion or approval of the drug or device. This suggests that for all trials that the sponsor wishes to publish, the manuscripts will have to be submitted for peer review before the 1 y postapproval mark if they hope to allay journal concerns about publishing trials whose primary and secondary outcome results have already been publicly posted. Thus, we would expect the time-to-publication curves in Figures 2 and 3 to shift left.

Paradoxically, however, this new law may increase rather than decrease publication bias. Might sponsors feel less compelled to publish equivocal trials because the basic results will already be in the public domain? Might the time pressure to submit manuscripts by 1 y postapproval focus sponsor efforts even more on submitting positive trials and trials of greatest interest to journals? Might the journals, if they accept manuscripts of trials with publicly posted results, change the criteria by which publication importance is judged, and how might this affect acceptance rates [28]? When more detailed protocol information must also be posted on ClinicalTrials.gov, to start no later than October 2010, the effect on publication practices is even harder to anticipate. Our data document the current degree of publication bias and provide a baseline for assessing the evolving publication practices of trials supporting FDA-approved drugs as mandatory basic results reporting takes effect.

Supporting Information

Acknowledgments

We thank Sophia Jeng, MS and James Hamrick, MD for study design, data collection, and data analysis assistance; Chau Ong, PharmD and Roop Prabhu for assistance with data collection; and Erika Campbell for assistance with obtaining drug labels.

Author Contributions

KL codesigned the study; analyzed the data; cowrote the first draft; contributed to the writing of the paper; and reviewed final paper. PB helped design the study; helped analyze the data; contributed to the writing of the paper; and reviewed final paper. IS originated the idea for study, designed the study; helped analyze the data; co-wrote the first draft; contributed to the writing of the paper; and reviewed final paper.

References

  1. 1.[No author listed] (2006) Content and format of an application. Code of Federal Regulations (CFR) Title 21, Pt. 314.50.
  2. 2.[No author listed] (2006) Availability for public disclosure of data and information in an application or abbreviated application. CFR Title 21, Pt. 314.430.
  3. 3.[No author listed] (2006) Freedom of Information Act. Title 5 US Code 552(b)(4).
  4. 4.Simes RJ (1986) Publication bias: the case for an international registry of clinical trials. J Clin Oncol 4: 1529–1541.
  5. 5.Chalmers I (1990) Underreporting research is scientific misconduct. JAMA 263: 1405–1408.
  6. 6.Nissen SE, Wolski K (2007) Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 356: 2457–2471.
  7. 7.New York State CourtJun. 2004) Spitzer v. GlaxoSmithKline PLC. New York Superior Court. No. 04/401707.
  8. 8.Topol EJ (2004) Failing the public health—Rofecoxib, Merck, and the FDA. N Engl J Med 351: 1707–1709.
  9. 9.Mathews A, Martinez BNov. 2004) E-mails suggest Merck knew Vioxx's dangers at early stage. Wall Street Journal. (01.
  10. 10.Benjamin DK Jr., Smith PB, Murphy MD, Roberts R, Mathis L, et al. (2006) Peer-reviewed publication of clinical trials completed for pediatric exclusivity. JAMA 296: 1266–1273.
  1. 11.Sim I, Chan AW, Gülmezoglu AM, Evans T, Pang T (2006) Clinical Trial Registration: Transparency is the Watchword. The Lancet 367: 1631–1633.
  1. 12.Laine C, Horton R, Deangelis CD, Drazen JM, Frizelle FA, et al. (2007) Clinical trial registration: Looking back and moving ahead. N Engl J Med 356: 2734–2736.
  1. 13.Committee on the Assessment of the US Drug Safety System (2007) The future of drug safety: Promoting and protecting the health of the public. In: Baciu A, Stratton K, Burke SP, editors. Washington (D. C.): National Academies Press.
  2. 14.Hemminki E (1980) Study of information submitted by drug companies to licensing authorities. BMJ 280: 833–836.
  1. 15.Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003) Evidence b(i)ased medicine—Selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326: 1171–1173.
  1. 16.Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.
  1. 17.MedAdNews (2005) Top 50 companies. Available: http://www.pharmalive.com/magazines/medad/view.cfm?articleID=3799&f=3797. Accessed 3 August 2007.
  2. 18.Moher D, Schulz KF, Altman DG (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med 134: 657–662.
  1. 19.Khan A, Khan SR, Leventhal RM, Krishnan KR, Gorman JM (2002) An application of the revised CONSORT standards to FDA summary reports of recently approved antidepressants and antipsychotics. Biol Psychiatry 52: 62–67.
  1. 20.[No authors listed] (2007) PhRMA clinical study results database. Available: http://www.clinicalstudyresults.org/home/. Accessed 14 July 2008.
  2. 21.Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith H Jr. (1987) Publication bias and clinical trials. Control Clin Trials 8: 343–353.
  1. 22.Krzyzanowska MK, Pintilie M, Tannock IF (2003) Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 290: 495–501.
  1. 23.Smith R (2005) Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med 2: e138.
  1. 24.Lee KP, Boyd EA, Holroyd-Leduc JM, Bacchetti P, Bero LA (2006) Predictors of publication: Characteristics of submitted manuscripts associated with acceptance at major biomedical journals. Med J Aust 184: 621–626.
  1. 25.Olson CM, Rennie D, Cook D, Dickersin K, Flanagin A, et al. (2002) Publication bias in editorial decision making. JAMA 287: 2825–2828.
  1. 26.Moher D, Jones A, Lepage L (2001) Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 285: 1992–1995.
  1. 27.Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291: 2457–2465.
  1. 28.Groves T (2008) Mandatory disclosure of trial results for drugs and devices. BMJ 336: 170.