What is the probability of discovering breakthrough interventions in industry versus publicly sponsored randomized controlled trials (original) (raw)
Related papers
BMJ open, 2014
To understand how often 'breakthroughs,' that is, treatments that significantly improve health outcomes, can be developed. We applied weighted adaptive kernel density estimation to construct the probability density function for observed treatment effects from five publicly funded cohorts and one privately funded group. 820 trials involving 1064 comparisons and enrolling 331,004 patients were conducted by five publicly funded cooperative groups. 40 cancer trials involving 50 comparisons and enrolling a total of 19,889 patients were conducted by GlaxoSmithKline. We calculated that the probability of detecting treatment with large effects is 10% (5-25%), and that the probability of detecting treatment with very large treatment effects is 2% (0.3-10%). Researchers themselves judged that they discovered a new, breakthrough intervention in 16% of trials. We propose these figures as the benchmarks against which future development of 'breakthrough' treatments should be measu...
2020
Rationale, aims and objectives New therapies are increasingly approved by regulatory agencies such as the Food and Drug Administration (FDA) and the European Medicines Agency (EMA) based on testing in non-randomized clinical trials. These treatments have typically displayed "dramatic effects" (i.e., effects that are considered large enough to obviate the combined effects of bias and random error). The agencies, however, have not identified how large these effects should be to avoid the need for further testing in randomized controlled trials (RCTs). We investigated the effect size that would circumvent the need for further RCTs testing by the regulatory agencies. We hypothesized that the approval of therapeutic interventions by regulators is based on heuristic decision-making whose accuracy can be best characterized by the application of signal detection theory (SDT). Methods We merged the EMA and FDA database of approvals based on non-RCT comparisons. We excluded duplicate entries between the two databases. We included a total of 134 approvals of drugs and devices based on non-RCTs. We integrated Weber-Fechner law of psychophysics and recognition heuristics within SDT to provide descriptive explanations of the decisions made by the FDA and EMA to approve new treatments based on non-randomized studies without requiring further testing in RCTs. Results Our findings suggest that when the difference between novel treatments and the historical control is at least one logarithm (base 10) of magnitude, the veracity of testing in non-RCTs seems to be established. Conclusion Drug developers and practitioners alike can use the change in one logarithm of effect size as a benchmark to decide if further testing in RCTs should be pursued, or as a guide to interpreting the results reported in non-randomized studies. However, further research would be useful to better characterize the threshold of effect size above which testing in RCTs is not needed. When are randomized trials unnecessary? A signal detection theory approach to approving new treatments based on non-randomized studies Running head: When are randomized trials unnecessary?
A core principle of good public health practice is to base all policy decisions on the highest-quality scientific data, openly and objectively derived. 1 Determining whether data meet these conditions is difficult; uncertainty can lead to inaction by clinicians and public health decision makers. Although randomized, controlled trials (RCTs) have long been presumed to be the ideal source for data on the effects of treatment, other methods of obtaining evidence for decisive action are receiving increased interest, prompting new approaches to leverage the strengths and overcome the limitations of different data sources. 2-8 In this article, I describe the use of RCTs and alternative (and sometimes superior) data sources from the vantage point of public health, illustrate key limitations of RCTs, and suggest ways to improve the use of multiple data sources for health decision making. In large, well-designed trials, randomization evenly distributes known and unknown factors among control and intervention groups, reducing the potential for confounding. Despite their strengths, RCTs have substantial limitations. Although they can have strong internal validity, RCTs sometimes lack external validity ; generalizations of findings outside the study population may be invalid. 2,4,6 RCTs usually do not have sufficient study periods or population sizes to assess duration of treatment effect (e.g., waning immunity of vaccines) or to identify rare but serious adverse effects of treatment, which often become evident during post-marketing surveillance and long-term follow-up but could not be practically assessed in an RCT. The increasingly high costs and time constraints of RCTs can also lead to reliance on surrogate markers that may not correlate well with the outcome of interest. Selection of high-risk groups increases the likelihood of having adequate numbers of end points, but these groups may not be relevant to the broader target populations. These limitations and the fact that RCTs often take years to plan, implement, and analyze reduce the ability of RCTs to keep pace with clinical innovations; new products and standards of care are often developed before earlier models complete evaluation. These limitations also affect the use of RCTs for urgent health issues, such as infectious disease outbreaks, for which public health decisions must be made quickly on the basis of limited and often imperfect available data. RCTs are also limited in their ability to assess the individualized effect of treatment, as can result from differences in surgical techniques , and are generally impractical for rare diseases. Many other data sources can provide valid evidence for clinical and public health action. Observational studies, including assessments of results from the
Generating comparative evidence on new drugs and devices after approval
The Lancet, 2020
Certain limitations of evidence available on drugs and devices at the time of market approval often persist in the post-marketing period. Too often, post-marketing research landscape is fragmented. When regulatory agencies require pharmaceutical and device manufacturers to conduct studies in the post-marketing period, these studies may remain incomplete many years after approval. Even when completed, many post-marketing studies lack meaningful active comparators, have observational designs, and may not collect patient-relevant outcomes. It is crucial for regulators, in collaboration with the industry and patients, to ensure that the important questions that are unanswered at the time of drug and device approval are resolved in a timely fashion during the post-marketing phase. We propose a set of seven key guiding principles that we believe will provide the necessary incentives for pharmaceutical and device manufacturers to generate comparative data in the post-marketing period. First, regulators and pharmaceutical companies (for drugs), notified bodies and manufacturers (for devices) should develop customised evidence generation plans, ensuring that future post-approval studies address any limitations of the data available at the time of market entry that would influence the benefit-risk profiles of drugs and devices. Second, post-marketing studies should be designed hierarchically: priority should be given to efforts aimed at evaluating a product's net clinical benefit in randomised trials compared with current known effective therapy, whenever possible, to address common decisional dilemmas. Third, post-marketing studies should incorporate active comparators as appropriate. Fourth, use of non-randomised studies for the evaluation of clinical benefit in the post-marketing period should be limited to instances when the magnitude of effect is deemed to be very large or when it is possible to reasonably infer the comparative benefits or risks in settings where doing a randomised trial is not feasible. Fifth, efficiency of randomised trials should be improved by streamlining patient recruitment and data collection through innovative design elements. Sixth, governments should directly support and facilitate the production of comparative post-marketing data by investing in the development of collaborative research networks and data systems that reduce the complexity, cost, and waste of rigorous postmarketing research efforts. Seventh, financial incentives and penalties should be developed or more actively reinforced.
Molecular Therapy - Methods & Clinical Development, 2020
Advanced therapy medicinal products (ATMPs) comprising cell therapy, gene therapy, and tissue-engineered products, offer a multitude of novel therapeutic approaches to a wide range of severe and debilitating diseases. To date, several advanced therapies have received marketing authorization for a variety of indications. However, some products showed disappointing market performance, leading to their withdrawal. The available evidence for quality, safety, and efficacy at product launch can play a crucial rule in their market success. To evaluate the sufficiency of evidence in submissions of advanced therapies for marketing authorization and to benchmark them against more established biological products, we conducted a matched comparison of the regulatory submissions between ATMPs and other biologicals. We applied a quantitative assessment of the regulatory objections and divergence from the expected data requirements as indicators of sufficiency of evidence and regulatory flexibilty, respectively. Our results demonstrated that product manufacturing was challenging regardless of the product type. Advanced therapies displayed critical deficiencies in the submitted clinical data. The submitted non-clinical data packages benefited the most from regulatory flexibility. Additionally, ATMP developers need to comply with more commitments in the post-approval phase, which might add pressure on market performance. Mitigating such observed deficiencies in future product development, may leverage their potential for market success.
Clinical Trials Trends of 2023 Year and Visionary to the Future
International Journal of Clinical Investigation and Case Reports, 2023
Introduction: The importance of studying historical changes in the development of human activity is substantiated by the need to systematize such changes and the possibility of predicting them. Historical changes are extended in time and do not have clear boundaries, requiring greater involvement in their study and the prerequisites for their appearance. Clinical research is more than just the practical application of medical changes and discoveries. They make changes in medical practice but are subject to change. Changes in the clinical research industry are tendentious and develop gradually, requiring study and forecasting. According to the generally accepted temporal gradation of the forecast, there is an operational forecast of up to one month, a short-term forecast of up to one year, a medium-term forecast of up to five years, a long-term forecast of up to 20 years and a long-term forecast over long-term, and a short-term forecast is common in the clinical research industry. We analyzed publications in open sources from 1930 to 2023 by keywords in the Russian-language literature trends in the clinical trial industry and the English-language literature trends in the clinical trial industry. Discussion and Conclusion: Trends in the development of clinical trials until the end of 2023 can be divided into two groups, those related to changes in the conduct of clinical trials and changes in the products of clinical trials in nosologies. If in the first group, the trends remain similar to 2022, the ongoing digitalization of operations, the shift of centralized research towards decentralization, and the shift in protocol design towards patient-centricity, then in the second group, the number of expected drugs has decreased, and there is a shift of drugs towards biologics and gene therapy drugs.