Considerations when assessing heterogeneity of treatment effect in patient-centered outcomes research (original) (raw)
Related papers
BMC Medical Research Methodology, 2012
Implicit in the growing interest in patient-centered outcomes research is a growing need for better evidence regarding how responses to a given intervention or treatment may vary across patients, referred to as heterogeneity of treatment effect (HTE). A variety of methods are available for exploring HTE, each associated with unique strengths and limitations. This paper reviews a selected set of methodological approaches to understanding HTE, focusing largely but not exclusively on their uses with randomized trial data. It is oriented for the "intermediate" outcomes researcher, who may already be familiar with some methods, but would value a systematic overview of both more and less familiar methods with attention to when and why they may be used. Drawing from the biomedical, statistical, epidemiological and econometrics literature, we describe the steps involved in choosing an HTE approach, focusing on whether the intent of the analysis is for exploratory, initial testing, or confirmatory testing purposes. We also map HTE methodological approaches to data considerations as well as the strengths and limitations of each approach. Methods reviewed include formal subgroup analysis, meta-analysis and meta-regression, various types of predictive risk modeling including classification and regression tree analysis, series of n-of-1 trials, latent growth and growth mixture models, quantile regression, and selected non-parametric methods. In addition to an overview of each HTE method, examples and references are provided for further reading. By guiding the selection of the methods and analysis, this review is meant to better enable outcomes researchers to understand and explore aspects of HTE in the context of patient-centered outcomes research.
The good, the bad, and the different: a primer on aspects of heterogeneity of treatment effects
Journal of managed care pharmacy: JMCP
The concept of heterogeneity is concerned with understanding differences within and across patients and studies. Heterogeneity of treatment effects is nonrandom variability in response to treatment and includes both benefits and harms. Because not all patients respond the same way, treatment decisions applied in a "one size fits all" fashion based on the average response observed in clinical trials may lead to suboptimal outcomes for some patients. Variation in outcomes among patients may be caused by observable and nonobservable factors. Changes in patients' health status over time can contribute to variability among patients. Assuming that the results from clinical trials are homogeneous across patients may fail to take into account clinically significant variability where some patients may receive benefit and others harm. Subgroup analyses and prediction models are 2 tools to explain variability observed within a study. Evidence synthesis with meta-analysis can prov...
Assessing Heterogeneity of Treatment Effects: Are Authors Misinterpreting Their Results?
Health Services Research, 2010
Objective. To determine whether investigations of heterogeneity of treatment effects (HTE) in randomized-controlled trials (RCTs) are prespecified and whether authors' interpretations of their analyses are consistent with the objective evidence. Data Sources/Study Setting. We reviewed 87 RCTs that reported formal tests for statistical interaction or heterogeneity (HTE analyses), derived from a probability sample of 541 articles. Data Collection/Extraction. We recorded reasons for performing HTE analysis; an objective classification of evidence for HTE (termed ''clinicostatistical divergence'' [CSD]); and authors' interpretations of findings. Authors' interpretations, compared with CSD, were coded as understated, overstated, or adequately stated. Principle Findings. Fifty-three RCTs (61 percent) claimed prespecified covariates for HTE analyses. Trials showed strong (6), moderate (11), weak (25), or negligible (16) evidence for CSD (29 could not be classified due to inadequate information). Authors stated that evidence for HTE was sufficient to support differential treatment in subgroups (10); warranted more research (31); was absent (21); or provided no interpretation (25). HTE was overstated in 22 trials, adequately stated in 57 trials, and understated in 8 trials. Conclusions. Inconsistencies in performance and reporting may limit the potential of HTE analysis as a tool for identifying HTE and individualizing care in diverse populations. Recommendations for future studies on the reporting and interpretation of HTE analyses are provided.
Heterogeneity of Treatment Effects: Implications for Guidelines, Payment, and Quality Assessment
The American Journal of Medicine, 2007
Randomized controlled trial results are needed for developing guidelines, payment rules, and quality-ofcare measures; however, 2 phenomena reduce the usefulness of randomized controlled trial findings. First, these studies now enroll patients with less severe disease, who are less likely to benefit from a drug or treatment. Second, patients are living longer but, as a result, have more chronic diseases. Although randomized controlled trials often exclude these older patients, trial findings continue to be generalized to them. Together, these phenomena impose challenges to the usefulness of the results of randomized controlled trials for clinical and policy applications. The convergence of these phenomena makes the current research paradigm underlying evidence-based medicine, guideline development and quality assessment fundamentally flawed in 2 ways. First, the "evidence" includes patients who may have a minimal benefit from the treatment being tested. This could reduce the power for the trial and yield negative or null results, leading to undertreatment of a group of patients with potential for a greater-than-observed benefit. Second, attempts to generalize the results from positive trials to patients who have been excluded from those trials may result in the overtreatment of those who could not benefit (e.g., because they will die from other causes before the benefit of treatment would occur) and therefore represents a parallel hazard. In this article, we describe sources of heterogeneity of treatment effects (HTE) within trials, which can compromise the interpretation and generalizability of results. We also examine strategies for understanding and managing HTE in practice, to increase the usefulness of trial results.
Journal of dental, oral and craniofacial epidemiology, 2013
Because each patient's baseline (pre-treatment) characteristics differ (e.g., age, sex, socioeconomic status, ethnicity/race, biomarkers), treatments do not work the same for every patient-some can even cause detrimental effects. To improve patient care, it is critical to identify such heterogeneity of treatment effects. But the standard analytic approach dichotomizes baseline characteristics (low vs. high) which often leads to a loss of critical patient-care information and power to detect heterogeneity, as the results may depend strongly on the cut-points chosen. A more powerful analytic approach is to analyze baseline characteristics (i.e., covariates) measured on a continuous scale that retains all of the information available for the covariate. In this article, we show how the Johnson-Neyman (J-N) method can be used to identify the prognostic and predictive value of baseline covariates measured on a continuous scale - findings that often cannot be determined using the tradi...
Risk adjustment and outcome research. Part I
Journal of Cardiovascular Medicine, 2006
Objective The increasing demand for comparative evaluation of outcomes requires the development and diffusion of epidemiologic research, the ability to correctly formulate hypotheses, to conduct analyses and to interpret the results. The purpose of this paper is to provide a detailed but easy-reading review of epidemiologic methods to compare healthcare outcomes, particularly riskadjustment methods. Methods The paper is divided into three parts. Part I describes confounding in observational studies, the ways confounding is identified and controlled (propensity adjustment and risk adjustment), and the methods for constructing the severity measures in risk-adjustment procedures. Conclusions It is becoming increasingly important for policy makers and planners to identify which factors may improve or worsen the effectiveness of treatments and services and to compare the performances of providers. Politicians, managers, epidemiologists, and clinicians should make their decisions based on the validity and precision of study results, by using the best scientific knowledge available. The statistical methods described in this review cannot measure 'reality' as it 'truly' is, but can produce 'images' of it, defining limits and uncertainties in terms of validity and precision. Studies that use credible risk-adjustment strategies are more likely to yield reliable and applicable findings.
Bayesian analysis of heterogeneous treatment effects for patient-centered outcomes research
Health Services and Outcomes Research Methodology, 2016
Evaluation of heterogeneity of treatment effect (HTE) is an essential aspect of personalized medicine and patient-centered outcomes research. Our goal in this article is to promote the use of Bayesian methods for subgroup analysis and to lower the barriers to their implementation by describing the ways in which the companion software beanz can facilitate these types of analyses. To advance this goal, we describe several key Bayesian models for investigating HTE and outline the ways in which they are well-suited to address many of the commonly cited challenges in the study of HTE. Topics highlighted include shrinkage estimation, model choice, sensitivity analysis, and posterior predictive checking. A case study is presented in which we demonstrate the use of the methods discussed.
Pilot and Feasibility Studies, 2018
Background: We used various methods for identifying and prioritizing patient-centered outcomes (PCOs) for comparative effectiveness research (CER). Methods: We considered potential PCOs ("benefits" and "harms") related to (1) gabapentin for neuropathic pain and (2) quetiapine for bipolar depression. Part 1 (April 2014 to March 2015): we searched for PCO research and core outcome sets (COSs). We conducted electronic searches of bibliographic databases and key websites and examined FDA prescribing information and reports of clinical trials and systematic reviews. We asked patient and clinician co-investigators to identify PCOs. Part 2 (not part of our original study protocol): in 2015, we surveyed members of The TMJ Association, Ltd., a patient group associated with temporomandibular disorders (4130 invitations sent). Participants prioritized (1) the importance of six potential benefits and (2) 21 potential harms selected by the investigators in part 1, using stated preference methods. We calculated descriptive statistics. Results: In part 1, we identified a COS for pain, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) recommendations. The COS identified several important benefits, but it lacked specific recommendations about which potential harms to include in CER. We did not identify a COS for bipolar depression. Research reports, prescribing information, and patient co-investigators helped identify but not prioritize outcomes. We abandoned our electronic search for PCO research because we found it would be resource-intensive and yield few relevant reports. In part 2, surveying patients was useful for prioritizing PCOs. Members of The TMJ Association, Ltd., completed the survey (N = 746) and successfully prioritized both benefits and harms. Participants did not identify many benefits other than those we identified in part 1; several participants identified additional harms. Conclusions: These exploratory results could inform future research about identifying and prioritizing PCOs. We found that stakeholder co-investigators and research reports contributed to identifying PCOs; surveying a patient group contributed to prioritizing PCOs. Prioritizing potential harms was particularly challenging because there are many more potential harms than potential benefits. Methods for identifying and prioritizing potential benefits for CER might not be appropriate for harms. Further research is needed to determine the generalizability of these results.