CLINICAL HETEROGENEITY IN SYSTEMATIC REVIEWS AND HEALTH TECHNOLOGY ASSESSMENTS: SYNTHESIS OF GUIDANCE DOCUMENTS AND THE LITERATURE | International Journal of Technology Assessment in Health Care | Cambridge Core (original) (raw)

Abstract

Objectives: The aim of this study was to synthesize best practices for addressing clinical heterogeneity in systematic reviews and health technology assessments (HTAs).

Methods: We abstracted information from guidance documents and methods manuals made available by international organizations that develop systematic reviews and HTAs. We searched PubMed® to identify studies on clinical heterogeneity and subgroup analysis. Two authors independently abstracted and assessed relevant information.

Results: Methods manuals offer various definitions of clinical heterogeneity. In essence, clinical heterogeneity is considered variability in study population characteristics, interventions, and outcomes across studies. It can lead to effect-measure modification or statistical heterogeneity, which is defined as variability in estimated treatment effects beyond what would be expected by random error alone. Clinical and statistical heterogeneity are closely intertwined but they do not have a one-to-one relationship. The presence of statistical heterogeneity does not necessarily indicate that clinical heterogeneity is the causal factor. Methodological heterogeneity, biases, and random error can also cause statistical heterogeneity, alone or in combination with clinical heterogeneity.

Conclusions: Identifying potential modifiers of treatment effects (i.e., effect-measure modifiers) is important for researchers conducting systematic reviews and HTAs. Recognizing clinical heterogeneity and clarifying its implications helps decision makers to identify patients and patient populations who benefit the most, who benefit the least, and who are at greatest risk of experiencing adverse outcomes from a particular intervention.

References

2.Berlin, JA. Invited commentary: Benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemiol. 1995;142:383–387.CrossRefGoogle ScholarPubMed

3.Berlin, JA, Santanna, J, Schmid, CH, Szczech, LA, Feldman, HI. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: Ecological bias rears its ugly head. Stat Med. 2002;21:371–387.CrossRefGoogle ScholarPubMed

4.Borenstein, M, Hedges, LV, Higgins, JPT, Rothstein, HR. Introduction to meta-analysis. New York: John Wiley and Sons, Ltd.; 2009.CrossRefGoogle Scholar

6.Cochran, W. The combination of estimates from different experiments. Biometrics. 1954;10:101–121.CrossRefGoogle Scholar

7.Colditz, GA, Burdick, E, Mosteller, F. Heterogeneity in meta-analysis of data from epidemiologic studies: A commentary. Am J Epidemiol. 1995;142:371–382.CrossRefGoogle ScholarPubMed

8.Dias, S, Welton, NJ, Caldwell, DM, Ades, AE. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 29:932–944.CrossRefGoogle Scholar

12.Greenland, S, O'Rourke, K. Meta-analysis. In: Rothman, KJ, Greenland, S, Lash, TL eds. Modern Epidemiology. Philadelphia, PA: Lippincott Williams & Wilkins; 2008:652–682.Google Scholar

14.Higgins, JP, Thompson, SG, Deeks, JJ, Altman, DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–560.CrossRefGoogle ScholarPubMed

16.Ioannidis, JP, Patsopoulos, NA, Evangelou, E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335:914–916.CrossRefGoogle ScholarPubMed

17.Kennedy, SH, Fulton, KA, Bagby, RM, et al. Sexual function during bupropion or paroxetine treatment of major depressive disorder. Can J Psychiatry. 2006;51:234–242.CrossRefGoogle ScholarPubMed

18.Kravitz, RL, Duan, N, Braslow, J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82:661–687.CrossRefGoogle ScholarPubMed

19.Layton, D, Pearce, GL, Shakir, SA. Safety profile of tolterodine as used in general practice in England: Results of prescription-event monitoring. Drug Saf. 2001;24:703–713.CrossRefGoogle ScholarPubMed

20.Morton, SC, Adam, JL, Suttorp, MJ, Shekelle, PG. Meta-regression approaches: What, why, when, and how? Technical Review 8 prepared by Southern California-RAND-Evidence-based Practice Center, under Contract No 290-97-0001. AHRQ Publication No 04–033, Rockville, MD: Agency for Healthcare Research and Quality, 2004.Google Scholar

22.Poole, C, Greenland, S. Random-effects meta-analyses are not always conservative. Am J Epidemiol. 1999;150:469–475.CrossRefGoogle Scholar

23.Shadish, WR, Cook, TD, Campbell, DT. Experimental and quasi-experimental designs for generalized causal inference. Florence, KY: Houghton Mifflin College; 2003.Google Scholar

25.West, SL, Gartlehner, G, Mansfield, AJ et al. , Comparative effectiveness review methods: Clinical heterogeneity. Rockville, MD. Agency for Healthcare Research and Quality: RTI International-University of North Carolina Evidence-based Practice Center; 2010.Google Scholar