Moving from efficacy to effectiveness trials in prevention research (original) (raw)

Abstract

Efficacy trials test whether interventions work under optimal, highly controlled conditions whereas effectiveness trials test whether interventions work with typical clients and providers in real-world settings. Researchers, providers, and funding bodies have called for more effectiveness trials to understand whether interventions produce effects under ecologically valid conditions, which factors predict program effectiveness, and what strategies are needed to successfully implement programs in practice settings. The transition from efficacy to effectiveness with preventive interventions involves unique considerations, some of which are not shared by treatment research. The purpose of this article is to discuss conceptual and methodological issues that arise when making the transition from efficacy to effectiveness research in primary, secondary, and tertiary prevention, drawing on the experiences of two complimentary research groups as well as the existing literature. We address (a) program of research, (b) intervention design and conceptualization, (c) participant selection and characteristics, (d) providers, (e) context, (f) measurement and methodology, (g) outcomes, (h) cost, and (i) sustainability. We present examples of research in eating disorder prevention that demonstrate the progression from efficacy to effectiveness trials. Keywords effectiveness; efficacy; translational research; eating disorder prevention Efficacy trials are designed to evaluate whether interventions produce effects under optimal conditions, in which providers are well-trained and closely supervised, interventions are delivered in adequately staffed research clinics, and participants are often homogeneous (Flay, 1986; Glasgow, Lichtenstein, & Marcus, 2003). In contrast, effectiveness trials are designed to evaluate whether interventions produce effects when delivered by endogenous providers (e.g., school counselors, hospital staff), under real world conditions in natural settings with heterogeneous samples (Flay, 1986; Glasgow et al., 2003). In reality much research lies on a continuum between the two; "pragmatic" randomized controlled trials are a hybrid of these, blending aspects of experimental control and external validity (Schwartz

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (65)

  1. Backer T, Liberman R, Kuehnel T. Dissemination and adoption of innovative psychosocial interventions. Journal of Consulting and Clinical Psychology. 1986; 54:111-118. [PubMed: 3958295]
  2. Baron R, Kenny D. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986; 51:1173-1182. [PubMed: 3806354]
  3. Barrera M, Sandler I. Prevention: A report of progress and momentum into the future. Clinical Psychology: Science and Practice. 2006; 13(3):221-226.
  4. Bausell R. Translation research [Special issue]. Evaluation and the Health Professions. 2006; 29(1)
  5. Becker CB, Bull S, Schaumberg K, Cauble A, Franco A. Effectiveness of peer-led eating disorders prevention: A replication trial. Journal of Consulting and Clinical Psychology. 2008; 76:347-354. [PubMed: 18377130]
  6. Becker CB, Smith LM, Ciao AC. Reducing eating disorder risk factors in sorority members: A randomized trial. Behavior Therapy. 2005; 36:245-253.
  7. Becker CB, Smith LM, Ciao AC. Peer facilitated eating disorders prevention: A randomized effectiveness trial of cognitive dissonance and media advocacy. Journal of Counseling Psychology. 2006; 53:550-555.
  8. Becker CB, Stice E, Shaw H, Woda S. Use of empirically-supported interventions for psychopathology: Can the participatory approach move us beyond the research-to-practice gap? Behaviour Research and Therapy. 2009; 47:265-274. [PubMed: 19281965]
  9. Becker CB, Wilson C, Williams A, Kelly M, McDaniel L, Elmquist J. Peer-facilitated cognitive dissonance versus healthy weight eating disorders prevention: A randomized comparison. Body Image. 2010; 7:280-288. [PubMed: 20638351]
  10. Chambless D, Hollon S. Defining empirically supported therapies. Journal of Consulting and Clinical Psychology. 1998; 66(1):7-18. [PubMed: 9489259]
  11. Clarke G. Improving the transition from basic efficacy research to effectiveness studies: Methodological issues. Journal of Consulting and Clinical Psychology. 1995; 63(5):718-725. [PubMed: 7593864]
  12. Cohen, J.; Neumann, P. Cost-savings and cost-effectiveness of clinical preventive care. Robert Wood Johnson Foundation -The Synthesis Project, Research Synthesis Report No. 18. 2009. Retrieved October 14, 2010 from http://www.rwjf.org/files/research/ 100709.policysnythesis.preventivecare.report.pdf
  13. Dolan P. Developing methods that really do value the "Q" in the QALY. Health Economics, Policy, and Law. 2008; 3:69-77.
  14. Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health. 2004; 94:416-422. [PubMed: 14998805]
  15. Fenwick E, Byford S. A guide to cost-effectiveness acceptability curves. British Journal of Psychiatry. 2005; 187:106-108. [PubMed: 16055820]
  16. Festinger, L. A theory of cognitive dissonance. Stanford: Stanford University Press; 1957.
  17. Fixsen D, Blase K, Naoom S, Wallace F. Core implementation components. Research on Social Work Practice. 2009; 19:531-540.
  18. Flay B. Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Preventive Medicine. 1986; 15:451-474. [PubMed: 3534875]
  19. Flay B, Biglan A, Boruch R, Castro F, Gottfredson D, Kellam S, et al. Standards of evidence: Criteria for efficacy, effectiveness, and dissemination. Prevention Science. 2005; 6(3):151-175. [PubMed: 16365954]
  20. Glasgow R, Green L, Klesges L, Abrams D, Fisher E, Goldstein M, Hayman L, Ockene J, Orleans T. External validity: We need to do more. Annals of Behavioral Medicine. 2006; 31:105-108. [PubMed: 16542124]
  21. Glasgow R, Lichtenstein E, Marcus A. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health. 2003; 93(8):1261-1267. [PubMed: 12893608]
  22. Gordon R. An operational classification of disease prevention. Public Health Reports. 1983; 98:107- 109. [PubMed: 6856733]
  23. Green L, Glasgow R. Relevance, generalization, and applicability of research: Issues in external validation and translation methodology. Evaluation and the Health Professions. 2006; 29(1):126- 153. [PubMed: 16510882]
  24. Green M, Scott N, Diyankova I, Gasser C, Pederson E. Eating disorder prevention: An experimental comparison of high level dissonance, low level dissonance, and no-treatment control. Eating Disorders. 2005; 13:157-169. [PubMed: 16864339]
  25. Herschell A, McNeil C, McNeil D. Clinical child psychology's progress in disseminating empirically supported treatments. Clinical Psychology: Science and Practice. 2004; 11(3):267-288.
  26. Hoagwood K, Hibbs E, Brent D, Jensen P. Introduction to the special section: Efficacy and effectiveness in studies of child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology. 1995; 63:683-687. [PubMed: 7593860]
  27. Hoagwood K, Olin S. The NIMH blueprint for change report: Research priorities in child and adolescent mental health. Journal of the American Academy of Child and Adolescent Psychiatry. 2002; 41(7):760-767. [PubMed: 12108799]
  28. Hoch J, Briggs A, Willan A. Something old, something new, something borrowed, something blue: A framework for the marriage of health econometrics and cost-effectiveness analysis. Health Economics. 2002; 11:415-430. [PubMed: 12112491]
  29. Holder H, Flay B, Howard J, Boyd G, Voas R, Grossman M. Phases of alcoholism prevention research. Alcoholism: Clinical and Experimental Research. 1999; 23(1):183-194.
  30. Israel, BA.; Eng, E.; Schulz, AJ.; Parker, EA. Introduction to methods in community-based participatory research for health. In: Israel, BA.; Eng, E.; Shulz, AJ.; Parker, EA., editors. Methods in community-based participatory research for health. San Francisco: Jossey-Bass; 2005. p. 3-26.
  31. Kazdin, AE. Research Design in Clinical Psychology. 4. Boston, MA: Allyn and Bacon; 2003.
  32. Klesges L, Eastabrooks P, Dzewaltowski D, Bull S, Glasgow R. Beginning with the application in mind: Designing and planning health behavior change interventions to enhance dissemination. Annals of Behavioral Medicine. 2005; 29(suppl):66-75. [PubMed: 15921491]
  33. Knapp M, Mangalore R. The trouble with QALY's…. Social Psychiatry and Psychiatric Epidemiology. 2007; 61:348-358.
  34. Lorig K, Hurwicz M, Sobel D, Hobbs M, Ritter P. A national dissemination of an evidence-based self- management program: A process evaluation study. Patient Education and Counseling. 2005; 59:69-79. [PubMed: 16198220]
  35. Lynch F, Striegel-Moore R, Dickerson J, Perrin N, DeBar L, Wilson G, Kraemer H. Cost-effectiveness of guided self-help treatment for recurrent binge eating. Journal of Consulting and Clinical Psychology. 2010; 78:322-333. [PubMed: 20515208]
  36. Matusek JA, Wendt SJ, Wiseman CV. Dissonance thin-ideal and didactic healthy behavior eating disorder prevention programs: Results from a controlled trial. International Journal of Eating Disorders. 2004; 36:376-388. [PubMed: 15558649]
  37. Merrell K, Juskelis M, Tran O, Buchanan R. Social and emotional learning in the classroom: Evaluation of strong kids and strong teens on students' social-emotional knowledge and symptoms. Journal of Applied School Psychology. 2008; 24:209-224.
  38. Mitchell KS, Mazzeo SE, Rausch SM, Cooke KL. Innovative interventions for disordered eating: Evaluating dissonance-based and yoga interventions. International Journal of Eating Disorders. 2007; 40:120-128. [PubMed: 17089413]
  39. Mrazek, P.; Haggerty, R. Reducing risks for mental disorders: Frontiers for preventive intervention research. Washington, DC: National Academy Press; 1994.
  40. Perez M, Becker CB, Ramirez-Cash A. Transportability of an empirically supported dissonance-based prevention program for eating disorders. Body Image. 2010; 7:179-186. [PubMed: 20335084]
  41. Pineda, GG. Tesis doctoral inédita. Facultad de Psicología. Universidad Nacional Autónoma de México; México: 2006. Estrategias preventivas de factores de riesgo en trastornos de la conducta alimentaria.
  42. Poulin F, Dishion TJ, Burraston B. 3-year iatrogenic effects associated with aggregating high-risk adolescents in preventive interventions. Applied Developmental Science. 2001; 5:214-224.
  43. Prochaska J, DiClemente C, Norcross J. In search of how people change: Applications to addictive behaviors. American Psychologist. 1992; 47(9):1102-1114. [PubMed: 1329589]
  44. Reynolds K, Spruijt-Metz D. Translational research in childhood obesity prevention. Evaluation and the Health Professions. 2006; 29(2):219-245. [PubMed: 16645185]
  45. Roehrig M, Thompson JK, Brannick M, van den Berg P. Dissonance-based eating disorder prevention program: A preliminary dismantling investigation. International Journal of Eating Disorders. 2006; 39:1-10. [PubMed: 16254869]
  46. Roy-Byrne P, Sherbourne C, Craske M, Stein M, Katon W, Sullivan G, et al. Moving treatment research from clinical trials to the real world. Psychiatric Services. 2003; 54(3):327-332. [PubMed: 12610239]
  47. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. Journal of Chronic Disease. 1967; 20:637-648. Reprinted in Journal of Clinical Epidemiology, 62, 499-505.
  48. Solomon J, Card J, Malow R. Adapting efficacious interventions: Advancing translational research in HIV prevention. Evaluation and the Health Professions. 2006; 29(2):162-194. [PubMed: 16645183]
  49. Stice E. Risk and maintenance factors for eating pathology: A meta-analytic review. Psychological Bulletin. 2002; 128:825-848. [PubMed: 12206196]
  50. Stice E, Marti N, Spoor S, Presnell K, Shaw H. Dissonance and healthy weight eating disorder prevention programs: Long-term effects from a randomized efficacy trial. Journal of Consulting and Clinical Psychology. 2008a; 76:329-340. [PubMed: 18377128]
  51. Stice, E.; Presnell, K. The Body Project: Promoting Body Acceptance and Preventing Eating Disorders, Facilitators Guide. New York: Oxford University Press; 2007.
  52. Stice E, Rohde P, Seeley J, Gau J. Brief cognitive-behavioral depression prevention program for high- risk adolescents outperforms alternative interventions: A randomized efficacy trial. Journal of Consulting and Clinical Psychology. 2008b; 76:595-606. [PubMed: 18665688]
  53. Stice E, Rohde P, Gau J, Shaw H. An effectiveness trial of a dissonance-based eating disorder prevention program for high-risk adolescent girls. Journal of Consulting and Clinical Psychology. 2009; 77:825-834. [PubMed: 19803563]
  54. Stice E, Shaw H. Eating disorder prevention programs: A meta-analytic review. Psychological Bulletin. 2004; 130:206-227. [PubMed: 14979770]
  55. Stice E, Shaw H, Burton E, Wade E. Dissonance and healthy weight eating disorder prevention programs: A randomized efficacy trial. Journal of Consulting and Clinical Psychology. 2006a; 74(2):263-275. [PubMed: 16649871]
  56. Stice E, Shaw H, Marti CN. A meta-analytic review of obesity prevention programs for children and adolescents: The skinny on interventions that work. Psychological Bulletin. 2006b; 132:667-691. [PubMed: 16910747]
  57. Stinnett A, Mullahy J. Net health benefits: A new framework for the analysis of uncertainty in cost- effectiveness analysis. Medical Decision Making. 1998; 18(Suppl 2):68-80.
  58. Stouthard M, Essink-Bot M, Bonsel G. Disability weights for diseases: A modified protocol and results for a Western European region. European Journal of Public Health. 2000; 10:24-30.
  59. Sussman S, Valente T, Rohrbach L, Skara S, Pentz M. Translation in the health professions: Converting science into action. Evaluation and the Health Professions. 2006; 29(1):7-32. [PubMed: 16510878]
  60. Tunis S, Stryer D, Clancy C. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. Journal of the American Medical Association. 2003; 12:1624-1632. [PubMed: 14506122]
  61. Wells KB. Treatment research at the crossroads: The scientific interface of clinical trials and effectiveness research. American Journal of Psychiatry. 1999; 156(1):5-10. [PubMed: 9892291]
  62. Wilson, GT. Manual-based treatment: Evolution and evaluation. In: Treat, TA.; Bootzin, RR.; Baker, TB., editors. Psychological clinical science: Papers in honor of Richard M. McFall. Modern pioneers in psychological science. New York, NY: Psychology Press; 2007. p. 105-132.
  63. Zubrick S, Ward K, Silburn S, Lawrence D, Williams A, Blair E, et al. Prevention of child behavior problems through primary implementation of a group behavioral family intervention. Prevention Science. 2005; 6(4):287-304. [PubMed: 16160760]
  64. Zwarentstein M, Treweek S. What kind of randomized trials do we need? Canadian Medical Association Journal. 2009; 180:998-1000. [PubMed: 19372438]
  65. Zwarenstein M, Treweek S, Gagnier J, Altman D, Tunis S, Haynes B, Oxman A, Moher D. Improving the reporting of pragmatic trials: An extension of the CONSORT statement. British Medical Journal. 2008; 337:a2390-a2397. [PubMed: 19001484]