Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update (original) (raw)
Related papers
BMJ
The UK Medical Research Council's widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research. Complex interventions are commonly used in the health and social care services, public health practice, and other areas of social and economic policy that have consequences for health. Such interventions are delivered and evaluated at different levels, from individual to societal levels. Examples include a new surgical procedure, the redesign of a healthcare programme, and a change in welfare policy. The UK Medical Research Council (MRC) published a framework for researchers and research funders on developing and evaluating complex interventions in 2000 and revised guidance in 2006. 1-3 Although these documents continue to be widely used and are now accompanied by a range of more detailed guidance on specific aspects of the research process, 4-8 several important conceptual, methodological and theoretical developments have taken place since 2006. These developments have been included in a new framework commissioned by the National Institute of Health Research (NIHR) and the MRC. 9 The framework aims to help researchers work with other stakeholders to identify the key questions about complex interventions, and to design and conduct research with a diversity of perspectives and appropriate choice of methods.
Background: Once a proposed topic has been identified for a systematic review and has undergone a question formulation stage, a protocol must be developed that specifies the scope and research questions in detail and outlines the methodology for conducting the systematic review. Rationale: Framework modifications are often needed to accommodate increased complexity. We describe and give examples of adaptations and alternatives to traditional analytic frameworks. Discussion: This article identifies and describes elements of frameworks and how they can be adapted to inform the protocol and conduct of systematic reviews of complex interventions. Modifications may be needed to adapt the population, intervention, comparators, and outcomes normally used in protocol development to successfully describe complex interventions; in some instances, alternative frameworks may be better suited. Possible approaches to analytic frameworks for complex interventions that illustrate causal and associative linkages are outlined, including time elements, which systematic reviews of complex interventions may need to address. The need for and specifics of the accommodations vary with details of a specific systematic review. This in turn helps determine whether traditional frameworks are sufficient, can be refined, or if alternate frameworks must be adopted.
AHRQ Series on Complex Intervention Systematic Reviews - Paper 5: Advanced Analytic Methods
Journal of clinical epidemiology, 2017
Advanced analytic methods for synthesizing evidence about complex interventions continue to be developed. In this paper, we emphasize that the specific research question posed in the review should be used as a guide for choosing the appropriate analytic method. We present advanced analytic approaches that address four common questions that guide reviews of complex interventions: 1) How effective is the intervention?; 2) For whom does the intervention work and in what contexts?; 3) What happens when the intervention is implemented?; and 4) What decisions are possible given the results of the synthesis? The analytic approaches presented in this paper are particularly useful when each primary study differs in components, mechanisms of action, context, implementation, timing, and many other domains.
Systematic reviews, 2015
In order to enable replication of effective complex interventions, systematic reviews need to provide evidence about their critical features and clear procedural details for their implementation. Currently, few systematic reviews provide sufficient guidance of this sort. Through a worked example, this paper reports on a methodological approach, Intervention Component Analysis (ICA), specifically developed to bridge the gap between evidence of effectiveness and practical implementation of interventions. By (a) using an inductive approach to explore the nature of intervention features and (b) making use of trialists' informally reported experience-based evidence, the approach is designed to overcome the deficiencies of poor reporting which often hinders knowledge translation work whilst also avoiding the need to invest significant amounts of time and resources in following up details with authors. A key strength of the approach is its ability to reveal hidden or overlooked interve...
Framework for design and evaluation of complex interventions to improve health
British Medical Journal, 2000
Randomised controlled trials are widely accepted as the most reliable method of determining effectiveness, but most trials have evaluated the effects of a single intervention such as a drug. Recognition is increasing that other, non›pharmacological interventions should also be rigorously evaluated. 1-3 This paper examines the design and execution of research required to address the additional problems resulting from evalua›
Trials, 2014
The Medical Research Councils' framework for complex interventions has been criticized for not including theory-driven approaches to evaluation. Although the framework does include broad guidance on the use of theory, it contains little practical guidance for implementers and there have been calls to develop a more comprehensive approach. A prospective, theory-driven process of intervention design and evaluation is required to develop complex healthcare interventions which are more likely to be effective, sustainable and scalable. We propose a theory-driven approach to the design and evaluation of complex interventions by adapting and integrating a programmatic design and evaluation tool, Theory of Change (ToC), into the MRC framework for complex interventions. We provide a guide to what ToC is, how to construct one, and how to integrate its use into research projects seeking to design, implement and evaluate complex interventions using the MRC framework. We test this approach b...
Journal of Clinical Epidemiology, 2013
Objectives: Although there is increasing interest in the evaluation of complex interventions, there is little guidance on how evidence from complex interventions may be reviewed and synthesized, and the relevance of the plethora of evidence synthesis methods to complexity is unclear. This article aims to explore how different meta-analytical approaches can be used to examine aspects of complexity; describe the contribution of various narrative, tabular, and graphical approaches to synthesis; and give an overview of the potential choice of selected qualitative and mixed-method evidence synthesis approaches. Study Design and Setting: The methodological discussions presented here build on a 2-day workshop held in Montebello, Canada, in January 2012, involving methodological experts from the Campbell and Cochrane Collaborations and from other international review centers (Anderson L, Petticrew M, Chandler J, et al. Introduction: systematic reviews of complex interventions. In press). These systematic review methodologists discussed the broad range of existing methods and considered the relevance of these methods to reviews of complex interventions. Results: The evidence from primary studies of complex interventions may be qualitative or quantitative. There is a wide range of methodological options for reviewing and presenting this evidence. Specific contributions of statistical approaches include the use of metaanalysis, meta-regression, and Bayesian methods, whereas narrative summary approaches provide valuable precursors or alternatives to these. Qualitative and mixed-method approaches include thematic synthesis, framework synthesis, and realist synthesis. A suitable combination of these approaches allows synthesis of evidence for understanding complex interventions. Conclusion: Reviewers need to consider which aspects of complex interventions should be a focus of their review and what types of quantitative and/or qualitative studies they will be including, and this will inform their choice of review methods. These may range from standard meta-analysis through to more complex mixed-method synthesis and synthesis approaches that incorporate theory and/or user's perspectives.