Measuring societal effects of transdisciplinary research projects: Design and application of an evaluation method (original) (raw)
Related papers
Environmental Science & Policy, 2020
Participation of relevant stakeholders, knowledge integration, responsive and emergent design and effective boundary management are four key features of transdisciplinary research (TDR). These features pose significant challenges to both undertaking TDR and evaluating its societal impact. We argue that TDR's context specificity and complexity warrant an evaluation approach that supports the coordinating team in developing these key features. In light of this, this article aims to reconcile two distinct foci of TDR evaluation, namely supporting transdisciplinary capacity building and impact evaluation. We share the results from a combined approach in which the authors acted both as facilitators and evaluators of a TDR project, to conduct an embedded, formative evaluation. Our findings show that the approach allowed for better access to the participants and sensitivity to their perspectives on impact, and for enhanced understanding of complex internal and external project dynamics and how these shaped the project. This resulted in a meaningful assessment of TDR's societal impacts and enabled attributing these to specific process elements. Moreover, the approach supported the coordinating TDR team's capacities for developing key TDR features. Four TDR capacities were identified: building TDR ownership, openness and transparency for integrating divergent TDR needs, purposeful responsiveness to emergent TDR needs and navigating institutional realities and TDR ambitions. The approach presented may serve as stepping stone for the TDR community to further the conversation on (the impact of) inclusive, reflexive and responsive research.
Research Evaluation
There is a call for more transdisciplinary (TD) research, from academia, society, and funding agencies. Consequently, the field of TD research is searching for ways of proving the value and providing evidence to support the effectiveness of such research. The main challenge for evaluating TD research is attribution, that is how to link societal change to the TD research process. However, little attention has been paid to the relationship between the quality of the research process and the effects and impacts that are being evaluated. Building upon earlier attempts at evaluating TD research, this article tests three key aspects of effective sustainability research: its relevance, credibility, and legitimacy. To explore the link between the quality of process and societal effects, we analyze and compare outputs, outcomes, and impact of five TD projects. Overall, our analysis shows that while relevance, credibility, and legitimacy gave important insights regarding the links between process and impacts, they are not adequate for evaluating TD research impact. Process qualities such as practitioner motivation and perceived importance of the project, together with breadth of perspectives, the openness/flexibility of participants, and in-depth exchanges of expertise and knowledge, contributed to producing internally relevant, credible, and legitimate results. However, we also saw a need to develop the relevance, credibility, and legitimacy framework, in relation to the external dynamics of the project process, heterogeneous stakeholder groups, and the credibility of practice-based knowledge, which together with institutional factors and the political context significantly shape the possibility of impact.
Design features for social learning in transformative transdisciplinary research
This article analyses social learning in transdisciplinary research processes by a systematic comparative analysis of 20 completed or nearly completed projects in the field of sustainable development. This article considers the social learning generated by transdisciplinary processes in a broad way. It looks how social learning is embedded in the practical interaction processes between new scientific knowledge, practitioners’ life-world experiences and social experimentation. The analysis finds that three factors in particular play an important role in social learning: the clarification of the normative orientations, the co-construction of the research question and practical problem situation, and the balancing of power asymmetries. While a single criterion may not allow increasing social learning alone, the analysis supports the hypothesis that a combination of these three criteria systematically increases the strength of the social learning generated. Other factors, such as active facilitation modes and the presence of collective interest advocacy organizations, only play a strong role as a condition for generating social learning in some specific types of transdisciplinary research.
Ecological Econonomics, 2020
Open access at https://doi.org/10.1016/j.ecolecon.2020.106751\. Transdisciplinary research is a well-recognised approach to address complex real-world problems. However, the literature on a central aspect of transdisciplinarity, namely stakeholder involvement, largely lacks a reflection on its objectives. In response, we present a framework defining four general rationales for stakeholder involvement: normative, substantive, social-learning, and implementation objectives. We demonstrate the applicability of the framework and analyse how the design and processes of three collaborative research projects dealing with sustainable land management in Southern Africa, Southeast Asia, and South America were affected by motivations to include stakeholders. Our assessment indicates that at the projects' outset, many scientists pursued a normative rationale and saw stakeholder involvement as a burden. In the course of the projects, the substantive objective became more relevant as being closely linked to the core mandate of scientists. The projects also aimed for social learning and implementation processes, which, however, did not remain uncontested among team members. Overall, our study indicates that jointly negotiating, clarifying, communicating, and reflecting the underlying objectives of stakeholder involvement can help developing more effective interaction strategies and clarifying expectations. The conceptual framework can guide a systematic reflective and reflexive practise and support the planning and co-designing of future transdisciplinary research projects.
Sustainability Science, 2016
Transdisciplinary (TD) research is an example of a participatory research approach that has been developed to address the complexity of societal problems through the exchange of knowledge and expertise across diverse groups of societal actors. The concept of knowledge exchange is central to the ability of TD research to produce usable knowledge. There is, however, limited theoretical attention to the processes that enable knowledge exchange, namely learning. In this article we analyze the "transferability" of knowledge generated in TD research settings from a practice-based approach. In this approach, learning and knowing are seen as situated in social practices, in meaning making processes where the involved participants make sense of what they do and why they do it. We describe and analyze three TD projects, and discuss the role of practitioners´ perspectives in the interpretation of the tasks and realization of TD, and in the consequences this has for the organization of the research process and the usability of its results. The analysis shows that while the project teams were given the same task and framework, they did not understand or enact TD in a similar fashion. The three projects created different goals and organizations. They also resulted in different challenges, which could be identified and analyzed by the use of a practice-based approach to learning. In the conclusions, we identify aspects for both practice and research that are important for creating sufficient conditions for learning in TD research processes so that they can better promote contributions to societal change.
Developing and Testing a General Framework for Conducting Transdisciplinary Research
Sustainability
Complex societal problems cannot be resolved without transdisciplinary research (TDR). Currently, there is no focused communication platform or commonly shared research framework for conducting TDR. The current study is a continuation of the exploration of collaborations in multidisciplinary, interdisciplinary, and TDR to identify ideas that could contribute to developing and testing a general framework for conducting TDR. The systematic literature review in this study discovered three main themes (TDR initiation, TDR management, and transdisciplinary knowledge exchange). These themes formed three phases of a general framework for conducting TDR. The novelty of the presented phased general framework for conducting TDR relates to the type of learning and outputs that are required at the end of each related action of all associated stages of the three phases to enable all participants to participate in TDR. The logical sequence of these actions and associated stages and phases were ve...
Health Research Policy and Systems
The potential use, influence and impact of health research is seldom fully realised. This stubborn problem has caused burgeoning global interest in research aiming to address the implementation 'gap' and factors inhibiting the uptake of scientific evidence. Scholars and practitioners have questioned the nature of evidence used and required for healthcare, highlighting the complex ways in which knowledge is formed, shared and modified in practice and policy. This has led to rapid expansion, expertise and innovation in the field of knowledge mobilisation and funding for experimentation into the effectiveness of different knowledge mobilisation models. One approach gaining prominence involves stakeholders (e.g. researchers, practitioners, service users, policy-makers, managers and carers) in the coproduction, and application, of knowledge for practice, policy and research (frequently termed integrated knowledge translation in Canada). Its popularity stems largely from its potential to address dilemmas inherent in the implementation of knowledge generated using more reductionist methods. However, despite increasing recognition, demands for co-produced research to illustrate its worth are becoming pressing while the means to do so remain challenging. This is due not only to the diversity of approaches to co-production and their application, but also to the ways through which different stakeholders conceptualise, measure, reward and use research. While research coproduction can lead to demonstrable benefits such as policy or practice change, it may also have more diffuse and subtle impact on relationships, knowledge sharing, and in engendering culture shifts and research capacity-building. These relatively intangible outcomes are harder to measure and require new emphases and tools. This opinion paper uses six Canadian and United Kingdom case studies to explore the principles and practice of co-production and illustrate how it can influence interactions between research, policy and practice, and benefit diverse stakeholders. In doing so, we identify a continuum of co-production processes. We propose and illustrate the use of a new 'social model of impact' and framework to capture multi-layered and potentially transformative impacts of co-produced research. We make recommendations for future directions in research co-production and impact measurement.
Measuring the effects of transdisciplinary research: the case of a social farming project
Futures, 2016
Social farming (SF) is an innovative concept belonging to a grey zone occupied by agriculture, social, education and health sectors. It involves various private and public actors who work together to co-create and share new collective knowledge. SF initiatives also involve many policies and tools that need to be reframed in order to facilitate the evaluation of these practices. Research in SF includes the active role of researchers in medium-term initiatives, involving a large number of stakeholders. Thus an evaluation of SF practices is crucial in developing and planning future actions. The complex nature of SF has led to the use of a transdisciplinary approach for the evaluation of its initiatives. This paper explains the transdisciplinary process used in a SF project, describes the nature of the collaborative relationships between researchers and others stakeholders, and examines the factors that inhibit and facilitate this collaboration. The paper highlights the important effects the transdisciplinary approach could have on the future of SF, in terms of network building, the co-production of knowledge, and the development of innovative practices.
2020
This paper is based on a keynote address given at the 2019 IIILs 2018 conference in Orlando, Florida on March 14. In that address, I spoke about how Design Science Research could help bridge the rigor-relevance gap in management science, and probably in other fields as well. I showed that by weaving design, testing and iterations of the two processes together in a logical and systematic manner, new actionable knowledge can be created along with new scientific knowledge. In this paper I explore the concept of rigor-relevance from a different approach, namely Transdisciplinary Research. Transdisciplinary Research is a process that involves both academic researchers and individuals from professional practice collaborating on finding a possible solution to a complex problem. Knowledge artifacts from the Transdisciplinary Research process contribute to the body of scientific knowledge while at the same time developing solution concepts that can be used by practitioners. In other words br...