Local Wisdom Matters: Reflections on Evaluation Theory and Practice from South Asia (original) (raw)
Related papers
The Institutionalisation of Evaluation in Asia-Pacific, 2023
valuation is practised in India for seventy years in various sectors of public sphere, and in recent years by the civil society and industries also. Even though there is no Constitutional mandate or national policies or statutes, evaluations are commissioned both by the Central (federal) government and States as and when needed, and in recent years, are generally carried out by independent research institutions. While there are a few courses in evaluation as a part of higher academic courses, no independent and full-time programs are on offer. Most of the evaluation capacity building takes place either through international programmes or short part-time courses in India. Evaluation Community of India (ECOI) set up in 2015 as a body of professional evaluators, along with its constituent EvalYouth India, has been championing the cause of evaluation capacity building through innovative approaches and the adoption of an evaluation policy. Civil society’s appreciation of evaluation has been growing, and so is the media’s reporting of evaluative evidence and of national evaluation policy. The authors provide a vivid picture of the state of institutionalization of evaluation in India from angles of Constitutional and legal mandate, existence of national policies, executive instructions and guidelines on the evaluation practice. They argue that despite the general absence of any national mandates or guidelines, evaluation practice has evolved over the past seven decades in response to national needs. Evaluations are mainly country-led, carried out by independent institutions and employ mixed methods. Development Monitoring and Evaluation Office (DMEO), which is the apex agency for monitoring and evaluation, works towards a strong evaluation ecosystem. Evaluation capacity-building takes place mainly through short-term training, and the higher academic institutions do not offer any formal courses. Evaluation Community of India (ECOI) is the professional association striving to develop evaluation capacities, widen awareness of evidence-based decision process and to promote a national evaluation policy. The authors find evidence of civil society organizations, industry and media increasingly participating in evaluations and reporting evaluation findings.
Valuing Evaluation: Building Capacities for Evaluative Thinking and Learning in Indian States.
ME @70. Strengthening India’s Evidence System for Accelerated Reforms and Inclusive Growth: A Compendium of Essays. NITI Aayog, Development Monitoring and Evaluation Office, Government of India., 2022
In 2022 India celebrates 75 years of its independence. This year also marks 70 years of the establishment of the Programme Evaluation Organization (PEO) by the Government of India. This marks a formal recognition of the monitoring and evaluation function within the government system. India is at a critical juncture-with future development contingent on carefully designed investments that are implemented efficiently. This will yield the intended developmental outcomes and help achieve our national priorities. Monitoring and Evaluation (M&E) is the cornerstone of this paradigm of output-outcome based performance measurement and public expenditure efficiency. Since 2015, the Development Monitoring and Evaluation Office (DMEO) at NITI Aayog has supported rigorous, data-driven, citizen-centric, and outcomes-driven program management and policy-making. DMEO has consistently undertaken initiatives to strengthen M&E systems and capacities necessary for addressing the complexities, diversities, and inequities across the country. DMEO is also partnering with States for facilitating cross-learning across all levels of the government. It has increasingly become evident that M&E is imperative for sustainable development. In this context, DMEO has curated this Compendium of Essays encompassing various themes. These themes address different aspects of strengthening the M&E ecosystem at the national and state levels. They will improve the understanding of the challenges in diverse M&E contexts and exploring the potential of data, technology and innovation in M&E for supporting evidence-based policy making. These essays will enable identifying institutional gaps in M&E systems and sharing good practices for course correction. Apart from having a coherent policy, in the current Volatile, Uncertain, Complex and Ambiguous (VUCA) world, M&E has to be capable of addressing the uncertainties of exogenous shocks. The essay "A Framework for Optimizing Government Monitoring and Evaluation Systems During Crises" articulates, an M&E implementation framework to address the challenges of the VUCA world. The M&E Framework needs to be supported by appropriate methodology and tools. While gold standard evaluations involve carefully calibrated before-and-after research designs, real-world policymaking and exogenous shocks often do not allow for planned evaluations FROM THE EDITOR'S DESK From the Editor's Desk x M&E @70: Strengthening India's Evidence Systems for Accelerated Reforms and Inclusive Growth with phased programme implementation. The Essay titled "Leveraging Longitudinal Data for Evaluation" highlights how monitoring instruments can be pivoted into evaluation instruments with a longitudinal focus and incorporation of a causal lens. While we explore causality, it is imperative to go beyond "causality" to understand the "Why" and "How". The Essay titled "Mixed Methods in Evaluation: Beyond "causality"; to understand the 'Why' and 'How" highlights how mixed methods can strengthen M&E. Use of data not only necessitates availability of data but also a cultural perspective. Supporting the mixed method approach in the Essay, "Building capacities for evaluative thinking and learning in Indian states: centering the role of values and norms" the authors draw on their experience of working with the Government of Meghalaya to center values and norms as a critical precursor to decision-making for evaluation. The Essay highlights the importance of long-term, process-focused, mixed-method, complex evaluations and the capacities that are required to support them. "Capacity Building of National Rural Livelihood Mission Personnel on Evaluations and Data Utilization for Evidence-based Decision Making" further amplifies importance of capacity building, especially for largescale scheme evaluation programs. Culture and capacity are an integral part of successful M&E. "Developing a Culture of Evidence Use: Experiences from Uttar Pradesh" articulates a culture of data use for decision making in the Health Department of Government of Uttar Pradesh supported by the Uttar Pradesh Technical Support Unit (UPTSU). In evidence-based policy making, quality of data is key, and technology can be leveraged for the same. Using a case experience, the Essay titled "Community Monitoring for strengthening responsible participation leveraging ICT" highlights how simple communication technology can be leveraged for unbiased, free and efficient participative monitoring. Moving beyond survey and administrative data; the essay "Big Data: Transforming Evidence based Policy Making" highlights how governance can be improved, made more efficient and effective through the use of Big Data. Development must be inclusive of the marginalized classes as well. "Inclusivity in Evaluations and Equity-Centric Budgeting", attempts to underscore the role of equitycentric evaluations for empowering marginalized groups while maintaining utmost sensitivity to their needs and requirements. One of the important dimensions of inclusivity is accessibility and affordability of food. The Essay, "Road to Zero Hunger: Review of Evidence on India's Food Safety Nets" undertakes a review of available literature on the effect of food safety nets on achieving food security and improving nutrition outcomes for the beneficiaries. The Essay "Enabling Coherence in Monitoring by Breaking Informational Silos" details how India's ability to break information silos can pave the way to a new phase of equitable, efficient, effective and sustainable resource utilization. In "Improving States' Capacities for Responsive Social Protection Systems", the author describes the relevance of building State capacity for designing, implementing, and financing social protection.
Evaluation Policy – The Core of Evaluation Ecosystem
M&E @70: STRENGTHENING INDIA'S EVIDENCE SYSTEMS FOR ACCELERATED REFORMS AND INCLUSIVE GROWTH
Development evaluations strengthen evidence-based decision-making and help the policy makers achieve the SDGs. National evaluation policies provide the necessary institutional frameworks and mechanisms to systematise the evaluation processes and standards that contribute to enhance the demand for evaluations and create scope for building capacities of the development professionals and institutions. Evaluation policies steer, coordinate and systematize evaluations, helping in institutionalizing evaluations and contributing to the development of a strong evaluation ecosystem by bringing all relevant stakeholders to work together. Even though India recognized the value of evaluations as early as 1950, and has been conducting evaluative studies at national and state levels since then, the absence of coherent national evaluation policy renders them ad hoc exercises. The paper elaborates the importance of such a policy and what aspects it should cover, such as to what development interventions need to be evaluated and how often, adherence to national technical and ethical standards, necessary institutional mechanisms for capacity building and professionalization of evaluation through the involvement of professional associations and academic institutions, building partnerships with various institutional stakeholders, allocation of adequate resources, etc. Once a national policy is adopted, the next step would be to encourage various state governments through approaches of cooperative federalism to adopt similar policies and weave them together into a coherent evaluation ecosystem for the country. The paper argues that it will not only lead to optimal utilisation of resources but also systematise the evaluation process and practice.
Appreciative inquiry approach to evaluation practices in South Asia (English version)
2010
Appreciative inquiry approach to evaluation is a recent development in the evaluation realm. Though some evaluators have used this approach independently, it has been used partially to complement other evaluation approaches in many cases. Similar to other evaluation it measures the changes, develops programme logic model, clarifies the evaluation purpose, identifies the stakeholders, determines the evaluation key questions, develops indicators
Strengthening Regional Evaluation Capacity-Building through Local Partnerships
2019
Africa is seeing increasing interest and investment in evaluation as a way to improve governance, accountability and the quality of development policies and programmes. For the past six years, CLEAR-AA has been one of the leading organisations working to build evaluation capacity and strengthen evaluation systems in Africa. We have worked across Ghana, Zambia, Ethiopia, Benin, Uganda and South Africa, driving oversight of parliamentary capacity development and strengthening national evaluation systems. In South Africa alone, CLEAR-AA has carried out two Evaluation Training Authority evaluations and provides ongoing technical assistance in the education and transport sectors. During this time, CLEAR-AA has also developed a flagship training programme, the Development Evaluation Training Programme in Africa and has produced publications, including books, policy briefs and learning notes.
Relevance of cross-cultural understanding for evaluations in development practice
Evaluation Journal of Australasia, 2016
In the evaluation profession, there is shared agreement that cross-cultural understanding is relevant for those working in communities and countries different from their own. This helps inform the selection of appropriate approaches, tools and processes. Perhaps more importantly, the concept of evaluation itself could be considered through a cross-cultural lens; such consideration may reveal a lack of shared understanding between different cultures. The idea of evaluation reflects values about change, development and risk; knowledge and wisdom; access to information and education; the completion of tasks; and, the maintenance of harmonious relationships. These values are not necessarily shared universally. This article draws links between understanding differences in cultural values and the practice of evaluation in international development—beyond the level of culturally appropriate method selection. A widely used model for understanding cultural value differences is applied to the...
The Worldwide Expansion of Evaluation: a World of Possibilities for Rural Development
The growing importance of the evaluation field is exemplified by its increasing use in international organizations and public policies at a regional, national and local level, as well as by the exponential growth of evaluation societies around the world. Focusing on rural development, this expansion is due to the utility of evaluation in providing evidence to help in the decision-making and improve interventions , as well as being a value system that establishes criteria and explicit standards by which to judge these interventions. This paper aims to provide clarity on these aspects through a comprehensive analysis of the evolution and the current situation of the evaluation field. The literature reviewed and our evaluation experiences confirm the evaluation's ability to provide a scientific basis to the decision-making in rural development programmes.
Zeitschrift für Evaluation, 2024
The volume 'The Institutionalisation of Evaluation in Asia-Pacific' , edited by Reinhard Stockmann, Wolfgang Meyer, and Niklas Zierke (2023), is the latest release in a monumental series aimed at assessing the level of institutionalisation of evaluation globally. This comprehensive project, known as Evaluation Globe, has previously covered Europe (Stockman et al., 2020), and the Americas (Stockman et al., 2022). Together, these three volumes provide an updated overview of the state of evaluation in 39 countries and a group of transnational organisations, serving as a valuable resource for scholars, experts, and students in the field of evaluation. Given the global scope of this effort, the analysis comprises countries with diverse historical, political, social, and economic contexts. This third volume on Asia-Pacific is particularly significant as it examines a group of 11 countries that collectively represent nearly half of the world's population (47%). Among these, China and India each have populations exceeding one billion, while Pakistan, Bangladesh, Japan, and the Philippines each have over 100 million inhabitants. Smaller countries such as Korea, Nepal, Australia, Taiwan, and Sri Lanka (the smallest of the group, with 21.8 million people) are also included. Interesting insights arise from observing the state of evaluation in countries characterized by high levels of economic well-being, such as Japan and Australia, recent democracies, such as Nepal, and developing countries. The latter is the case of Bangladesh, which since 2000 has seen a steady average GDP growth of around 6%. The volume also addresses the institutionalization of evaluation in two transnational organizations in Asia: the Asian Development Bank, and UNICEF (East Asia and Pacific Region 2). From a European point of view, this volume moreover offers an extraordinary opportunity to understand the state of evaluation in both demo-1 Politecnico di Milano, Dipartimento di Ingegneria Gestionale 2 Previous volumes examined other organizations, notably the European Union in the first volume and the CLEAR LAC (Centers for Learning on Evaluation and Results-Latin America and the Caribbean), the Inter-American Development Bank Group (IDB Group), and the Independent Evaluation Group (IEG) of the World Bank Group.
History and Status of Development Evaluation: Indian Context
2018
Development evaluation is systematic assessment of community development programs aimed at analysing the level of achievement against set criteria. Evaluation has a history of over a century, while development evaluation started in the aid-era post second world war and picked up with the millennium development goals. Important international agencies involved in it, and shaping the field are United Nations Evaluation Group, Independent Evaluation Group, International Organisation for Cooperation in Evaluation, etc. and these organisations are pushing nations to commit to evaluation and assisting in building evaluation capacity. Professional-Academic bodies, such as American Evaluation Association, are at the forefront of knowledge development in evaluation, including evaluation theories, logic of evaluation and various classifications. India started evaluating planned social programs through the Program Evaluation Office under Planning Commission in the 1950’s and all states also established their own evaluation offices in the early 60’s. After an initial phase of great growth and focus, later evaluation function was slowly degraded and ignored, till the turn of the century when it picked up again with international stimulus. This literature review captures the historical timeline of development evaluation globally, and in India, and discusses the current scenario, as a precursor to building in depth knowledge about the field of evaluation. It lays out the basics and terminologies of evaluation, including classification types, theories, and logic of evaluation, which are currency of the field. Important journals in evaluation are also listed. The review concludes with some future directions of research.