Importance of qualitative methods in Social Program Evaluation (original) (raw)
Related papers
BMC Public Health
Background: A multi pronged community based strategy, known as National Rural Health Mission (NRHM), was implemented from 2005-06 to 2012-13 in India to curtail maternal and child health (MCH) disparities between poor and rich, rural and urban areas, and boys and girls,. This study aimed to determine the degree to which MCH plans of NRHM implemented, and resulted in improving the MCH outcomes and reducing the inequalities. Methods: An explanatory sequential mixed methods study was conducted, first to assess the degree of implementation of MCH plans by estimating the budget utilization rates of each MCH plan, and the effectiveness of these plans by comparing demographic health surveys data conducted post (2012-13), during (2007-08) and pre-(2002-04) NRHM implementation period, in the quantitative study. Then, perceptions and beliefs of stakeholders regarding extent and effectiveness of NRHM in Haryana were explored in the qualitative study during 2013. A logistic regression analysis was done for quantitative data, and inductive applied thematic analysis for qualitative data. The findings of the quantitative and qualitative parts of study were mixed at the interpretation level. Results: The MCH plans, like free ambulance service, availability of free drugs and logistics, accredited social health activists were fully implemented according to the budget spent on implementing these activities in Haryana. This was also validated by qualitative study. Availability of free medicines and treatment in the public health facilities had benefitted the poor patients the most. Accredited Social Health Activists scheme was also the most appreciated scheme that had increased the institutional delivery rates. There was acute shortage of human resources in-spite of full utilization of funds allocated for this plan. The results of the qualitative study validated the findings of quantitative study of significant (p < 0.05) improvement in MCH indicators and reduction in MCH disparities between higher and lower socioeconomic groups, and rural and urban areas. Conclusions: MCH plans of NRHM might have succeeded in improving the MCH outcomes and reducing the geographical and socioeconomic MCH inequalities by successfully implementing the schemes like accredited social health activists, free ambulance services, free treatment and medicines in hospitals for the poor and in rural areas.
Integrating qualitative and quantitative approaches in program evaluation
The impact of economic policies on …
8 Integrating Qualitative and Quantitative Approaches in Program Evaluation Vijayendra Rao and Michael Woolcock This chapter outlines some of the ways and means by which inte- grating qualitative and quantitative approaches in development research and program evaluation ...
Health organizations around the globe regularly make evidence-based decisions for effective health programming. Qualitative evaluation fulfills an important role in rigorous evaluation of programs. The strength of qualitative evaluation is its ability to provide valuable insight into complex issues, which quantitative methods may not provide. Qualitative data sources can answer the “why” behind program successes or challenges. Additionally, qualitative data illuminate the uniquely human side of health programming and bring to light important contextual factors, such as culture, gender, or societal norms. Qualitative evaluation may be used to complement quantitative data, answer a question not accessible quantitatively, or provide a cost-effective data source when one would not otherwise be available. This syllabus covers a training that is meant to assist health professionals in using qualitative evaluation skills in sound and rigorous evaluation of their program. The sessions go beyond basic concepts to explore important considerations of qualitative methods in the context of rigorous evaluation. Through session content and participatory exercises, participants will gain basic skills in rigorous qualitative data collection, analysis, and use. This syllabus provides an overview of the ten-day (8.5 working days) training workshop, including presentations, facilitator and participant guides, practical sessions, case studies, and sample agendas. A complete collection of resources for MEASURE Evaluation's Qualitative Methods in Evaluation of Public Health Programs Curriculum can be found here. Filed under: Data Demand and Use, Gender, Curriculum, Data Quality Review, Data Quality, Evaluation, Data use, Qualitative Evaluation, Gender Norms
2020
Background: Maternal health is a subject of serious debate today. Maternal and infant mortality has become a major issue in many rural parts of the country. The Government of India has launched Janani Suraksha Yojana (JSY) under NRHM progrmame in 2005. This programme is intended in reducing maternal and neo-natal mortality by increasing institutional delivery among the poor pregnant women including postpartum proving cash and medical benefits. Objective: The major objective is to assess the knowledge, attitude, response and the extent of availability of JSY services and their utilization by the beneficiaries in the selected districts in Karnataka, Tamil nadu and Andra Pradesh (undivided) states of south India. Methodology: This study has done using mixed methods in three South Indian states using both qualitative and quantitative technique. Conclusion: This schema has played a key role in reducing the rate of maternal mortality and morbidity in the rural areas and also it concludes ...
International Journal of Qualitative Studies on Health and Well-Being, 2014
This short essay aims at commenting on the origin, development, rationale, and main characteristics of qualitative evaluation (QE), emphasizing the value of this methodological tool to evaluate health programs and services. During the past decades, different approaches have come to light proposing complementary alternatives to appraise the performance of public health programs, mainly focusing on the implementation process involved rather than on measuring the impact of such actions. QE is an alternative tool that can be used to illustrate and understand the process faced when executing health programs. It can also lead to useful suggestions to modify its implementation from the stakeholders' perspectives, as it uses a qualitative approach that considers participants as reflective subjects, generators of meanings. This implies that beneficiaries become involved in an active manner in the evaluated phenomena with the aim of improving the health programs or services that they receive. With this work we want to encourage evaluators in the field of public health to consider the use of QE as a complementary tool for program evaluation to be able to identify areas of opportunity to improve programs' implementation processes from the perspective of intended beneficiaries.
History and Status of Development Evaluation: Indian Context
2018
Development evaluation is systematic assessment of community development programs aimed at analysing the level of achievement against set criteria. Evaluation has a history of over a century, while development evaluation started in the aid-era post second world war and picked up with the millennium development goals. Important international agencies involved in it, and shaping the field are United Nations Evaluation Group, Independent Evaluation Group, International Organisation for Cooperation in Evaluation, etc. and these organisations are pushing nations to commit to evaluation and assisting in building evaluation capacity. Professional-Academic bodies, such as American Evaluation Association, are at the forefront of knowledge development in evaluation, including evaluation theories, logic of evaluation and various classifications. India started evaluating planned social programs through the Program Evaluation Office under Planning Commission in the 1950’s and all states also established their own evaluation offices in the early 60’s. After an initial phase of great growth and focus, later evaluation function was slowly degraded and ignored, till the turn of the century when it picked up again with international stimulus. This literature review captures the historical timeline of development evaluation globally, and in India, and discusses the current scenario, as a precursor to building in depth knowledge about the field of evaluation. It lays out the basics and terminologies of evaluation, including classification types, theories, and logic of evaluation, which are currency of the field. Important journals in evaluation are also listed. The review concludes with some future directions of research.
Qualitative Analysis For Improving Healthcare Systems
Science Direct, 2018
Addressing health and health inequities in underserved communities has been a missing part of puzzle in Big Data Science.Every individuals should have the equal opportunity to reach their full potential for health, but reality is far behind this. Because of the reason that so many individuals lack the opportunity for their overall well-being, the expulsion of health disparity has been emerged as a major worldwide public health objective. This project focuses on the analysis of the reasons for the prevalence of health disparity in the rural communities in India and thus preventing those at low cost and high speed.
Valuing Evaluation: Building Capacities for Evaluative Thinking and Learning in Indian States.
ME @70. Strengthening India’s Evidence System for Accelerated Reforms and Inclusive Growth: A Compendium of Essays. NITI Aayog, Development Monitoring and Evaluation Office, Government of India., 2022
In 2022 India celebrates 75 years of its independence. This year also marks 70 years of the establishment of the Programme Evaluation Organization (PEO) by the Government of India. This marks a formal recognition of the monitoring and evaluation function within the government system. India is at a critical juncture-with future development contingent on carefully designed investments that are implemented efficiently. This will yield the intended developmental outcomes and help achieve our national priorities. Monitoring and Evaluation (M&E) is the cornerstone of this paradigm of output-outcome based performance measurement and public expenditure efficiency. Since 2015, the Development Monitoring and Evaluation Office (DMEO) at NITI Aayog has supported rigorous, data-driven, citizen-centric, and outcomes-driven program management and policy-making. DMEO has consistently undertaken initiatives to strengthen M&E systems and capacities necessary for addressing the complexities, diversities, and inequities across the country. DMEO is also partnering with States for facilitating cross-learning across all levels of the government. It has increasingly become evident that M&E is imperative for sustainable development. In this context, DMEO has curated this Compendium of Essays encompassing various themes. These themes address different aspects of strengthening the M&E ecosystem at the national and state levels. They will improve the understanding of the challenges in diverse M&E contexts and exploring the potential of data, technology and innovation in M&E for supporting evidence-based policy making. These essays will enable identifying institutional gaps in M&E systems and sharing good practices for course correction. Apart from having a coherent policy, in the current Volatile, Uncertain, Complex and Ambiguous (VUCA) world, M&E has to be capable of addressing the uncertainties of exogenous shocks. The essay "A Framework for Optimizing Government Monitoring and Evaluation Systems During Crises" articulates, an M&E implementation framework to address the challenges of the VUCA world. The M&E Framework needs to be supported by appropriate methodology and tools. While gold standard evaluations involve carefully calibrated before-and-after research designs, real-world policymaking and exogenous shocks often do not allow for planned evaluations FROM THE EDITOR'S DESK From the Editor's Desk x M&E @70: Strengthening India's Evidence Systems for Accelerated Reforms and Inclusive Growth with phased programme implementation. The Essay titled "Leveraging Longitudinal Data for Evaluation" highlights how monitoring instruments can be pivoted into evaluation instruments with a longitudinal focus and incorporation of a causal lens. While we explore causality, it is imperative to go beyond "causality" to understand the "Why" and "How". The Essay titled "Mixed Methods in Evaluation: Beyond "causality"; to understand the 'Why' and 'How" highlights how mixed methods can strengthen M&E. Use of data not only necessitates availability of data but also a cultural perspective. Supporting the mixed method approach in the Essay, "Building capacities for evaluative thinking and learning in Indian states: centering the role of values and norms" the authors draw on their experience of working with the Government of Meghalaya to center values and norms as a critical precursor to decision-making for evaluation. The Essay highlights the importance of long-term, process-focused, mixed-method, complex evaluations and the capacities that are required to support them. "Capacity Building of National Rural Livelihood Mission Personnel on Evaluations and Data Utilization for Evidence-based Decision Making" further amplifies importance of capacity building, especially for largescale scheme evaluation programs. Culture and capacity are an integral part of successful M&E. "Developing a Culture of Evidence Use: Experiences from Uttar Pradesh" articulates a culture of data use for decision making in the Health Department of Government of Uttar Pradesh supported by the Uttar Pradesh Technical Support Unit (UPTSU). In evidence-based policy making, quality of data is key, and technology can be leveraged for the same. Using a case experience, the Essay titled "Community Monitoring for strengthening responsible participation leveraging ICT" highlights how simple communication technology can be leveraged for unbiased, free and efficient participative monitoring. Moving beyond survey and administrative data; the essay "Big Data: Transforming Evidence based Policy Making" highlights how governance can be improved, made more efficient and effective through the use of Big Data. Development must be inclusive of the marginalized classes as well. "Inclusivity in Evaluations and Equity-Centric Budgeting", attempts to underscore the role of equitycentric evaluations for empowering marginalized groups while maintaining utmost sensitivity to their needs and requirements. One of the important dimensions of inclusivity is accessibility and affordability of food. The Essay, "Road to Zero Hunger: Review of Evidence on India's Food Safety Nets" undertakes a review of available literature on the effect of food safety nets on achieving food security and improving nutrition outcomes for the beneficiaries. The Essay "Enabling Coherence in Monitoring by Breaking Informational Silos" details how India's ability to break information silos can pave the way to a new phase of equitable, efficient, effective and sustainable resource utilization. In "Improving States' Capacities for Responsive Social Protection Systems", the author describes the relevance of building State capacity for designing, implementing, and financing social protection.
A STUDY DESING USING QUALITATIVE METHODS FOR PROGRAM EVALUATION
To determine how a program is put into practice, evaluators are to carry out fieldwork and collect detailed data about to what extent the program is implemented in the field, to what extent projected implementations are reflected in the field, what sort of teaching/learning process the program creates when implemented within the framework of environment, time and other opportunities. It is considered that the employment of qualitative methods, which regard the researcher as a part of the problem under investigation and as a participant observant and which seek to collect rich data and understanding and explaining the problem within the framework of the causal relations leading to the problem using such scientific data collection methods as interview, observation, document analysis, in program evaluation process, especially for the evaluation of input, process and product dimensions is necessary for the program development process. The aim of this study is to reveal within the current theoretical framework, why and how qualitative research methods are to be used in a scholarly study for program evaluation, at what points qualitative research methods and program evaluation coincide, which major problems are to be addressed to provide insights to researchers in a program evaluation study employing qualitative methods, which scientific data collection methods are to be used to reach these answers and how data can be analyzed.