Mixed Methods Research (original) (raw)

What are mixed methods?

Mixed methods research integrates both quantitative and qualitative research methods within a single study or across multiple related studies.

Mixed methods research is particularly useful when a research question requires a multifaceted approach that can simultaneously explore trends in data and the nuances of individual experiences.

Integration, a key concept in mixed methods research, is the intentional combining of quantitative and qualitative research in such a way that they become interdependent and work together to achieve a common research goal.

For example, examining changes in school funding (quantitative) alongside teacher and student narratives (qualitative) about educational quality in a school district can provide a more comprehensive understanding of the relationship between funding and the actual experiences of those within the school system.

Several factors make mixed methods research distinct from conducting separate quantitative and qualitative studies. A few of these defining factors are:

Examples

Here are some examples of how people use mixed methods research in real life:

Mixed methods research is a powerful tool that can be used to answer complex research questions in a way that neither quantitative nor qualitative research can do alone:

  1. Researchers could conduct a study to understand the impact of a new school-based mental health program on student well-being. Qualitative data could be collected through interviews with students and teachers to explore their experiences with the program and identify any barriers or facilitators to implementation. This data could then be used to explain variations in quantitative data on student mental health outcomes, such as changes in depression or anxiety symptoms.
  2. A mixed methods study could be used to investigate the relationship between patient satisfaction and health outcomes following a specific medical procedure, such as surgery. Qualitative data from patient interviews or focus groups could provide insights into the reasons behind varying levels of satisfaction. Researchers could then connect these qualitative findings with quantitative data on post-surgical complications, recovery time, or readmission rates to see if there are correlations or patterns.
  3. Researchers could use a mixed methods approach to examine the effectiveness of a teacher training program on student academic achievement in a particular subject, like mathematics. Quantitative data on student test scores before and after the training could be combined with qualitative data from teacher interviews and classroom observations. This integration of data could help determine if improvements in student performance are related to changes in teaching practices resulting from the training.

When to use mixed methods research

Researchers should clearly articulate their reasons for using a mixed methods research design. This rationale helps reviewers and other researchers understand why this design is the most appropriate for addressing the research questions.

  1. Triangulation: When researchers want to double-check their findings, they can use mixed methods. This involves comparing results from quantitative and qualitative strands to confirm findings and enhance the validity of the study.
  2. Complementarity: When researchers need to explain confusing results, they can use mixed methods to get a clearer picture. This aims to elaborate on or clarify the findings of one strand with the results of the other strand. This approach can be particularly useful when quantitative findings are statistically significant but lack practical meaning or when qualitative findings need further clarification.
  3. Development: When researchers need to design a good survey or test, they can use mixed methods. This rationale involves using the results from one method to help develop or inform the other method. This can include using qualitative findings to develop and validate an instrument for the quantitative strand, or using quantitative findings to identify specific participants or groups for the qualitative strand.
  4. Initiation: This is when researchers want to explore differences in findings from different methods. By comparing different perspectives, they can develop new interpretations of what they’re studying. It leverages the strengths of each approach to clarify, contextualize, and enrich the overall findings, rather than focusing on resolving contradictions.
  5. Expansion: When researchers want to learn more about something, they can use mixed methods. This rationale seeks to expand the breadth and range of a study by using mixed methods to investigate different components of a research question or to study different research questions within the same study.

Mixed methods research designs provide researchers with a structured approach to combining qualitative and qualitative data. Creswell and Plano Clark (2018) identify three core mixed methods designs:

mixed methods design

Convergent Parallel Design

Convergent parallel design involves simultaneously collecting quantitative and qualitative data, analyzing these datasets separately, and then merging the results for interpretation.

For example, this design could be used to understand domestic violence and abuse among gay and bisexual men by matching data from semi-structured interviews with survey data.

Here’s how it works:

Explanatory Sequential Design

An explanatory sequential design is used in research when you want to use one type of data to help explain the findings of another type.

For example, this design can be used to learn more about why students stayed enrolled in online education programs. The researchers could first look at survey data and then followed up with interviews to get a deeper understanding of the factors involved.

Here’s how it works:

Exploratory Sequential Design

An exploratory sequential design is a type of research that involves two phases of data collection and analysis, with the qualitative phase coming first.

The exploratory sequential design is most effective when researchers have a clear rationale for using a mixed methods approach and when the research questions lend themselves to both qualitative and quantitative exploration.

Here’s how it works:

An example:

Researchers were interested in understanding the factors that influence brain donation decisions among older minorities.

They started by conducting interviews with individuals from this population to explore their thoughts, beliefs, and experiences related to brain donation.

The themes and insights from these interviews were then used to develop a survey instrument to measure the factors identified as potentially influencing donation decisions.

Embedded (or nested) Design

Embedded or Nested Designs in Health Sciences

Embedded designs use quantitative and qualitative approaches together, with one embedded in the other, to yield greater insight. This might look like supplemental qualitative data embedded in a larger quantitative study design, such as an experimental trial. These types of designs may be a variation of convergent or sequential designs.

Embedded design is a method for linking qualitative and quantitative data collection and analysis at multiple points, particularly useful in intervention research.

Qualitative data may be used prior to the intervention to inform strategies to best recruit individuals or to develop the intervention, during the experiment to examine the process being experienced by participants, or after the experiment to follow up and better understand the quantitative outcomes.

Embedding involves combining connecting, building, or merging data.

Here’s how it works:

Integration

Integration is the intentional combination of quantitative and qualitative research, resulting in a synergistic and interdependent relationship between the two approaches. It is a multifaceted concept that manifests across various stages of the research process, from design to reporting.

Integration Trilogy: Design, Methods, and Interpretation & Reporting

Integration in mixed methods research operates at three distinct but interconnected levels, often referred to as the integration trilogy:

  1. Integration at the Design Level: This involves selecting a mixed methods design that outlines the sequence, priority, and purpose of integrating the quantitative and qualitative strands. Common designs include exploratory sequential, explanatory sequential, and convergent designs.
  2. Integration at the Methods Level: Integration strategies are the methods employed to combine quantitative and qualitative research elements. These strategies are not mutually exclusive and can be used in various combinations to achieve a holistic understanding of the research problem.
    • Merging: This involves analyzing data from both strands and assessing whether the findings converge, diverge, or expand upon one another. This can involve comparing themes with statistical data, exploring the quantitative profile of qualitative themes, or transforming qualitative data into quantitative data for statistical analysis. Joint displays, such as tables or matrices, can visually represent merged data.
    • Connecting: This involves using one type of data to inform the sampling frame of the other. For instance, quantitative data can be used to identify a subset of participants for qualitative interviews. This strategy is particularly useful in sequential designs, where one strand precedes the other.
    • Building: This approach uses one database (qualitative or quantitative) to inform the data collection approach of the other. This could involve developing a quantitative instrument based on themes identified through qualitative research or refining a qualitative interview guide based on quantitative findings.
  3. Integration at the Interpretation and Reporting Level: This involves combining and presenting the findings in a way that highlights the synergistic insights gained from integrating the two strands. This can be achieved through narrative techniques like weaving and contiguous approaches or through data transformation and joint displays.
    • Integrating through narrative involves using a single report, or a series of reports, to describe the quantitative and qualitative results. The Survival After Acute Myocardial Infarction (SAMI) study provides an example of a narrative integration.
    • Integrating through data transformation involves converting one type of data (qualitative or quantitative) into the other type. For example, qualitative data may be converted into numerical counts, which are then integrated with other numerical data for analysis.
    • Integrating through joint displays involves using visual elements, such as tables, matrices, or figures, to present the integrated data. An example of a study using joint displays is a mixed-methods evaluation that explored ethical aspects of adaptive clinical trial designs.

Narrative Integration

Integrating through narrative in mixed methods research involves describing qualitative and quantitative findings within a single report or a series of reports. The manner in which these findings are presented can take on three distinct approaches: weaving, contiguous, or staged.

Data Transformation

Mixed methods analysis encompasses the entire process of analyzing and interpreting both quantitative and qualitative data within a single study or a program of research.

This involves selecting appropriate analytic techniques for each strand and implementing integration strategies to merge, connect, or build upon the findings.

Mixed methods data transformationinvolves converting data from one form to another. This can involve:

Data transformation facilitates merging data and conducting analyses that cut across the quantitative-qualitative divide.

By adhering to these principles and employing these strategies, researchers can leverage the strengths of mixed methods research to address complex research questions and generate rich, insightful, and impactful findings.

Joint Displays: Visual Aid for Integration

Joint displays are visual representations, such as tables, matrices, figures, or graphs, that bring together quantitative and qualitative data to facilitate interpretation and draw new insights beyond what each strand could achieve independently.

They are particularly helpful in merging data, comparing results, and representing meta-inferences, the novel insights that emerge from integrating the two strands.

Types of Joint Displays:

Assessment of Fit and Integrated Interpretation

Assessment of fit of integration involves evaluating the coherence and consistency between the quantitative and qualitative findings. This assessment can reveal three potential outcomes:

Integrated interpretation involves synthesizing the quantitative and qualitative findings to develop a holistic understanding of the research problem, acknowledging both points of convergence and divergence.

This process requires integrated thinking, a mindset that values both approaches equally and seeks a synergistic understanding that transcends the limitations of either method in isolation.

Steps for conducting mixed methods research

Remember that mixed methods research is an iterative process. Researchers should remain flexible and adaptable throughout the study, adjusting plans as needed based on emerging findings or unexpected challenges.

The dynamic interplay between quantitative and qualitative approaches is a hallmark of mixed methods research, and embracing this fluidity contributes to the richness and depth of the findings.

Step1: Formulating the Research Problem

Step 2: Designing the Study

Step3: Implementing the Study

Step 4: Integrating and Interpreting Findings

Key Considerations for Integration

Mixed Methods Quality and Publication

The quality of a mixed methods study hinges on several factors:

Mixed methods publications often employ a weaving approach, integrating findings throughout the manuscript, or a contiguous approach, presenting quantitative and qualitative results in separate sections but linking them through cross-referencing and discussion.

A staged approach involves publishing multiple papers, each focusing on a specific aspect of the mixed methods study.

Benefits of mixed methods research

  1. Enhanced Understanding: Mixed methods research provides a more complete comprehension of research problems by combining quantitative and qualitative approaches. This approach is especially valuable in positive psychology, where constructs often have reciprocal relationships, and in understanding complex processes and systems in health and healthcare. For instance, in studying adolescent bullying, a mixed methods approach allows researchers to explore risk factors, PTSD symptoms, and individual experiences.
  2. Increased Validity: Using multiple methods can strengthen the validity of findings by allowing triangulation, where qualitative and quantitative data are compared to corroborate results. For example, a study on domestic violence among gay and bisexual men used a convergent design, matching semistructured interviews with survey data to enhance the validity of their findings.
  3. Complementarity: Mixed methods research allows researchers to examine different facets of a research question using the strengths of each approach. Quantitative methods can identify trends and relationships, while qualitative methods provide nuanced insights. This approach is beneficial when a single method cannot fully capture the complexity of a phenomenon, such as language learning motivation.
  4. Development and Refinement: Researchers can leverage mixed methods research to develop and refine research instruments and interventions. For instance, qualitative data can inform the creation of quantitative surveys, ensuring they are culturally relevant and address specific research questions.
  5. Explanation of Findings: Qualitative data can be particularly useful in explaining and contextualizing quantitative results. This allows researchers to move beyond statistical associations and gain a deeper understanding of the underlying mechanisms and individual experiences.
  6. Methodological Innovation: Mixed methods research encourages methodological innovation by blending different research traditions. It promotes flexibility and allows researchers to adapt their approaches to best suit their research questions. This is especially valuable for exploring novel research areas or addressing complex social issues.

Disadvantages of mixed methods research

  1. Labor Intensity: Mixed methods research demands significant time, resources, and effort compared to single-method studies. The integration of qualitative and quantitative approaches necessitates expertise in designing and implementing both phases, potentially posing challenges for researchers with a predominant quantitative or qualitative orientation.
  2. Expertise Requirements: Conducting rigorous mixed methods research requires researchers to have expertise in both quantitative and qualitative methodologies. This can be challenging, as researchers often specialize in one approach. The lack of expertise in either strand can compromise the rigor of the study and lead to methodological concerns, particularly in the qualitative strand.
  3. Potential for Bias: The inherent differences between quantitative and qualitative data can make integration challenging. Ensuring that data transformations are defensible and addressing potential biases between methods is crucial for drawing valid inferences.
  4. Integration Challenges: Achieving meaningful integration of quantitative and qualitative data can be difficult. The lack of clear guidelines and the potential for irreconcilable data sources can hinder the synergistic potential of mixed methods research, sometimes leading to separate publications of quantitative and qualitative results instead of a unified, integrated analysis.
  5. Limited Consensus on Terminology and Quality Assessment: The field of mixed methods research lacks a universally agreed-upon terminology, leading to ambiguity and challenges in comparing and evaluating studies. The absence of standardized quality assessment criteria further complicates the evaluation of rigor and trustworthiness in mixed methods research.
  6. Power Imbalances in Data Integration: Integrating data from samples with different power dynamics, such as those at different levels of a bureaucracy, can raise concerns about the validity of triangulation and the interpretation of findings. The potential for power imbalances to influence the identification of differences or paradoxes necessitates careful consideration during data integration.

References

Bazeley, P. (2018). Integrating analyses in mixed methods esearch. SAGE.

Brown, K. M., Elliott, S. J., Leatherdale, S. T., & Robertson-Wilson, J. (2015). Searching for rigour in the reporting of mixed methods population health research: a methodological review. Health Education Research, 30(6), 811-839.

Caracelli, V. J., & Greene, J. C. (1997). Crafting mixed-method evaluation designs. New directions for evaluation, 74, 19-32.

Collins, K. M., Onwuegbuzie, A. J., & Sutton, I. L. (2006). A model incorporating the rationale and purpose for conducting mixed-methods research in special education and beyond. Learning disabilities: a contemporary journal, 4(1), 67-100.

Creamer, E. G. (2018). Striving for methodological integrity in mixed methods research: The difference between mixed methods and mixed‐up methods. Journal of Engineering Education, 107(4), 526-530.

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE.

Dellinger, A. B., & Leech, N. L. (2007). Toward a unified validation framework in mixed methods research. Journal of mixed methods research, 1(4), 309-332.

Fàbregues, S., & Molina-Azorín, J. F. (2017). Addressing quality in mixed methods research: A review and recommendations for a future agenda. Quality & Quantity, 51, 2847-2863.

Fetters, M. D. (2019). The mixed methods research workbook: Activities for designing, implementing, and publishing projects (Vol. 7). Sage Publications.

Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—principles and practices. Health services research, 48(6pt2), 2134-2156.

Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. The Annals of Family Medicine, 13(6), 554-561.

Hesse-Biber, S., & Johnson, R. B. (2013). Coming at things differently: Future directions of possible engagement with mixed methods research. Journal of Mixed Methods Research, 7(2), 103-109.

Neto, D. D., Baptista, T. M., & Dent‐Brown, K. (2015). Development and validation of a system of assimilation indices: A mixed method approach to understand change in psychotherapy. British Journal of Clinical Psychology, 54(2), 147-162.

O’cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of mixed methods studies in health services research. Journal of health services research & policy, 13(2), 92-98.

Palinkas, L. A., Mendon, S. J., & Hamilton, A. B. (2019). Innovations in mixed methods evaluations. Annual review of public health, 40, 423-442.

Shannon-Baker, P. (2016). Making paradigms meaningful in mixed methods research. Journal of mixed methods research, 10(4), 319-334.

Shim, M., Johnson, B., Bradt, J., & Gasson, S. (2021). A mixed methods–grounded theory design for producing more refined theoretical models. Journal of Mixed Methods Research, 15(1), 61-86.

Examples

Bacchus, L. J., Buller, A. M., Ferrari, G., Brzank, P., & Feder, G. (2018). “It’s always good to ask”:
A mixed methods study on the perceived role of sexual health practitioners asking gay and bisexual men about experiences of domestic violence and abuse. Journal of Mixed Methods Research, 12(2), 221–243.

Campbell, R., Fehler-Cabral, G., Bybee, D., & Shaw, J. (2017). Forgotten evidence: A mixed methods study of why sexual assault kits (SAKs) are not submitted for DNA forensic testing. Law and human behavior, 41(5), 454.

Clark, R. S., & Plano Clark, V. L. (2019). Grit within the context of career success: A mixed methods study. International Journal of Applied Positive Psychology, 4(3), 91-111.

Clark, R. S., & Stubbeman, B. L. (2021). “I had hope. I loved this city once.”: A mixed methods study of hope within the context of poverty. Journal of Community Psychology, 49(5), 1044-1062.

O’Keeffe, S., Martin, P., Target, M., & Midgley, N. (2019). ‘I just stopped going’: A mixed methods investigation into types of therapy dropout in adolescents with depression. Frontiers in psychology, 10, 423542.

Roques, M., Laimou, D., Camps, F. D., Mazoyer, A. V., & El Husseini, M. (2020). Using a mixed-methods approach to analyze traumatic experiences and factors of vulnerability among adolescent victims of bullying. Frontiers in psychiatry, 10, 890.

Roysircar, G., Thompson, A., & Geisinger, K. F. (2019). Trauma coping of mothers and children among poor people in Haiti: Mixed methods study of community-level research. American Psychologist, 74(9), 1189.

Journals

Download