How to Assess Quality in Mixed Methods

Why Standard Quality Criteria Are Insufficient

Quantitative research has well-established criteria for quality: internal validity, external validity, reliability, and objectivity. Qualitative research has its own parallel criteria: credibility, transferability, dependability, and confirmability. Mixed methods research must meet both sets of standards for each respective strand, but it also requires an additional layer of quality assessment that addresses the integration itself.

A mixed methods study can have a perfectly executed survey and impeccably conducted interviews and still fall short if the integration between them is weak, inconsistent, or absent. This means that evaluating mixed methods quality involves at least three assessments: the quantitative strand's rigor, the qualitative strand's trustworthiness, and the integrity of the integration process.

Researchers and reviewers who apply only single-method criteria to mixed methods work risk missing fundamental quality issues. A study that meets all quantitative benchmarks but treats its qualitative component as decorative fails the mixed methods standard, regardless of how strong the statistics are.

Legitimation Types in Mixed Methods

The concept of legitimation was developed specifically for mixed methods research to address quality concerns that arise from combining paradigms. Multiple legitimation types have been proposed. Sample integration legitimation asks whether the sampling strategies for each strand are appropriate both individually and in relation to each other. Weakness minimization legitimation asks whether the strengths of one strand compensate for the weaknesses of the other.

Inside-outside legitimation examines whether the study adequately represents both the insider perspective captured by qualitative methods and the outsider perspective captured by quantitative methods. Sequential legitimation assesses whether the transition between phases in sequential designs is logically sound and well-documented.

These legitimation types function as a checklist that researchers can use during both the design and the evaluation stages. By systematically considering each type, investigators can identify potential quality threats before they compromise the study and can argue persuasively in their manuscripts that those threats have been addressed.

Reporting Standards for Mixed Methods Studies

Several professional organizations have published reporting guidelines specifically for mixed methods research. These guidelines specify what information should be included in each section of a manuscript: the rationale for using mixed methods, the design chosen and why, the integration strategy, the quality criteria applied, and the meta-inferences drawn from the integrated findings.

Adhering to established reporting standards serves multiple purposes. It helps authors structure their manuscripts in a way that reviewers expect, increasing the likelihood of a favorable review. It ensures that readers have enough information to evaluate the study's quality and replicate its procedures. And it raises the overall standard of mixed methods publications, which benefits the field as a whole.

Students preparing dissertations should familiarize themselves with these guidelines early and use them as a framework for organizing their methodology and results chapters. Following reporting standards from the proposal stage reduces the revisions needed later and signals to committee members that the student has a mature understanding of mixed methods scholarship.

Practical Strategies for Enhancing Quality

Beyond meeting formal criteria, several practical strategies can enhance the quality of a mixed methods study. Peer debriefing, where colleagues review your integration logic and challenge your interpretations, provides an external perspective that can catch blind spots. An audit trail that documents every decision from design through analysis makes the study transparent and replicable.

Member checking, inviting participants to review and comment on the qualitative findings, strengthens the credibility of that strand and by extension the integration. Pilot testing both instruments and the integration procedures on a small sample before the full study can reveal logistical and analytical problems that are much easier to fix early.

Finally, reflexivity, the practice of examining and documenting your own assumptions, biases, and analytical choices, is essential in mixed methods work. Because the researcher is making interpretive judgments about how two different data types relate, transparency about those judgments is critical for maintaining credibility. A reflexive journal kept throughout the study provides evidence of thoughtful, self-aware analytical practice.