Advanced Quasi-Experimental Designs: RD, DiD & Propensity Scores Explained

Regression Discontinuity: Exploiting Cutoff Points

Regression discontinuity designs take advantage of situations where assignment to an intervention is determined by whether individuals fall above or below a specific threshold. In healthcare, this might involve a clinical score that triggers eligibility for a treatment program, an age cutoff for screening eligibility, or a poverty threshold for insurance subsidies. Individuals just above and just below the cutoff are presumed to be very similar, so any jump in outcomes at the threshold can be attributed to the intervention.

The logic is straightforward: if the only thing that changes abruptly at the cutoff is exposure to the intervention, then differences in outcomes right at that boundary reflect the intervention's causal impact. The farther participants are from the cutoff, the less comparable they become, so the analysis focuses on a narrow band around the threshold.

Researchers must verify that the cutoff was not manipulated—participants should not be able to position themselves strategically on one side. Graphical displays of the data around the cutoff, along with formal tests for sorting, help establish the design's credibility. When properly executed, regression discontinuity offers some of the strongest causal evidence available outside of randomized experiments.

Difference-in-Differences for Policy Evaluation

Difference-in-differences is a widely used method for evaluating the impact of policies or programs that are implemented in some areas or institutions but not others. The approach compares the change in outcomes over time between the group affected by the policy and an unaffected comparison group. By differencing out both pre-existing differences between groups and common time trends, the method isolates the effect attributable to the intervention.

A classic healthcare application involves examining the impact of Medicaid expansion on emergency department utilization. States that expanded coverage serve as the treatment group, while non-expanding states serve as the comparison. The researcher measures utilization before and after expansion in both sets of states, and the difference in the changes provides the estimated policy effect.

The critical assumption underlying this method is the parallel trends assumption: absent the intervention, both groups would have followed similar trajectories over time. Researchers typically display pre-intervention trends graphically to demonstrate that the groups were indeed on comparable paths before the policy took effect. Violations of this assumption weaken the causal interpretation considerably, so careful diagnostic work is essential.

Propensity Score Methods for Balancing Groups

When randomization is unavailable, treated and untreated groups often differ systematically on characteristics that also affect outcomes. Propensity score methods address this problem by modeling the probability that each individual receives the intervention based on observed covariates, then using that probability to create more balanced comparisons.

The propensity score itself is generated through logistic regression or machine learning algorithms that predict treatment assignment from baseline variables such as age, comorbidities, socioeconomic status, and prior healthcare utilization. Once scores are calculated, researchers can match treated individuals with untreated individuals who have similar scores, stratify the sample into propensity score quintiles, or weight observations inversely by their probability of treatment.

Each technique has trade-offs. Matching is intuitive but may discard unmatched individuals, reducing sample size. Inverse probability weighting retains the full sample but can produce extreme weights when some individuals have very high or very low probabilities. Students should understand that propensity score methods only balance groups on measured variables; unmeasured confounders remain a concern, and sensitivity analyses are needed to assess their potential influence.

Choosing Among Advanced Quasi-Experimental Approaches

Selecting the appropriate method depends on the research context. Regression discontinuity requires a clear assignment cutoff, which limits its applicability but provides strong internal validity when the conditions are met. Difference-in-differences is best suited to natural experiments where a policy change affects some groups but not others, provided the parallel trends assumption is plausible. Propensity score methods are the most flexible, applicable whenever sufficient baseline data exist to model treatment assignment.

In practice, researchers often combine methods to strengthen their conclusions. A study might use difference-in-differences as the primary analysis and propensity score weighting to adjust for baseline imbalances between treatment and comparison groups. Presenting multiple analytic strategies that converge on the same finding dramatically increases confidence in the results.

Students should approach these tools as a toolkit rather than a hierarchy. No single method is universally best. The hallmark of a skilled healthcare researcher is the ability to assess the data environment, identify the most credible analytic strategy, and transparently communicate the assumptions each method requires. Mastering these advanced designs opens doors to rigorous program evaluation and health policy research.