Quasi-Experimental Designs: Zero to Hero Tutorial

This tutorial introduces quasi-experimental designs for situations where randomization is not feasible, ethical, or practical. You will learn how difference-in-differences, interrupted time series, regression discontinuity, matching, and natural experiments support causal inference when assignment is not randomized.

Prerequisites and Background Concepts
What Is a Quasi-Experiment?
Threats to Validity
Difference-in-Differences
Interrupted Time Series
Regression Discontinuity
Matching and Comparison Groups
Natural Experiments
Using DataStatPro
Worked Examples
Common Mistakes and How to Avoid Them
Quick Reference Cheat Sheet

1. Prerequisites and Background Concepts

You should understand:

Treatment or exposure: The intervention or event of interest.
Comparison group: Units not exposed to the intervention.
Counterfactual: What would have happened without the intervention.
Confounding: Systematic differences between exposed and unexposed groups.
Pre-period and post-period: Time before and after an intervention.
Selection bias: Bias from non-random treatment assignment.

2. What Is a Quasi-Experiment?

A quasi-experiment estimates causal effects without random assignment. Instead of relying on randomization, it uses design features such as time trends, thresholds, policy timing, or matched comparison groups.

Feature	Randomized Experiment	Quasi-Experiment
Assignment	Random	Natural, administrative, or self-selected
Main strength	Strong causal identification	Feasible in real settings
Main threat	Noncompliance or attrition	Confounding and selection

3. Threats to Validity

Quasi-experiments must address:

Confounding.
Secular trends.
Regression to the mean.
Policy anticipation.
Differential measurement.
Changes in population composition.
Spillover effects.

The design should make the counterfactual credible.

4. Difference-in-Differences

Difference-in-differences compares the change over time in a treated group with the change over time in a comparison group.

Group	Before	After	Change
Treated	$\bar{Y}_{T,0}$	$\bar{Y}_{T,1}$	$\bar{Y}_{T,1}-\bar{Y}_{T,0}$
Control	$\bar{Y}_{C,0}$	$\bar{Y}_{C,1}$	$\bar{Y}_{C,1}-\bar{Y}_{C,0}$

The estimator is:

$DID = (\bar{Y}_{T,1}-\bar{Y}_{T,0}) - (\bar{Y}_{C,1}-\bar{Y}_{C,0})$

Key assumption: parallel trends in the absence of treatment.

5. Interrupted Time Series

Interrupted time series evaluates whether the level or slope of an outcome changes after an intervention.

Segmented regression:

$Y_t = \beta_0 + \beta_1Time_t + \beta_2Intervention_t + \beta_3TimeAfter_t + \varepsilon_t$

Interpretation:

$\beta_1$ = baseline trend.
$\beta_2$ = immediate level change.
$\beta_3$ = slope change after intervention.

Use enough pre- and post-intervention time points.

6. Regression Discontinuity

Regression discontinuity uses a cutoff rule for treatment assignment.

Example: Students with score below 50 receive tutoring.

The treatment effect is estimated near the cutoff:

$\tau = \lim_{x \downarrow c} E[Y \mid X=x] - \lim_{x \uparrow c} E[Y \mid X=x]$

Key assumption: units cannot precisely manipulate assignment around the cutoff.

7. Matching and Comparison Groups

Matching creates a comparison group similar to the treated group on observed covariates.

Common approaches:

Exact matching.
Nearest-neighbour matching.
Propensity score matching.
Coarsened exact matching.

Matching only addresses observed confounding. Unmeasured confounding remains possible.

8. Natural Experiments

Natural experiments use external events or policy changes that assign exposure in a way that is plausibly unrelated to potential outcomes.

Examples:

Sudden policy eligibility changes.
Lottery-based program access.
Weather shocks.
Administrative cutoff dates.

The credibility depends on the assignment mechanism, not the label "natural experiment."

9. Using DataStatPro

Use DataStatPro to:

Summarize pre-post outcomes.
Plot trends before modeling.
Run regression models for adjusted comparisons.
Analyze time series and interrupted trends.
Estimate confidence intervals and effect sizes.
Export transparent tables and figures.

Visual diagnostics are essential for quasi-experimental claims.

10. Worked Examples

Example 1: Policy Change

A city introduces a smoking ban. Compare hospital admissions before and after the policy against a similar city without a policy change using difference-in-differences.

Example 2: Website Release

Conversion rate is tracked weekly before and after a checkout redesign. Use interrupted time series to estimate level and trend changes.

Example 3: Scholarship Eligibility

Students above a cutoff receive funding. Use regression discontinuity near the cutoff to estimate the local effect of scholarship eligibility.

11. Common Mistakes and How to Avoid Them

Mistake	Why It Matters	Better Practice
No comparison group in pre-post study	Cannot separate intervention from time trend	Add a comparable control
Ignoring pre-trends	Weakens DID credibility	Plot and test pre-period trends
Too few time points	ITS estimates become unstable	Collect enough observations before and after
Broad RD bandwidth	Compares non-comparable units	Focus near the cutoff
Matching on post-treatment variables	Introduces bias	Match only on pre-treatment covariates

12. Quick Reference Cheat Sheet

Design	Best Use	Key Assumption
Difference-in-differences	Treated and control groups over time	Parallel trends
Interrupted time series	Clearly timed intervention with repeated data	No concurrent shock
Regression discontinuity	Assignment by cutoff	No precise manipulation
Matching	Observational comparison	No unmeasured confounding
Natural experiment	External exposure assignment	Assignment is plausibly exogenous

Key formula: