EvalCompassEvalCompass
Causal inference & experimentsCausal inferenceObservational dataMatchingSelection bias adjustmentQuasi-experiment

Propensity Score Matching

Propensity Score Matching
Also known as: PSM、Propensity score method、Matching method

Estimates participation probabilities from observed pre-intervention characteristics and compares matched targets with similar probabilities. It cannot resolve unobserved confounding or lack of comparators. [1]

Propensity score matching design and diagnostics worksheet — openReferences and checked sections
Difficulty
Advanced
Cost
Medium
Duration
Illustrative planning durations, not common standards: preparation 1–2 weeks, analysis/diagnostics 1–2 weeks
Content updated: 2026-09-13Source checked: 2026-09-13 (includes AI cross-checking)No documented human reviewIncludes fictional examples

How it works at a glance

The propensity score is participation probability conditional on observed pre-intervention traits. Check post-match balance and overlap and report excluded people. [1]

Unobserved factors affecting both participation and outcomes bias estimates. Examine comparative-design validity rather than predictive fit alone. [1]

  1. Identify covariates and confounders
  2. Estimate propensity scores
  3. Match and check common support
  4. Diagnose post-match balance
  5. Estimate treatment effects
  6. Sensitivity analysis: Rosenbaum bounds
Sequence of implementation steps. See “Implementation steps” for details of each stage.

When to use it and when it is unsuitable

When to use it

  • Compare participants/non-participants using observational administrative or survey data without randomization
  • Participant traits clearly differ and require alignment
  • Rich covariates and relatively limited presumed unobserved confounding
  • An alternative in retrospective observational evaluation without feasible DiD or RDD

When it is unsuitable

  • Strong unobserved/unavailable participation determinants: scores cannot remove unobserved confounding
  • Extremely narrow common support leaves many without matches
  • Small samples shrink substantially after matching, reducing precision

Participants, roles, and deliverables

Participants and roles

1–2 econometric/statistical specialists using R MatchIt, Stata or similar software; covariate choice needs domain expertise

Main deliverables

  • Score model coefficients and fit statistics
  • Pre/post balance tables and Love plots
  • Mean differences/effects with target population and uncertainty
  • Rosenbaum bounds sensitivity results

Implementation steps

  1. Identify covariates and confounders
    Use domain knowledge to select pre-intervention confounders. Do not indiscriminately include post-intervention variables or outcome components; list unmeasured factors too.
  2. Estimate propensity scores
    Estimate treatment/participation probabilities from pre-covariates. Do not optimize prediction alone; check overlap and post-match balance.
  3. Match and check common support
    Apply selected nearest-neighbor, kernel or other matching. Check overlapping score distributions and decide treatment of observations outside common support.
  4. Diagnose post-match balance
    Compare standardized differences and distributions before/after. Check important distributional differences beyond means; no single threshold proves causal comparability.
  5. Estimate treatment effects
    Estimate differences for the population implied by matching and weights. With treated exclusions, do not equate effects with original-group ATT. Use appropriate variance estimation rather than unconditionally applying ordinary bootstrap.
  6. Sensitivity analysis: Rosenbaum bounds
    Show how strong unobserved confounding would need to be to alter results using Rosenbaum bounds or similar analyses. Honestly report robustness limits.

Strengths and limitations

Strengths

  • Reduces selection bias and enables fairer observational comparisons where RCTs are impossible
  • Condenses many covariates into one score, easing high-dimensional matching and the curse of dimensionality
  • Uses existing data with substantially lower time/cost than RCTs
  • Balance diagnostics transparently report covariate alignment

Limitations

  • Cannot address unobserved confounding: the greatest limitation. Causal validity depends on coverage of relevant variables.
  • Excluding controls reduces sample and precision
  • Narrow support means matched samples represent only part of original treated groups, reducing external validity
  • Sensitive to functional forms, included variables and interactions

Common pitfalls

  • Do not assume PSM removed all confounding. Observed balance may improve while hidden confounding remains; sensitivity analysis and honest discussion are essential.
  • Do not skip diagnostics. Examine standardized differences and plots; no single indicator or graph guarantees causal comparability.
  • Without overlapping scores, matching extrapolates and cannot provide credible comparisons.
  • Distinguish ATT, ATE and matched-subgroup effects. State population, weights and changes from exclusions.

Conditions of use and evidence

  • Common support and balance do not prove absence of unobserved confounding. [1]
Fictional example; not actual results

Find comparators for vocational consultation participants

A fictional explanation matching 3 pairs on pre-traits; not a sample-size recommendation for effect testing.

PairParticipant employmentComparator employment
111
210
300

Common errors in judgment

Common mistake

Match using post-intervention motivation

How to revise

Choose variables from pre-traits and causal relationships

ReasonAdjusting intervention-affected variables changes the estimand.

Common mistake

Assume high AUC establishes comparability

How to revise

Assess post-match balance and common support

ReasonParticipation prediction differs from comparison validity.

Organize your thinking by writing

Propensity score matching design and diagnostics worksheet

A practical template is available to fill in and print on A4 paper.

Open worksheet

Related practical guides

What is impact evaluation?

Related methods

Randomized Controlled TrialDifference-in-DifferencesRegression Discontinuity DesignRegression AnalysisContribution Analysis

References and checked sections

  1. Impact Evaluation in Practice, Second Edition
    Paul J. Gertler et al. / World Bank and Inter-American Development Bank · 2016
    Section consulted
    Chapter 7: Difference-in-Differences / Chapter 8: Matching
    How it relates to the text
    Common support and balance do not prove absence of unobserved confounding.
    Source checked: (includes AI cross-checking)

Editorial and checking information

Content updated: Source checked: (includes AI cross-checking)No documented human reviewIncludes fictional examplesEditorial policy and scope of checkingCite this pageReport a correction

How to cite this page

EvalCompass Editorial Team. “Propensity Score Matching”. EvalCompass, updated 2026-09-13. https://evalcompass.jun-nakatani.com/en/methods/psm/
Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history
  • — Checked sources for definitions and conditions and added fictional examples, judgment rationales and completed entries for every field. Revised unsupported historical or uptake claims. Reviewed procedures and cautions, adjusting fixed counts, thresholds and institutional generalizations. Preserved field positions, types and counts while revising fixed conditions and thresholds.