Propensity score matching design and diagnostics worksheet

PSM balances observed pre-intervention covariates. Explicitly state that hidden confounding and insufficient support may remain.

When to use this worksheet and what completion involves Fictional completed example

EvalCompass Worksheet

Propensity score matching design and diagnostics worksheet

Method: Propensity Score Matching

PSM balances observed pre-intervention covariates. Explicitly state that hidden confounding and insufficient support may remain.

Basic information

Enter evaluation basics

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Program name: Vocational consultation evaluation
Analyst: Analyst A (fictional)
Dataset and source: Pre-intervention survey and employment records (fictional)
Creation date: July 1, 2026 (fictional)

Reason for this entry: Distinguish pre/post measurement times.

Program name

Analyst

Dataset and source

Creation date

Writing tips and explanation for this section

About this section

Enter evaluation basics

Distinguish pre/post measurement times.

View the method explanation

Writing tips

Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.

Read the full fictional example

Confounder list

List all factors affecting participation and outcomes and whether observed

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Scroll horizontally to view the table

Variable	Participation association: high / medium / low	Outcome association: high / medium / low	Data available? ○ / ×	Proxy if unavailable
Pre-intervention employment experience	Hypothesized high	Hypothesized high	○	—
Pre-intervention employment motivation	Hypothesized high	Hypothesized high	×	No suitable proxy; retain as hidden-confounding limitation

Reason for this entry: Associations are illustrative hypotheses, not observations.

Scroll horizontally to view the table

Variable	Participation association: high / medium / low	Outcome association: high / medium / low	Data available? ○ / ×	Proxy if unavailable

Writing tips and explanation for this section

About this section

List all factors affecting participation and outcomes and whether observed

Associations are illustrative hypotheses, not observations.

View the method explanation

Writing tips

Keep the information in each row aligned. Scroll wide tables horizontally to see every column.

Read the full fictional example

Propensity score specification

Enter variables and functional form

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Model, such as logistic regression: Consider logistic regression
Included covariates: Age, employment experience and education, all pre-intervention
Interactions and squared terms, if any: Examine necessity through balance diagnostics
Diagnostics: overlap and post-match balance, not prediction alone: Prioritize overlap and balance, not AUC as success

Reason for this entry: The model is a matching tool.

Model, such as logistic regression

Included covariates

Interactions and squared terms, if any

Diagnostics: overlap and post-match balance, not prediction alone

Writing tips and explanation for this section

About this section

Enter variables and functional form

The model is a matching tool.

View the method explanation

Writing tips

Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.

Read the full fictional example

Matching and common support

Enter method and support findings

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Nearest-neighbor, kernel or other matching: Plan nearest-neighbor 1:1 matching
Matched intervention sample: 3 people: arithmetic example
Matched control sample: 3 people: arithmetic example
Excluded numbers and proportions outside support: Exclusions unspecified in illustration; report counts/proportions in real analysis
Balance summary, such as maximum SMD: Not calculated: no covariate data in illustration

Reason for this entry: Do not fabricate SMD from an outcome-only table.

Nearest-neighbor, kernel or other matching

Matched intervention sample

Matched control sample

Excluded numbers and proportions outside support

Balance summary, such as maximum SMD

Writing tips and explanation for this section

About this section

Enter method and support findings

Do not fabricate SMD from an outcome-only table.

View the method explanation

Writing tips

Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.

Read the full fictional example

Analysis checklist

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

☐ Not checked: Confounders identified from experts and prior research
☐ Not checked: Checked model diagnostics/overlap without judging prediction alone
☐ Not checked: Checked SMD/distributions and recorded criteria and residual differences
☐ Not checked: Prepared Love plot or balance table
☐ Not checked: Checked support and recorded excluded-observation treatment
☑ Checked: Specified effect population and differences from original treated-group ATT after exclusion
☐ Not checked: Conducted Rosenbaum bounds or other sensitivity analysis
☑ Checked: Reported possible hidden confounding and limitations

Reason for this entry: Define estimand as mean difference for matched participants; separately assess generalization to all-participant ATT. Covariate diagnostics and sensitivity analysis remain undone.

Confounders identified from experts and prior research
Checked model diagnostics/overlap without judging prediction alone
Checked SMD/distributions and recorded criteria and residual differences
Prepared Love plot or balance table
Checked support and recorded excluded-observation treatment
Specified effect population and differences from original treated-group ATT after exclusion
Conducted Rosenbaum bounds or other sensitivity analysis
Reported possible hidden confounding and limitations

Writing tips and explanation for this section

About this section

Define estimand as mean difference for matched participants; separately assess generalization to all-participant ATT. Covariate diagnostics and sensitivity analysis remain undone.

View the method explanation

Writing tips

Read each item and check those you have confirmed. Do not use the fictional example’s checkbox states as findings about your own program.

Read the full fictional example

Before writing, check when to use the worksheet and what completion involves

Propensity score matching design and diagnostics worksheet is a worksheet for applying Propensity Score Matching in meetings or individual work. For the overall duration of the method, use “Illustrative planning durations, not common standards: preparation 1–2 weeks, analysis/diagnostics 1–2 weeks” as a reference, adjusting for scale and resources. Fill in what you know provisionally and leave unresolved matters as questions to check next.

When this worksheet is useful

Compare participants/non-participants using observational administrative or survey data without randomization
Participant traits clearly differ and require alignment
Rich covariates and relatively limited presumed unobserved confounding

Participants and preparation

1–2 econometric/statistical specialists using R MatchIt, Stata or similar software; covariate choice needs domain expertise

Bring the materials used for decisions, existing data, and differing stakeholder views to help complete the worksheet.

Suggested order

1. Basic informationEnter evaluation basics
2. Confounder listList all factors affecting participation and outcomes and whether observed
3. Propensity score specificationEnter variables and functional form
4. Matching and common supportEnter method and support findings
5. Analysis checklist

Signs the worksheet is ready for review

Score model coefficients and fit statistics
Pre/post balance tables and Love plots
Mean differences/effects with target population and uncertainty

Cautions for review

Do not assume PSM removed all confounding. Observed balance may improve while hidden confounding remains; sensitivity analysis and honest discussion are essential.
Do not skip diagnostics. Examine standardized differences and plots; no single indicator or graph guarantees causal comparability.
Without overlapping scores, matching extrapolates and cannot provide credible comparisons.

Guides to read before starting

Read a completed fictional example (your entries will not change)

Fictional example; not actual results

A fictional explanation matching 3 pairs on pre-traits; not a sample-size recommendation for effect testing.

Basic information

Program name: Vocational consultation evaluation
Analyst: Analyst A (fictional)
Dataset and source: Pre-intervention survey and employment records (fictional)
Creation date: July 1, 2026 (fictional)

Reason for this entry: Distinguish pre/post measurement times.

Confounder list

Scroll horizontally to view the table

Variable	Participation association: high / medium / low	Outcome association: high / medium / low	Data available? ○ / ×	Proxy if unavailable
Pre-intervention employment experience	Hypothesized high	Hypothesized high	○	—
Pre-intervention employment motivation	Hypothesized high	Hypothesized high	×	No suitable proxy; retain as hidden-confounding limitation

Reason for this entry: Associations are illustrative hypotheses, not observations.

Propensity score specification

Model, such as logistic regression: Consider logistic regression
Included covariates: Age, employment experience and education, all pre-intervention
Interactions and squared terms, if any: Examine necessity through balance diagnostics
Diagnostics: overlap and post-match balance, not prediction alone: Prioritize overlap and balance, not AUC as success

Reason for this entry: The model is a matching tool.

Matching and common support

Nearest-neighbor, kernel or other matching: Plan nearest-neighbor 1:1 matching
Matched intervention sample: 3 people: arithmetic example
Matched control sample: 3 people: arithmetic example
Excluded numbers and proportions outside support: Exclusions unspecified in illustration; report counts/proportions in real analysis
Balance summary, such as maximum SMD: Not calculated: no covariate data in illustration

Reason for this entry: Do not fabricate SMD from an outcome-only table.

Analysis checklist

☐ Not checked: Confounders identified from experts and prior research
☐ Not checked: Checked model diagnostics/overlap without judging prediction alone
☐ Not checked: Checked SMD/distributions and recorded criteria and residual differences
☐ Not checked: Prepared Love plot or balance table
☐ Not checked: Checked support and recorded excluded-observation treatment
☑ Checked: Specified effect population and differences from original treated-group ATT after exclusion
☐ Not checked: Conducted Rosenbaum bounds or other sensitivity analysis
☑ Checked: Reported possible hidden confounding and limitations

Common errors in judgment

Match using post-intervention motivation

How to revise: Choose variables from pre-traits and causal relationships

Reason: Adjusting intervention-affected variables changes the estimand.

Assume high AUC establishes comparability

How to revise: Assess post-match balance and common support

Reason: Participation prediction differs from comparison validity.

Editorial and checking information

Content updated: 2026-09-13Source checked: 2026-09-13 (includes AI cross-checking)No documented human reviewIncludes fictional examplesEditorial policy and scope of checking Cite this page Report a correction

References and checked sections

Impact Evaluation in Practice, Second Edition
Paul J. Gertler et al. / World Bank and Inter-American Development Bank · 2016
Section consulted
Chapter 7: Difference-in-Differences / Chapter 8: Matching
How it relates to the text
Common support and balance do not prove absence of unobserved confounding.
Source checked: 2026-09-13 (includes AI cross-checking)

How to cite this page

EvalCompass Editorial Team. “Propensity score matching design and diagnostics worksheet”. EvalCompass, updated 2026-09-13. https://evalcompass.jun-nakatani.com/en/worksheets/psm/

Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history

2026-09-13 — Checked sources for definitions and conditions and added fictional examples, judgment rationales and completed entries for every field. Revised unsupported historical or uptake claims. Reviewed procedures and cautions, adjusting fixed counts, thresholds and institutional generalizations. Preserved field positions, types and counts while revising fixed conditions and thresholds.