Causal inference & experimentsCausal inferenceNatural experimentPanel dataPolicy evaluationQuasi-experiment
Difference-in-Differences
Difference-in-Differences
Also known as: DiD、Double-difference method、Difference of differences
Estimates effects by subtracting before/after change in the comparison group from that in the intervention group. Assumes parallel changes without intervention. [1]
- Difficulty
- Advanced
- Cost
- Medium
- Duration
- Illustrative planning durations, not common standards: data preparation 1–4 weeks, analysis/interpretation 1–2 weeks
Content updated: 2026-10-07Source checked: 2026-10-07 (includes AI cross-checking)No documented human reviewIncludes fictional examples
How it works at a glance
In the basic two-group, two-timepoint design, subtract the comparison group's change from the intervention group's change. Fixed level differences are removed, but time-varying changes affecting only one group are not. [1]
Check pre-trends, concurrent policies and spillovers. Similar past trends do not prove the post-intervention counterfactual. Staggered adoption across groups requires an appropriate estimation design. [1]
When to use it and when it is unsuitable
When to use it
- Phased or geographically limited policies with before/after data for both groups
- Retrospective effects of policies that could not be randomized
- Use timing or eligibility differences to compare group changes
- Cost-efficient use of administrative records or existing panels
When it is unsuitable
- Only one post-intervention cross-section without pre-data
- Clearly different pre-trends, violating parallel trends
- Intervention selection strongly depends on outcomes, such as targeting worse-performing areas
Participants, roles, and deliverables
Participants and roles
1–2 econometric evaluators/analysts collaborating with managers authorized to access administrative data
Main deliverables
- Table of DiD, standard errors and confidence intervals
- Before/after time-series plots including pre-trends
- Robustness results such as placebos
- Policy implications
Implementation steps
- Define intervention and control groupsDefine recipients and non-recipient comparison groups; choose controls with similar pre-trends.
- Collect data and check structurePrepare both groups' before/after outcomes. Specify same-person panels versus repeated cross-sections and check population/measurement comparability. Examine multiple pre-periods where available.
- Visually examine parallel trendsPlot pre-trends and consider policy selection to assess supporting evidence and threats. Failure to detect pre-differences does not prove counterfactual post-period parallelism.
- Calculate DiDSubtract control before/after mean change from intervention before/after mean change. In regression, the treated-group × post-period interaction coefficient corresponds to DiD.
- Control covariates and check robustnessSelect adjustments by timing and causal role. Use standard errors fitting assignment and error dependence. Examine sensitivity through placebos/specifications. Carefully select extensions for staggered adoption.
- Report assumptions' grounds and findingsReport estimates/uncertainty plus parallel trends, anticipation, spillovers, concurrent policies and composition changes. Statistical tests alone do not certify identification assumptions.
Strengths and limitations
Strengths
- Uses natural experiments or existing data for causal inference without randomization
- Removes group-specific fixed differences: time-invariant unobserved factors
- Low-cost retrospective evaluation using administrative data
- Allows critical assessment of assumptions through pre-trends and comparator rationale
Limitations
- Counterfactual parallel trends cannot be directly proven; anticipation, spillovers and composition also matter
- Cannot address time-varying unobserved confounding affecting only intervention groups
- Few units, such as 1–2 treated units, sharply reduce precision
- Concurrent policy changes make individual effects difficult to separate
Common pitfalls
- Do not assume parallelism without checking trends and comparator context. State limits of sparse pre-data and explain that graphs/tests cannot prove the assumption.
- Spillovers through proximity or networks mean controls no longer represent a pure counterfactual.
- For multiple outcomes, predefine primary indicators and adjust multiple testing.
- Few aggregate treated units can underestimate standard errors; consider synthetic controls or other methods.
Conditions of use and evidence
Compare before/after a consultation hub opens
A fictional calculation comparing scheme application rates in intervention and non-intervention areas.
Scroll horizontally to view the table
| Area | Before intervention | After intervention | Change |
|---|---|---|---|
| Intervention area | 50% | 65% | +15 percentage points |
| Comparison area | 40% | 45% | +5 percentage points |
| Difference-in-differences | — | — | +10 percentage points |
Common errors in judgment
Common mistake
Attribute the full 15-point increase to intervention
How to revise
Subtract the comparison area's 5-point increase
ReasonCommon time changes are included.
Common mistake
Assume a non-significant pre-trend test proves assumptions
How to revise
Consider trends, power, assignment history and concurrent policies together
ReasonTests alone cannot establish counterfactual assumptions.
Organize your thinking by writing
DiD design and analysis checklist
A practical template is available to fill in and print on A4 paper.
Open worksheetRelated practical guides
Related methods
References and checked sections
- Impact Evaluation in Practice, Second EditionPaul J. Gertler et al. / World Bank and Inter-American Development Bank · 2016
- Section consulted
- Chapter 7: Difference-in-Differences / Chapter 8: Matching
- How it relates to the text
- Unlike matching, DiD removes fixed group differences but requires assumptions on differential change over time.
- Magenta Book Annex A: Analytical methods for use within an evaluationHM Treasury / Evaluation Task Force · 2026
- Section consulted
- A2.7 pp.29–30, What is it? / How is it used? / Pros and cons: pre-intervention trends, comparable data, counterfactual parallel trends
- How it relates to the text
- Check comparable pre-trend data and the assumption of continued parallelism without intervention. Concurrent factors and data quality/quantity affect interpretation.
Editorial and checking information
How to cite this page
EvalCompass Editorial Team. “Difference-in-Differences”. EvalCompass, updated 2026-10-07. https://evalcompass.jun-nakatani.com/en/methods/did/
Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.
Change history
- — Checked sources for definitions and conditions and added fictional examples, judgment rationales and completed entries for every field. Revised unsupported historical or uptake claims. Reviewed procedures and cautions, adjusting fixed counts, thresholds and institutional generalizations. Preserved field positions, types and counts while revising fixed conditions and thresholds.
- — Added Magenta Book Annex A (May 2026) A2.7 solely for pre-trends, comparator data and counterfactual parallel-trends checks. Retained cautions that assumptions cannot be proven, World Bank sources, worksheet and examples.