DiD design and analysis checklist

Organize assumptions. Post-intervention counterfactual parallelism cannot be directly proven; examine evidence and threats through pre-data and institutional context.

When to use this worksheet and what completion involves Fictional completed example

EvalCompass Worksheet

DiD design and analysis checklist

Method: Difference-in-Differences

Organize assumptions. Post-intervention counterfactual parallelism cannot be directly proven; examine evidence and threats through pre-data and institutional context.

Basic information

Enter evaluation basics

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Program or policy name: Consultation hub opening
Analyst: Analyst A (fictional)
Analysis period: data timepoints: Monthly 2024–2026 (fictional)
Creation date: July 1, 2026 (fictional)

Reason for this entry: Separate analysis and intervention periods.

Program or policy name

Analyst

Analysis period: data timepoints

Creation date

Writing tips and explanation for this section

About this section

Enter evaluation basics

Separate analysis and intervention periods.

View the method explanation

Writing tips

Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.

Read the full fictional example

Define intervention and control groups

Specify groups and selection rationale

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Scroll horizontally to view the table

Group	Who / which area / which period	Selection rationale
Intervention group	Area A opening a hub in April 2026 (fictional)	Plan to confirm policy coverage
Comparison group	Area B without a contemporaneous opening (fictional)	Plan to check similar pre-trends/institutional conditions

Reason for this entry: Do not treat unconfirmed comparability as fact.

Scroll horizontally to view the table

Group	Who / which area / which period	Selection rationale

Writing tips and explanation for this section

About this section

Specify groups and selection rationale

Do not treat unconfirmed comparability as fact.

View the method explanation

Writing tips

Keep the information in each row aligned. Scroll wide tables horizontally to see every column.

Read the full fictional example

Data and outcome variables

Enter sources, timepoints and main outcomes

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Sources: administrative records, questionnaires, etc.: Scheme application aggregates (fictional)
Available pre/post timepoints: Plan 24 pre-periods and 3 post-periods
Primary outcome variable: Monthly applicants / eligible people
Secondary outcomes, if any: None in this example
Control variables: covariates: Pre-intervention composition and local employment

Reason for this entry: Check rate denominator changes as numbers change.

Sources: administrative records, questionnaires, etc.

Available pre/post timepoints

Primary outcome variable

Secondary outcomes, if any

Control variables: covariates

Writing tips and explanation for this section

About this section

Enter sources, timepoints and main outcomes

Check rate denominator changes as numbers change.

View the method explanation

Writing tips

Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.

Read the full fictional example

Plan to examine parallel trends

Record methods/evidence for grounds and threats and what cannot be checked

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Plot 24 pre-intervention months and examine scheme and composition changes. Non-significant pre-coefficients do not prove the assumption.

Reason for this entry: Separate counterfactual evidence and limitations.

Writing tips and explanation for this section

About this section

Record methods/evidence for grounds and threats and what cannot be checked

Separate counterfactual evidence and limitations.

View the method explanation

Writing tips

Use the example as a reference and record what you have established about your own program. Distinguish assumptions and unchecked matters from confirmed facts.

Read the full fictional example

Analysis checklist

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

☐ Not checked: Checked pre-period counts/trends and stated examinable scope
☐ Not checked: Plotted pre-trends
☐ Not checked: Conducted pre-trend tests such as event studies
☐ Not checked: Used suitable standard errors and considered clustering
☐ Not checked: Considered placebos and recorded actions/limits
☑ Checked: Examined spillovers
☐ Not checked: Used design-appropriate estimation and checked interaction interpretation if used

Reason for this entry: The table illustrates arithmetic; actual data acquisition/estimation has not occurred. Assume spillovers are recorded as a design issue.

Checked pre-period counts/trends and stated examinable scope
Plotted pre-trends
Conducted pre-trend tests such as event studies
Used suitable standard errors and considered clustering
Considered placebos and recorded actions/limits
Examined spillovers
Used design-appropriate estimation and checked interaction interpretation if used

Writing tips and explanation for this section

About this section

The table illustrates arithmetic; actual data acquisition/estimation has not occurred. Assume spillovers are recorded as a design issue.

View the method explanation

Writing tips

Read each item and check those you have confirmed. Do not use the fictional example’s checkbox states as findings about your own program.

Read the full fictional example

Before writing, check when to use the worksheet and what completion involves

DiD design and analysis checklist is a worksheet for applying Difference-in-Differences in meetings or individual work. For the overall duration of the method, use “Illustrative planning durations, not common standards: data preparation 1–4 weeks, analysis/interpretation 1–2 weeks” as a reference, adjusting for scale and resources. Fill in what you know provisionally and leave unresolved matters as questions to check next.

When this worksheet is useful

Phased or geographically limited policies with before/after data for both groups
Retrospective effects of policies that could not be randomized
Use timing or eligibility differences to compare group changes

Participants and preparation

1–2 econometric evaluators/analysts collaborating with managers authorized to access administrative data

Bring the materials used for decisions, existing data, and differing stakeholder views to help complete the worksheet.

Suggested order

1. Basic informationEnter evaluation basics
2. Define intervention and control groupsSpecify groups and selection rationale
3. Data and outcome variablesEnter sources, timepoints and main outcomes
4. Plan to examine parallel trendsRecord methods/evidence for grounds and threats and what cannot be checked
5. Analysis checklist

Signs the worksheet is ready for review

Table of DiD, standard errors and confidence intervals
Before/after time-series plots including pre-trends
Robustness results such as placebos

Cautions for review

Do not assume parallelism without checking trends and comparator context. State limits of sparse pre-data and explain that graphs/tests cannot prove the assumption.
Spillovers through proximity or networks mean controls no longer represent a pure counterfactual.
For multiple outcomes, predefine primary indicators and adjust multiple testing.

Guides to read before starting

Read a completed fictional example (your entries will not change)

Fictional example; not actual results

A fictional calculation comparing scheme application rates in intervention and non-intervention areas.

Basic information

Program or policy name: Consultation hub opening
Analyst: Analyst A (fictional)
Analysis period: data timepoints: Monthly 2024–2026 (fictional)
Creation date: July 1, 2026 (fictional)

Reason for this entry: Separate analysis and intervention periods.

Define intervention and control groups

Scroll horizontally to view the table

Group	Who / which area / which period	Selection rationale
Intervention group	Area A opening a hub in April 2026 (fictional)	Plan to confirm policy coverage
Comparison group	Area B without a contemporaneous opening (fictional)	Plan to check similar pre-trends/institutional conditions

Reason for this entry: Do not treat unconfirmed comparability as fact.

Data and outcome variables

Sources: administrative records, questionnaires, etc.: Scheme application aggregates (fictional)
Available pre/post timepoints: Plan 24 pre-periods and 3 post-periods
Primary outcome variable: Monthly applicants / eligible people
Secondary outcomes, if any: None in this example
Control variables: covariates: Pre-intervention composition and local employment

Reason for this entry: Check rate denominator changes as numbers change.

Plan to examine parallel trends

Plot 24 pre-intervention months and examine scheme and composition changes. Non-significant pre-coefficients do not prove the assumption.

Reason for this entry: Separate counterfactual evidence and limitations.

Analysis checklist

☐ Not checked: Checked pre-period counts/trends and stated examinable scope
☐ Not checked: Plotted pre-trends
☐ Not checked: Conducted pre-trend tests such as event studies
☐ Not checked: Used suitable standard errors and considered clustering
☐ Not checked: Considered placebos and recorded actions/limits
☑ Checked: Examined spillovers
☐ Not checked: Used design-appropriate estimation and checked interaction interpretation if used

Reason for this entry: The table illustrates arithmetic; actual data acquisition/estimation has not occurred. Assume spillovers are recorded as a design issue.

Common errors in judgment

Attribute the full 15-point increase to intervention

How to revise: Subtract the comparison area's 5-point increase

Reason: Common time changes are included.

Assume a non-significant pre-trend test proves assumptions

How to revise: Consider trends, power, assignment history and concurrent policies together

Reason: Tests alone cannot establish counterfactual assumptions.

Editorial and checking information

Content updated: 2026-10-07Source checked: 2026-10-07 (includes AI cross-checking)No documented human reviewIncludes fictional examplesEditorial policy and scope of checking Cite this page Report a correction

References and checked sections

Impact Evaluation in Practice, Second Edition
Paul J. Gertler et al. / World Bank and Inter-American Development Bank · 2016
Section consulted
Chapter 7: Difference-in-Differences / Chapter 8: Matching
How it relates to the text
Unlike matching, DiD removes fixed group differences but requires assumptions on differential change over time.
Source checked: 2026-09-13 (includes AI cross-checking)
Magenta Book Annex A: Analytical methods for use within an evaluation
HM Treasury / Evaluation Task Force · 2026
Section consulted
A2.7 pp.29–30, What is it? / How is it used? / Pros and cons: pre-intervention trends, comparable data, counterfactual parallel trends
How it relates to the text
Check comparable pre-trend data and the assumption of continued parallelism without intervention. Concurrent factors and data quality/quantity affect interpretation.
Source checked: 2026-10-07 (includes AI cross-checking)

How to cite this page

EvalCompass Editorial Team. “DiD design and analysis checklist”. EvalCompass, updated 2026-10-07. https://evalcompass.jun-nakatani.com/en/worksheets/did/

Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history

2026-09-13 — Checked sources for definitions and conditions and added fictional examples, judgment rationales and completed entries for every field. Revised unsupported historical or uptake claims. Reviewed procedures and cautions, adjusting fixed counts, thresholds and institutional generalizations. Preserved field positions, types and counts while revising fixed conditions and thresholds.
2026-10-07 — Added Magenta Book Annex A (May 2026) A2.7 solely for pre-trends, comparator data and counterfactual parallel-trends checks. Retained cautions that assumptions cannot be proven, World Bank sources, worksheet and examples.