EvalCompassEvalCompass
Check the evidence

Evidence, sources, and editorial policy

Official primary sources supporting EvalCompass’s claims, with document editions, check dates and scope, interpretive limits, and editorial policy.

Content updated: Source checked: 2026-10-07 (includes AI cross-checking)No documented human reviewEditorial policy and scope of checkingCite this page
Check the evidence before using it

We connect the claims in the text with the primary sources used to check them. Listing a source and determining whether a method is suitable for a particular program are separate checks.

What these sources support in practice

Start with a theme to find sources and relevant learning materials.

View practical guides

Register of 10 primary sources

The following summaries were originally prepared in Japanese from the sources. The site’s worked examples and practical prompts are teaching materials created with reference to those sources.

Magenta Book: Central Government guidance on evaluationHM Treasury / Evaluation Task Force
Open original source
Document edition
May 2026; PDF cover verified
Document date
2026-05. Official HTML page updated: 2026-05-15
Source checks
Location checked
§1.6 pp.23–24: early planning / §2.2: evaluation scoping / §3.2–3.5 pp.71–84: method choice and causal approaches / §6.2 pp.144–145: use and dissemination
What the source supports
Integrate evaluation design into program design early, and select methods suited to the questions and intended use. In impact evaluation, distinguish observed change from causal contribution.
Limits of interpretation
The publication page’s history records replacement of the main document and Annex A on 2026-05-14 and addition of the main HTML document on 2026-05-15. Do not treat guidance for the UK government as a legal requirement for all programs in Japan.
Magenta Book Annex A: Analytical methods for use within an evaluationHM Treasury / Evaluation Task Force
Open original source
Document edition
May 2026; 60-page PDF
Document date
2026-05
Source checks
Location checked
A2.1 RCT pp.20–21 / A2.7 Difference-in-difference pp.29–30 / A5.4 Output or performance monitoring p.54
What the source supports
Covers RCT implementation conditions, pre-intervention trends and comparison conditions in DiD, and implementation and changes that monitoring can describe.
Limits of interpretation
Parallel trends before intervention do not prove parallel trends afterward.
Quality in qualitative evaluation (QQE)HM Treasury / Evaluation Task Force
Open original source
Document edition
official HTML currently served; substantive edition date not established
Document date
Unknown / not checked in this review. Official HTML page updated: 2026-05-15
Source checks
Location checked
§4.2: guiding principles / Framework questions 4, 7–8, 12: inference, sample coverage, diversity
What the source supports
Qualitative evaluation claims should explain the analysis process, case selection, differing voices, and the basis and limits of wider inference.
Limits of interpretation
Do not interpret the HTML update date or the 2026 copyright date as the publication date of a new 2026 methodological edition.
Supplementary Guide: Handling complexity in policy evaluationHM Treasury / Evaluation Task Force; developed by CECAN
Open original source
Document edition
official HTML currently served; substantive edition date not established
Document date
Unknown / not checked in this review. Official HTML page updated: 2026-05-15
Source checks
Location checked
Chapter 1: context/history, continuous learning / Chapter 2: Table 2 evaluation challenges / Chapter 4: complexity-appropriate approaches
What the source supports
For complex programs, track context, interactions, and unexpected changes, and revise the evaluation design as learning develops.
Limits of interpretation
Complexity does not remove the need for causal evidence. Do not infer substantive revisions from a page’s update date.
Applying Evaluation Criteria ThoughtfullyOECD Publishing
Open original source
Document edition
2021; DOI 10.1787/543e84ed-en; 81 pages
Document date
2021-03-15
Source checks
Location checked
Executive summary: two principles / Chapter 3: Applying criteria thoughtfully / Chapter 3: Choosing which criteria to use
What the source supports
The six criteria are perspectives for making value judgments. Select and use them carefully in light of the evaluation purpose, context, and power relationships among stakeholders.
Limits of interpretation
This is not a basis for selecting only criteria that favor a preferred conclusion. Do not label it as a 2026 publication.
CDC Program Evaluation Framework, 2024Centers for Disease Control and Prevention (CDC), MMWR
Open original source
Document edition
MMWR Recommendations and Reports 73(6):1–37
Document date
2024-09-26
Source checks
Location checked
Types of Evaluation / Step 1: evaluability assessment / Step 2: program roadmap / Step 4: Indicators and data sources / Step 6: Act on findings
What the source supports
Covers six steps from context to action, with collaboration, equity, and learning and use across all steps. Indicators may address activities as well as outcomes. Confirming outcomes alone does not establish causal attribution.
Limits of interpretation
Do not combine the CDC report’s “advance equity” wording with later wording on the general introduction page. Examples outside public health are presented as this site’s suggestions.
Choose methods and processesBetterEvaluation / Global Evaluation Initiative
Open original source
Document edition
live official guidance; date not shown
Document date
Unknown / not checked in this review
Source checks
Location checked
2: types of key evaluation questions / 3: situation / 5–6: complementary methods and evaluation matrix / 7–9: feasibility, review, revision
What the source supports
Distinguish descriptive, causal, evaluative, and action questions. Organize multiple methods, feasibility checks, and design review around the questions.
Limits of interpretation
Do not treat the footer’s 2025 date as a publication date. This source does not provide a universal ranking of methods.
Report & support use of findingsBetterEvaluation / Global Evaluation Initiative
Open original source
Document edition
Rainbow Framework live official guidance; date not shown
Document date
Unknown / not checked in this review
Source checks
Location checked
opening paragraphs / Identify reporting requirements / Ensure accessibility / Develop recommendations / Support use
What the source supports
Identify intended users, decision timing, and reporting formats from the outset, and support action based on the findings.
Limits of interpretation
Do not equate the number of reports submitted with actual use of evaluation findings.
UNEG Ethical Guidelines for EvaluationUnited Nations Evaluation Group (UNEG); official copy hosted by UNODC
Open original source
Document edition
2020 revised ethical guidelines; 35-page PDF
Document date
2020
Source checks
Location checked
pp.7–10: integrity, accountability, respect, beneficence / pp.20–22: conduct, consent, confidentiality, power / Annex 3 pp.30–35: phase checklists
What the source supports
Apply integrity, accountability, respect, and beneficence throughout, and specify consent, withdrawal, the limits of confidentiality, and measures to reduce harm.
Limits of interpretation
Do not treat UN evaluation ethics guidance as a uniform legal obligation for every program in Japan.
Standards for Applying the Principles of Social Value: The Complete SetSocial Value International
Open original source
Document edition
2026 complete consolidated set; 41-page PDF
Document date
2026
Source checks
Location checked
§4.4.3 p.21: sensitivity of prioritisation / §5.1–5.3 p.23: counterfactual, attribution, displacement / Section Three p.33 items 8–9: SROI ratio optional, sensitivity, recommendations / Appendix 3 p.37: ratio as range and assumptions
What the source supports
Social value analysis concerns changes experienced by stakeholders and their use in decisions. The 2026 reporting standards make the SROI ratio optional; when a ratio is disclosed, report sensitivity, assumptions, and scope.
Limits of interpretation
This register checks the 2026 standards. It does not certify that they replace the existing 2012 SROI Guide in its entirety.

Editorial policy and scope of checking

We provide practical explanations connecting questions, indicators, methods, interpretation, and the next decision. We prioritize official publisher pages and PDFs and record the sections checked. Numerical examples are marked as fictional teaching materials, not standard effects, costs, or durations.

  • Scope of this check:On October 7, 2026, AI compared the content of the 10 sources below with the chapters and locations listed. This check supports the two new worked-example guides and the explanations in this register.
  • Existing articles:Check the references and statements on each article. Checking these 10 sources does not constitute review of every article on the site or expert certification of all content.
  • Dates:Distinguish publication date, edition, official page update date, and the date this site checked the source. Mark unknown dates as unknown.
  • Limits of the claims:Before-and-after change, causal contribution, generalization to a whole population, and future prediction cannot all be established from the same evidence. Check the method’s assumptions and information gaps.

    • Updates and corrections

    October 7, 2026: added a register of 10 primary sources and fictional examples connecting questions to collection plans and findings to decisions. Checked the May 2026 Magenta Book, the CDC’s 2024 MMWR report, and SVI’s 2026 integrated standards. HTML update dates for QQE and the complexity guide are not treated as new publication years for the original works.

    If you find a content or link issue, contact the operator with the page URL and relevant section. Do not send identifiable research records; include only what is needed to check the correction.

    Operator and scope of editorial checking

    How to cite this page

    EvalCompass Editorial Team. “Evidence, sources, and editorial policy.” EvalCompass, updated October 7, 2026. https://evalcompass.jun-nakatani.com/en/evidence

    Add the date you accessed the page. Site text may be reused under CC BY 4.0. Linked primary sources have their own terms of use.