Connect applicant follow-up findings to decisions
A fictional example for illustration: 20 applicants, 18 attendees, 15 follow-up responses and 10 self-reports of procedure use.
| Results to check | Scope of interpretation | Next decision |
|---|---|---|
| 10/15 ≈ 66.7% | Respondents' self-reported use; 5 non-respondents unconfirmed | State denominators and voluntarily check non-response concentrated among experienced participants |
| 10/20 = 50% | Confirmed implementation among all applicants; not an established true overall proportion | Explain the 50–75% range under extreme missing-data-only assumptions, distinct from a confidence interval |
| 3 without opportunities; 2 non-users | Do not equate opportunities with lack of understanding or support | Plan small trials of situation-based exercises and voluntary consultation with owners and deadlines |
What this example supports
Distinguish observed change from causal training effects. Do not decide expansion from ratios alone; retain unknowns, burden, affected people's voices and next checks.
Common errors in judgment
Treat 10/15 as 66.7% of all 20
How to revise: Report respondents' 66.7%, confirmed use among all applicants of 50%, and 5 non-respondents together
Reason: Denominators and missing data change meaning.
Present the extreme-assumption 50–75% range as a 95% confidence interval
How to revise: Explain that it assumes all 5 non-respondents used or did not use procedures, and is not a confidence interval
Reason: It does not estimate sampling uncertainty or include other uncertainty such as self-report error.
Put it into practice next
Record users, decision timing, findings, actions, responsibilities and checks in the evaluation use worksheet
Frequently asked questions
Does before-and-after comparison show program effects?
It can establish observed changes, but other factors also contribute, so before-and-after differences alone do not establish causal effects. Design comparison conditions and methodological assumptions separately.
Can non-respondents be excluded from aggregation?
Respondent aggregation is possible, but state the denominator, total targets, non-respondent numbers and bias. Do not present respondent-only proportions as overall proportions.
References and checked sections
- Report & support use of findingsBetterEvaluation / Global Evaluation Initiative
- Section consulted
- opening paragraphs / Identify reporting requirements / Ensure accessibility / Develop recommendations / Support use
- How it relates to the text
- Identify users, decision timing and reporting formats from the start and support action on findings.
- CDC Program Evaluation Framework, 2024Centers for Disease Control and Prevention (CDC), MMWR
- Section consulted
- Types of Evaluation / Step 1: evaluability assessment / Step 2: program roadmap / Step 4: Indicators and data sources / Step 6: Act on findings
- How it relates to the text
- 6 stages from context to action, with collaboration, equity, learning and use across stages. Indicators can concern activities or outcomes; outcome checks alone do not establish attribution.
- Quality in qualitative evaluation (QQE)HM Treasury / Evaluation Task Force
- Section consulted
- §4.2: guiding principles / Framework questions 4, 7–8, 12: inference, sample coverage, diversity
- How it relates to the text
- Qualitative evaluation claims require analytical processes, case selection, different voices, and evidence and limits for broader inference.
- Magenta Book: Central Government guidance on evaluationHM Treasury / Evaluation Task Force
- Section consulted
- §1.6 pp.23–24: early planning / §2.2: evaluation scoping / §3.2–3.5 pp.71–84: method choice and causal approaches / §6.2 pp.144–145: use and dissemination
- How it relates to the text
- Integrating evaluation early into program design and selecting methods suited to questions and use. Impact evaluation distinguishes observed change from causal contribution.
- Applying Evaluation Criteria ThoughtfullyOECD Publishing
- Section consulted
- Executive summary: two principles / Chapter 3: Applying criteria thoughtfully / Chapter 3: Choosing which criteria to use
- How it relates to the text
- The 6 criteria are perspectives for value judgments, selected and applied carefully according to purpose, context and stakeholder power relations.
- Supplementary Guide: Handling complexity in policy evaluationHM Treasury / Evaluation Task Force; developed by CECAN
- Section consulted
- Chapter 1: context/history, continuous learning / Chapter 2: Table 2 evaluation challenges / Chapter 4: complexity-appropriate approaches
- How it relates to the text
- Track context, interactions and unexpected change in complex programs and revise designs as learning develops.
- UNEG Ethical Guidelines for EvaluationUnited Nations Evaluation Group (UNEG); official copy hosted by UNODC
- Section consulted
- pp.7–10: integrity, accountability, respect, beneficence / pp.20–22: conduct, consent, confidentiality, power / Annex 3 pp.30–35: phase checklists
- How it relates to the text
- Applying integrity, accountability, respect and beneficence throughout; specifying consent, withdrawal, confidentiality scope and harm reduction.
- Standards for Applying the Principles of Social Value: The Complete SetSocial Value International
- Section consulted
- §4.4.3 p.21: sensitivity of prioritisation / §5.1–5.3 p.23: counterfactual, attribution, displacement / Section Three p.33 items 8–9: SROI ratio optional, sensitivity, recommendations / Appendix 3 p.37: ratio as range and assumptions
- How it relates to the text
- Social value analysis concerns changes for affected people and decisions. SROI ratios are optional in the 2026 reporting items; disclosure includes sensitivity, assumptions and scope.
How to cite this page
Change history
- — A cross-guide review clarified indicator formats, outcome classifications and causal judgments, interpretation of tables, and the scope of SVI2026 where relevant.
- — Added a complete fictional community-training example linked to official primary sources. Specified denominators, missing data, causal scope and responsibility and timing for improvement decisions.









