What is an outcome? Meaning, time horizons and indicator development

Outcomes are changes for people, organizations or communities receiving activities or products. The OECD's 2023 glossary describes them as short- and medium-term effects of intervention outputs.

The subject matters. Consultation provided has the provider as subject and is not an outcome. Write with the target group as subject, such as consultees accessing necessary schemes.

Frameworks differ on whether knowledge and skill changes count as outcomes. The OECD glossary includes changes directly produced by activities within Outputs. Check the recipient's definitions first and note correspondence if they differ from this site's approach.

• The subject is the target group, organization or community
• Not necessarily directly controlled by the provider
• Can also be checked through operational or administrative records of behavior and conditions
• Decide checking times before measuring

Arranging outcomes in the order changes occur makes them easier to write. HM Treasury's Magenta Book recommends organizing expected pathways and assumptions and revising them according to evidence.

This site treats understanding and awareness as short-term, behavior and practice as medium-term, and conditions or social changes as long-term. These 3 categories are the site's practical approach. Some forms use 2 categories or initial/intermediate/final labels, so follow the recipient's terminology.

When unsure, think in sequence rather than duration. Changes that must happen before the next are on the shorter-term side. In training, explaining procedures comes first, workplace practice follows, and changed work decisions follow later.

Impact refers to significant higher-level effects, including unintended and negative effects. It is not another name for long-term outcomes.

Confusion arises because impact evaluation in Japanese often means causal-effect investigation. Impact as a term and impact evaluation as a method have different scopes. State which meaning is used in the document.

Distinguish classification from causal certainty. Specify the adopted framework's scope, treating outcomes as changes for people or organizations and impact as significant higher-level effects. Whatever label is used, whether observed changes are attributable to the program must be examined separately alongside other factors.

• Outcomes: changes for people or organizations
• Impact: significant higher-level effects, including positive, negative and unintended effects
• Do not mechanically replace long-term outcome with impact
• Distinguish impact from the method name impact evaluation

Outputs are products or services provided; outcomes are target-group changes. If unsure, check what is measured, not whether it comes from implementation records. Operational or administrative records documenting application starts, support use or conditions can also establish outcomes.

Participant, consultation and distribution counts are outputs; proportions applying learning, application starters and retention are outcomes. See the outputs and outcomes guide for comparisons and program examples.

• Delivery volume of products or services means outputs
• Changes in target-group behavior or conditions mean outcomes
• Operational or administrative records can provide outcome data when they measure target-group change
• Check measurement content and definitions rather than record names

These are fictional illustrations, not real programs. The same approach is applied to 4 sectors.

All examples describe changes that can be checked for the target group rather than what providers delivered. When adapting them, decide what changes for whom before considering how to check it.

Once outcomes are chosen, make them observable. Indicators are quantitative or qualitative items capturing conditions or progress in an intervention, outcomes or context. Since no indicator represents everything, combine several.

Define the target period, what will be judged against which criteria, exclusions and missing-data treatment, sources and checking times. Specify numerators and denominators for proportions, observation targets and judgment criteria for qualitative descriptions. Keep desired-level targets in separate fields.

Awareness increased cannot be measured as written. Specifying who becomes able to do what and how establishes a way to check it.

1. Write in one sentence what changes for whom
2. Identify situations where the change can be observed
3. Turn observable items into candidate indicators
4. Define period, judgment criteria and missing-data treatment, plus numerator and denominator for proportions
5. Decide sources and checking times
6. Keep targets in separate fields