EvalCompassEvalCompass

Evaluation process guide

Most evaluations follow broadly similar pathways. This guide divides evaluation practice into ten steps, with goals, tasks, deliverables, and useful methods for each. Follow it to stay on track, even in your first evaluation.

Contents: ten steps
1

Clarify the evaluation purpose and users

Goal: Be able to explain who will use the results, for what purpose and when

Main tasks

  • Identify evaluation commissioners and users, including funders, management, frontline staff and beneficiaries
  • Distinguish between accountability, learning and improvement, and decision-making purposes
  • Work backward from when the results will be used, such as budgeting or the next planning cycle, to draw up the schedule

Deliverables for this step

  • A one-sentence evaluation purpose
  • A list of intended users and uses
  • An overall schedule

Common difficulties

Before selecting methods, establish who will use the results, for which decisions and by what deadline. Allow time appropriate to the evaluation scale and decision schedule, and record points not yet agreed.

Useful methods

Utilization-Focused EvaluationStakeholder AnalysisEvaluation Standards and Ethics
2

Understand the program and organize its theory

Goal: Be able to explain on one page how the program is expected to produce outcomes: its program theory

Main tasks

  • Review program plans, implementation records and existing data
  • Understand the actual program through discussions with those responsible and frontline staff
  • Develop or update a logic model or theory of change with stakeholders

Deliverables for this step

  • A logic model or theory of change
  • A program overview note
  • An inventory of existing data

Common difficulties

A program on paper often differs from what happens in practice. Do not assume that document review alone provides understanding; always listen to people in the field.

Useful methods

Logic ModelTheory of ChangeDocument ReviewInterview
3

Assess evaluability

Goal: A realistic prospect that this evaluation can be carried out and produce meaningful findings

Main tasks

  • Check the clarity of the theory, data availability and stakeholder cooperation
  • Confirm the available budget, personnel and timeframe
  • Exclude questions that cannot be evaluated and narrow the evaluation scope

Deliverables for this step

  • Evaluability assessment findings
  • A scope definition
  • A list of risks and responses

Common difficulties

This step is the first safeguard against the expectation that everything should be evaluated. The willingness to say no here protects quality later.

Useful methods

Evaluability AssessmentStakeholder Analysis
4

Set evaluation questions

Goal: Questions that are needed for decisions and answerable under the current conditions are established

Main tasks

  • Turn what users really need to know into questions
  • Use frameworks such as the DAC criteria to check for gaps in the questions
  • Prioritize and scope questions according to their necessity for decisions and the available data, personnel and timeframe. 1 necessary question may be enough

Deliverables for this step

  • A prioritized list of evaluation questions

Common difficulties

Before adding questions, check the decisions the results will inform and the conditions needed to answer them. Explicitly recording questions that will not be answered this time makes it easier to share the data collection burden and the limits of claims.

Useful methods

Key Evaluation QuestionsOECD-DAC Evaluation CriteriaUtilization-Focused Evaluation
5

Select the evaluation design

Goal: An agreed approach to answering each question

Main tasks

  • Link each question to judgment criteria, indicators, data sources, collection methods and analysis methods
  • If causal investigation is needed, select a comparative approach: experimental, quasi-experimental or theory-based
  • Integrate these into an evaluation design matrix and obtain stakeholder agreement

Deliverables for this step

  • An evaluation design matrix
  • A causal inference design document, where needed

Common difficulties

Methods follow questions: form follows question. Designing backward from a fashionable method can leave the questions that matter unanswered.

Useful methods

Evaluation Design MatrixIndicator DesignRandomized Controlled TrialDifference-in-DifferencesContribution Analysis
6

Plan data collection and address ethics

Goal: An operational plan specifying what to collect, from whom and how, with ethical preparations in place

Main tasks

  • Design and pretest instruments such as questionnaires, interview guides and observation checklists
  • Decide the sampling approach and response targets
  • Establish rules for consent, personal information protection and data management

Deliverables for this step

  • A data collection plan
  • A set of data collection instruments
  • Consent forms and an ethics checklist

Common difficulties

Skipping a pretest can reveal unanswerable questions during the main study. Always try the instruments, even on a small scale. Ethical considerations cannot be added afterward.

Useful methods

Survey / QuestionnaireInterviewFocus Group DiscussionEvaluation EthicsEvaluation Standards and Ethics
7

Collect data

Goal: Data of the planned quality and quantity are available

Main tasks

  • Check progress, including response rates and completed activities, according to the collection period and time needed for corrections
  • Check data quality, including missing values and inconsistencies, alongside collection
  • Consider the burden on frontline staff and adjust the plan flexibly if necessary

Deliverables for this step

  • A quantitative and qualitative dataset
  • Collection records and field notes

Common difficulties

Check response patterns and non-response bias early in collection and act while adjustments are still possible. Set review intervals appropriate to the study period and allow time for data cleaning.

Useful methods

ObservationCase StudySecondary Data AnalysisDiary MethodPhotovoice
8

Analyze and integrate findings

Goal: Individual data have been turned into answers to the evaluation questions

Main tasks

  • Summarize and statistically analyze quantitative data, and code and thematically analyze qualitative data
  • Compare multiple data sources through triangulation and examine contradictions
  • Organize findings by evaluation question

Deliverables for this step

  • Analysis results
  • A list of findings by question

Common difficulties

Do not mix analysis results with interpretation. Distinguishing what the data show from the evaluator's inferences makes later judgments transparent.

Useful methods

Descriptive Statistics and Cross-tabulationThematic AnalysisMixed Methods ResearchContent Analysis
9

Make value judgments and develop recommendations

Goal: Clear evaluation judgments based on findings and feasible recommendations

Main tasks

  • Assess findings against predefined judgment criteria, such as rubrics
  • Hold a workshop to examine results with stakeholders and deepen interpretation
  • Limit recommendations to what is needed for decisions and specify who will do what and by when. 1 necessary recommendation may be enough

Deliverables for this step

  • Evaluation judgments against criteria
  • A list of recommendations

Common difficulties

The more recommendations there are, the less likely they are to be implemented. Prioritize them and do not write recommendations without an addressee responsible for implementation.

Useful methods

Rubric-Based EvaluationOECD-DAC Evaluation CriteriaWorld CaféBenchmarking
10

Report and support use

Goal: Results are actually used in decisions and learning

Main tasks

  • Tailor reporting products to readers, including the main report, summary, brief and slides
  • Share results through dialogue at reporting meetings and learning events
  • Obtain a management response and establish a system to track implementation of recommendations

Deliverables for this step

  • A set of evaluation reports
  • A management response and follow-up table

Common difficulties

Set responsibilities, deadlines and review dates according to when recommendations will be implemented and changes are expected. Agree a schedule that meets the next decision, rather than applying a uniform number of months.

Useful methods

Evaluation Report WritingEvaluation Dissemination and CommunicationManagement ResponseMeta-Evaluation

How to use the ten steps

These steps are the site’s own practical organization of evaluation work. Return to earlier stages as needed. First determine whose decisions the evaluation will inform, and align research demands with when the findings are needed.

See: CDC Program Evaluation Framework, 2024 (six steps and cross-cutting actions)
Content updated: No documented human reviewEditorial policy and scope of checkingCite this pageReport a correction

How to cite this page

EvalCompass Editorial Team. “Evaluation process guide”. EvalCompass, updated 2026-10-07. https://evalcompass.jun-nakatani.com/en/process
Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history
  • — Added static text for the evaluation process. Revised guidance to focus questions and recommendations on what decisions need, without imposing a fixed number.
  • — Added the role of the process guide, references, and citation information.