EvalCompassEvalCompass
Design & theoryEvaluation designData collection planningIndicator mappingEvaluation frameworkMixed methods

Evaluation Design Matrix

Evaluation Design Matrix
Also known as: Evaluation matrix、Evaluation framework table、Evidence matrix

A table linking evaluation questions, judgment criteria, required information and collection and analysis methods row by row. It reveals items that cannot answer the questions and information that would be collected but not used. [1]

Evaluation design matrix worksheet — openReferences and checked sections
Difficulty
Intermediate
Cost
Low
Duration
Illustrative estimate (adjust for scale and staffing): 1–2 days (after evaluation questions are finalized)
Content updated: 2026-10-06Source checked: 2026-09-13 (includes AI cross-checking)No documented human reviewIncludes fictional examples

How it works at a glance

An evaluation design matrix organizes the evidence and methods used to examine each evaluation question. Questions, indicators, information sources and collection methods appear side by side to check whether data can be obtained. [1]

Select columns to fit the evaluation. This site’s worksheet also records responsibilities and deadlines, connecting collection tasks to implementation planning. Quantitative and qualitative columns organize the information each question requires. [1]

Evaluation questionJudgment criteriaIndicatorData sourceCollection methodAnalysis methodQuestion 1Question 2Question 3One row = one question. Work from left to right to specify how it will be answered.Methods follow the question (form follows question)
A central evaluation-design document linking each evaluation question to judgment criteria, indicators, data sources, collection methods, and analysis methods.

When to use it and when it is unsuitable

When to use it

  • Preparing a detailed data collection and analysis plan after KEQs are finalized
  • Clarifying the role of each method in evaluations combining quantitative and qualitative data collection
  • Clarifying responsibilities within an evaluation team with multiple members
  • Explaining and agreeing on the overall design with the commissioner or funder

When it is unsuitable

  • Before evaluation questions are finalized; complete KEQ development first
  • Small, simple evaluations using a single method; a brief plan is sufficient instead of a detailed matrix

Participants, roles, and deliverables

Participants and roles

Evaluation officers (lead evaluator and data collectors). Finalize after commissioner review. Working with 2–5 people is an illustrative plan; adjust staffing to evaluation scale and required roles.

Main deliverables

  • Evaluation design matrix (overview table)
  • Data collection plan
  • Evaluation team responsibility table

Implementation steps

  1. Enter KEQs and organize subsidiary questions
    Enter finalized KEQs and subsidiary questions in the first column. One row per subsidiary question supports detailed design.
  2. Enter judgment criteria
    For each KEQ or subsidiary question, enter criteria for judging achievement, such as benchmarks, targets or normative standards.
  3. Set indicators
    Select and map the quantitative and qualitative indicators needed for each subsidiary question. Both are not always necessary; give reasons for unused fields. Make indicators specific using SMART criteria, and distinguish primary from supplementary indicators when using several.
  4. Identify data sources
    Record sources for each indicator, such as administrative statistics, existing reports, survey data or interviews. Check whether primary data are needed or secondary data suffice.
  5. Select data collection methods
    Record the collection method for each source, such as questionnaires, interviews, observation or document review, together with sample size, participants and timing.
  6. Specify analysis methods
    Record how each dataset will be analyzed, such as descriptive statistics, regression, thematic analysis or content analysis. Specify integration methods when combining quantitative and qualitative data.
  7. Assign responsibilities and schedules
    Assign a data collector and deadline to each row (question / indicator) to complete a feasible plan.

Strengths and limitations

Strengths

  • Combines all evaluation design elements in one table so overall consistency can be checked at a glance
  • Clarifies who should do what even when multiple methods and staff are involved
  • Maintains traceability from evaluation questions through data collection and analysis

Limitations

  • More detailed matrices take longer to prepare and may diverge from actual collection and analysis
  • Changes during implementation, such as stakeholder replacement or inaccessible data, require revisions, but matrices often remain outdated
  • Completing the format can become an end in itself, obscuring content quality such as the validity of judgment criteria

Common pitfalls

  • Writing only “achievement rate” or “satisfaction” in indicator fields without specifying measurement
  • Listing only method names while omitting the sample, participants, timing and responsibility
  • Filling in only quantitative data when a question requires qualitative information, leaving collection and analysis undesigned
  • Assuming the matrix completes evaluation design and skipping pilot tests or preliminary checks

Conditions of use and evidence

  • Adapt the illustrative matrix items to the evaluation being conducted. [1]
Fictional example; not actual results

Turning accessibility questions into a data collection plan

The following fictional example identifies records needed to evaluate improvements to counseling service intake.

KEQ / subsidiary questionJudgment criteriaIndicators (quantitative)Indicators (qualitative)Data sourceCollection methodAnalysis methodPerson responsibleDeadline
Does current intake reach those who need the service?Can explain reasons for non-use by time slot and possible responsesUnique users by time slot, with duplicates removedReasons for hesitating to use the serviceIntake records and consenting residentsRecord tabulation and voluntary individual interviewsCompare counts by time slot with categorized reasonsEvaluation officer ANovember 15, 2026

Common errors in judgment

Common mistake

Writing only “questionnaire” as an information source

How to revise

Specify whom to ask, when, and which survey items address each evaluation question

ReasonThe method name alone does not show whether the question can be answered.

Common mistake

Making it a goal to fill every row with both quantitative and qualitative information

How to revise

Choose needed information types and explain unused fields

ReasonEvidence suited to the question matters, rather than the number of methods.

Organize your thinking by writing

Evaluation design matrix worksheet

A practical template is available to fill in and print on A4 paper.

Open worksheet

Related practical guides

What is a logic model?How to develop evaluation indicatorsHow to write an evaluation planHow to develop evaluation questions

Related methods

Key Evaluation QuestionsIndicator DesignLogic ModelMixed Methods Research

References and checked sections

  1. How to Manage Gender-Responsive Evaluation: Evaluation Handbook
    UN Women · 2015
    Section consulted
    Tool 8: Evaluation matrix template, p. 138
    How it relates to the text
    Adapt the illustrative matrix items to the evaluation being conducted.
    Source checked: (includes AI cross-checking)

Editorial and checking information

Content updated: Source checked: (includes AI cross-checking)No documented human reviewIncludes fictional examplesEditorial policy and scope of checkingCite this pageReport a correction

How to cite this page

EvalCompass Editorial Team. “Evaluation Design Matrix”. EvalCompass, updated 2026-10-06. https://evalcompass.jun-nakatani.com/en/methods/evaluation-matrix/
Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history
  • — Aligned instructions for selecting necessary indicators with the completion checklist and repaired an incomplete staffing sentence.
  • — Updated explanations to correspond to the sources and added method-specific fictional completed examples, judgment examples and a change log.