Practical evaluation guide

How to conduct evaluation interviews: guides, interviewing and analysis

Practical guidance on interview design, question guides, interviewing, recording, analysis and ethics for evaluation.

Use questions to understand the meaning of experience

Main takeaway

Evaluation interviews are qualitative research for deeper understanding of target-group or stakeholder experiences, reasons for change and improvement implications. A semi-structured guide aligned with evaluation questions, specific examples in respondents' own words and thematic records provide decision information unavailable from numbers alone. [1]

Survey patterns of satisfaction or usefulness may not explain why people felt that way or who was not reached. Interviews address change processes and context.

A good interview does not aim merely to hear many accounts. Design whom to ask, what to ask and how to interpret it beforehand to answer necessary evaluation questions.

Develop questions that explore the other person’s experience

Identify when interviews are appropriate, then plan questions, records, and analysis of themes and exceptions.

When interviews are suitable

Interviews suit understanding reasons for outcomes, implementation barriers, users' experiences and unintended effects. They are especially useful for new programs, small groups, complex support and topics requiring respect for respondents' words.

They do not suit estimating overall proportions or frequencies. Combine with questionnaires or use records when asking how widespread a change is.

When interviews are suitable
What to learn	Suitable method	Supplementary information
Reasons and processes of change	Interviews	Can explore specific experiences and context
Overall trends and proportions	Questionnaire surveys	Check target scope and sampling and non-response bias
Actual behavior and situations	Observation	Can also compare words and behavior
Integrating multiple sources of evidence	Mixed methods	Combine numbers and accounts

[1]

Develop an interview guide

In semi-structured interviews, use predefined issues as prompts while adapting order and probes to experience. This guide proposes facts, specific situations, reasons and improvement ideas in sequence, without requiring the same flow for everyone.

Asking only what did you think tends to elicit abstract answers. Questions about timing, situations and actions, such as the last use occasion and what happened afterward, increase specificity.

Organize information needs by evaluation question
Arrange questions in respondents' experience sequence
Prepare questions on facts, examples, reasons and improvements
Avoid technical terms and leading wording
Pilot with a few people and revise unclear questions
Explain recording, notes and anonymization

[1]

Interviewing and recording

Too much interviewer explanation makes respondents' own words harder to elicit. Stay patient during silence, repeat their words to check understanding, and probe with what does that mean specifically?

Retain context, timing and distinctions between facts and interpretations, not just conclusions. Obtain prior consent for recording and decide retention, access and deletion.

Prepare an accessible opening and explanation of purpose
Check respondents' specific words and situations
Avoid questions revealing evaluator expectations
Separate facts, interpretations and proposals in records
Handle identifying information carefully

[3]

Analyze both themes and exceptions

Collect statements by evaluation question and organize common themes, different experiences and examples. Interpret their meaning for decisions rather than assuming frequency alone establishes importance.

Do not select only statements matching hypotheses; examine contrary cases and experiences too. When combining numbers, link which accounts explain which figures and avoid excessive generalization.

Organize statements by evaluation question
Examine exceptions and minority views alongside common themes
Anonymize statements used as evidence for conclusions
Explore contradictions with quantitative data
State interpretations and limitations in reports

[2]

Fictional example; not actual results

Explore experiences of not proceeding with applications

A fictional example for illustration: voluntarily interview people who did not start applications after consultation. Statements are fictional.

Question	Illustrative answer	Interpretation and next check
What happened the last time you tried to proceed?	I tried gathering documents but could not visit the office on weekdays	A hypothesis of time constraints; also check available alternatives
How did you understand the consultation explanation?	I understood it, but could not take time off work	Distinguish lack of understanding from practical conditions
Anything else you would like to mention?	I was consulting another office	Check other organizations' support pathways

What this example supports

Do not infer all non-applicants' reasons from one account. Check different experiences and discrepancies with records, and retain statements, evaluator interpretations and unconfirmed matters separately.

Common errors in judgment

Suggest a cause by saying so the explanation was unclear

How to revise: Ask the person to describe the situation and barriers in their own words

Reason: This avoids leading toward the evaluator's expected answer.

Present one striking statement as an overall trend

How to revise: Also explain selection and contrary cases

Reason: In-depth understanding differs from estimating proportions.

Put it into practice next

Prepare experience questions and follow-up checks in the interview worksheet

View an example and open the worksheet Read the method explanation

Frequently asked questions

How many evaluation interviews are needed?

It depends on purpose, participant diversity and time. Include representative perspectives and different experiences, adjusting according to whether the material can be understood sufficiently.

Is recording mandatory?

No, but it supports accurate records. Without recording, use two note-takers or organize detailed notes immediately afterward.

Can users be asked for negative opinions?

Ask within what they feel comfortable discussing. Explain their freedom not to answer and ensure methods do not affect their treatment in support. They need not be pressed to describe painful experiences.

References and checked sections

Interview study: qualitative studies
Office for Health Improvement and Disparities（GOV.UK） · 2020
Section consulted
How to carry out an interview study / Pros / Cons
How it relates to the text
Digital health evaluation guidance on semi-structured guides, exploring experience and limitations in estimating opinion proportions
Source checked: 2026-09-13 (includes AI cross-checking)
Quality in qualitative evaluation (QQE)
HM Treasury / Evaluation Task Force
Section consulted
4. The framework（Sample 7–8 / Analysis 12 / Reporting 14 / Ethics 17）
How it relates to the text
Selection rationale, different perspectives and negative cases, paths to interpretation, consent and confidentiality
Source checked: 2026-09-13 (includes AI cross-checking)
Guiding Principles
American Evaluation Association
Section consulted
Systematic Inquiry / Integrity / Respect for People
How it relates to the text
Explaining methods and limitations, honest reporting, consent, confidentiality and avoidance of harm
Source checked: 2026-09-13 (includes AI cross-checking)

Content updated: 2026-09-13Source checked: 2026-09-13 (includes AI cross-checking)No documented human reviewIncludes fictional examplesEditorial policy and scope of checking Cite this page Report a correction

How to cite this page

EvalCompass Editorial Team. “How to conduct evaluation interviews: guides, interviewing and analysis”. EvalCompass, updated 2026-09-13. https://evalcompass.jun-nakatani.com/en/guides/interview-guide/

Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history

2026-09-13 — Distinguished original methodological sources from practical guidance and added specific fictional examples, judgment examples and next actions. Revised categorical claims about classification and measurement conditions.

Use questions to understand the meaning of experience