What is process evaluation? Examining implementation for program improvement

Outcome evaluation examines target-group changes. Process evaluation examines whether the program intended to produce them was delivered to the intended group with the planned content, frequency and quality.

For low participation and unclear outcomes, process evaluation can identify problems in outreach, timing, registration or staff deployment. It prevents rejecting the whole program based solely on outcome figures.

This guide organizes practical checks into reach, quantity, quality, participant responses and implementation conditions. These are not a fixed common scale. Select implementation elements relevant to outcome pathways and check what followed plans and what changed.

Distinguish reach from implementation quality: whether targets were reached differs from whether content was provided as intended. Even high participation can imply different outcome interpretations if necessary support was insufficient.

• Reach: was the intended group reached?
• Quantity: did counts, time and support volume follow plans?
• Quality: were necessary content, procedures and expertise maintained?
• Participant responses: how accessible and acceptable was delivery?
• Conditions: what constraints concerned staffing, coordination, venue or institutions?

Requiring many new frontline records is unsustainable. First inventory attendance lists, consultation records, meeting notes, inquiries and staff reflections, then add only missing information in brief forms.

Combine brief surveys, interviews and observations of delivery to check user experiences. Alongside numbers, retain examples of practical innovations and barriers to make improvements easier to develop.

1. Select important implementation elements from the logic model
2. Set evaluation items for reach, quantity, quality and conditions
3. Check information available in existing records
4. Create concise forms only for missing information
5. Review numbers and examples in routine meetings
6. Record improvements and implementation-condition changes

Results inform where changes might improve outcomes as well as judgments of quality. Compare reach across target groups, sites with different conditions and cases with or without outcomes to develop hypotheses.

Avoid asserting conclusions from a few cases alone. Compare numbers, observations and staff/user accounts, design a small next improvement trial and check it again through monitoring.

• Compare results by target group, site and period
• Record implementation conditions in high-outcome cases
• Do not attribute problems solely to insufficient staff effort
• Avoid adding too many improvements at once
• Check post-change results in subsequent monitoring