Text mining implementation and validation worksheet

Results begin exploration and require human interpretation and returning to originals. Before AI use, check personal or confidential information and follow organizational rules.

When to use this worksheet and what completion involves Fictional completed example

EvalCompass Worksheet

Text mining implementation and validation worksheet

Method: Text Mining and AI-Assisted Analysis

Results begin exploration and require human interpretation and returning to originals. Before AI use, check personal or confidential information and follow organizational rules.

Basic information

Define the analytical framework.

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Program and evaluation issue: Explore open application responses
Data type and count, such as 500 open responses: 3 fictional illustrative sentences
Tool: KH Coder / R tidytext, etc.: KH Coder 3 is a candidate.
Personal or confidential information and response: Invented examples. For actual data, remove identifiers and check use conditions.
Responsible staff and analysis period: Staff member A / July 2026 (fictional)

Reason for this entry: These short 3 sentences are not a practically sufficient corpus.

Program and evaluation issue

Data type and count, such as 500 open responses

Tool: KH Coder / R tidytext, etc.

Personal or confidential information and response

Responsible staff and analysis period

Writing tips and explanation for this section

About this section

Define the analytical framework.

These short 3 sentences are not a practically sufficient corpus.

View the method explanation

Writing tips

Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.

Read the full fictional example

Preprocessing settings

Record preprocessing for reproducibility.

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Scroll horizontally to view the table

Setting	Value	Rationale
Co-occurrence unit	Plan to use each response document.	Do not create ties across respondents.
Negation	Check against originals.	Word presence alone can reverse meaning.

Reason for this entry: Retain reproducible settings.

Scroll horizontally to view the table

Setting	Value	Rationale

Writing tips and explanation for this section

About this section

Record preprocessing for reproducibility.

Retain reproducible settings.

View the method explanation

Writing tips

Keep the information in each row aligned. Scroll wide tables horizontally to see every column.

Read the full fictional example

Forced extraction and stop words

Record word settings for analytical accuracy.

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Scroll horizontally to view the table

Word	Type: forced extraction / exclusion	Selection rationale
Consultation desk	Candidate forced extraction	Do not split the concept.
Time	Do not exclude.	Important word with different meanings to compare in originals

Reason for this entry: Do not exclude solely for high frequency.

Scroll horizontally to view the table

Word	Type: forced extraction / exclusion	Selection rationale

Writing tips and explanation for this section

About this section

Record word settings for analytical accuracy.

Do not exclude solely for high frequency.

View the method explanation

Writing tips

Keep the information in each row aligned. Scroll wide tables horizontally to see every column.

Read the full fictional example

Theme interpretation notes

Link clusters to evaluation questions and always interpret against originals.

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

Scroll horizontally to view the table

Cluster or candidate theme	Frequent words included	Link to evaluation question	Source evidence supporting interpretation
Time burden of procedures	Time and entry	Conditions preventing applications	Not enough time, so stopped entering data (fictional). Do not group the other 2 sentences under this theme.

Reason for this entry: Link representative words to supporting sentences.

Scroll horizontally to view the table

Cluster or candidate theme	Frequent words included	Link to evaluation question	Source evidence supporting interpretation

Writing tips and explanation for this section

About this section

Link clusters to evaluation questions and always interpret against originals.

Link representative words to supporting sentences.

View the method explanation

Writing tips

Keep the information in each row aligned. Scroll wide tables horizontally to see every column.

Read the full fictional example

Completion check

Completed example for this section (fictional)

This fictional example is for explanation. Adapt it to your own program. Your entries will not change.

☑ Checked: Checked information handling against organizational rules
☑ Checked: Recorded stop words, minimum frequencies and other preprocessing
☑ Checked: Interpreted networks against source texts
☐ Not checked: Verified AI outputs against originals where used
☑ Checked: Reported surface-pattern limitations
☐ Not checked: Recorded tool names, versions and settings in methods

Reason for this entry: No generative AI is used in this example. Record actual software versions and settings during execution.

Checked information handling against organizational rules
Recorded stop words, minimum frequencies and other preprocessing
Interpreted networks against source texts
Verified AI outputs against originals where used
Reported surface-pattern limitations
Recorded tool names, versions and settings in methods

Writing tips and explanation for this section

About this section

No generative AI is used in this example. Record actual software versions and settings during execution.

View the method explanation

Writing tips

Read each item and check those you have confirmed. Do not use the fictional example’s checkbox states as findings about your own program.

Read the full fictional example

Before writing, check when to use the worksheet and what completion involves

Text mining implementation and validation worksheet is a worksheet for applying Text Mining and AI-Assisted Analysis in meetings or individual work. For the overall duration of the method, use “Planning example, not a universal standard duration: 1–4 weeks for preprocessing, analysis, interpretation and validation” as a reference, adjusting for scale and resources. Fill in what you know provisionally and leave unresolved matters as questions to check next.

When this worksheet is useful

When several hundred or more open responses make full manual thematic or content analysis difficult within available time
When comparing vocabulary or theme changes across years
When examining themes or keyword distributions in large interview or meeting datasets

Participants and preparation

Staff handling preprocessing, aggregation and source checks, understanding software units and settings and data-use conditions

Bring the materials used for decisions, existing data, and differing stakeholder views to help complete the worksheet.

Suggested order

1. Basic informationDefine the analytical framework.
2. Preprocessing settingsRecord preprocessing for reproducibility.
3. Forced extraction and stop wordsRecord word settings for analytical accuracy.
4. Theme interpretation notesLink clusters to evaluation questions and always interpret against originals.
5. Completion check

Signs the worksheet is ready for review

Frequent words and vocabulary statistics
Co-occurrence network with clusters or subgraphs
Text-mining-assisted codebook linking words and themes

Cautions for review

Presenting a network without interpretation. Diagrams begin analysis; explain clusters' meanings and relevance to questions.
Entering personal or confidential data into generative AI. Direct cloud transmission of beneficiary responses risks privacy or confidentiality violations. Anonymization and organizational rule checks are mandatory.
Using unverified AI outputs as findings. LLM coding can hallucinate false content; source comparison and human verification are essential.

Read a completed fictional example (your entries will not change)

Fictional example; not actual results

A fictional illustrative extraction of “time” from invented open responses.

Basic information

Program and evaluation issue: Explore open application responses
Data type and count, such as 500 open responses: 3 fictional illustrative sentences
Tool: KH Coder / R tidytext, etc.: KH Coder 3 is a candidate.
Personal or confidential information and response: Invented examples. For actual data, remove identifiers and check use conditions.
Responsible staff and analysis period: Staff member A / July 2026 (fictional)

Reason for this entry: These short 3 sentences are not a practically sufficient corpus.

Preprocessing settings

Scroll horizontally to view the table

Setting	Value	Rationale
Co-occurrence unit	Plan to use each response document.	Do not create ties across respondents.
Negation	Check against originals.	Word presence alone can reverse meaning.

Reason for this entry: Retain reproducible settings.

Forced extraction and stop words

Scroll horizontally to view the table

Word	Type: forced extraction / exclusion	Selection rationale
Consultation desk	Candidate forced extraction	Do not split the concept.
Time	Do not exclude.	Important word with different meanings to compare in originals

Reason for this entry: Do not exclude solely for high frequency.

Theme interpretation notes

Scroll horizontally to view the table

Cluster or candidate theme	Frequent words included	Link to evaluation question	Source evidence supporting interpretation
Time burden of procedures	Time and entry	Conditions preventing applications	Not enough time, so stopped entering data (fictional). Do not group the other 2 sentences under this theme.

Reason for this entry: Link representative words to supporting sentences.

Completion check

☑ Checked: Checked information handling against organizational rules
☑ Checked: Recorded stop words, minimum frequencies and other preprocessing
☑ Checked: Interpreted networks against source texts
☐ Not checked: Verified AI outputs against originals where used
☑ Checked: Reported surface-pattern limitations
☐ Not checked: Recorded tool names, versions and settings in methods

Reason for this entry: No generative AI is used in this example. Record actual software versions and settings during execution.

Common errors in judgment

Reading co-occurring words as causation

How to revise: Treat them as occurring within the same unit.

Reason: Co-occurrence does not establish causal direction.

Accepting AI theme names without review

How to revise: Check source evidence and counterexamples for each theme.

Reason: Summaries may omit negation or minority voices.

Editorial and checking information

Content updated: 2026-09-13Source checked: 2026-09-13 (includes AI cross-checking)No documented human reviewIncludes fictional examplesEditorial policy and scope of checking Cite this page Report a correction

References and checked sections

KH Coder 3 Reference Manual
Koichi Higuchi
Section consulted
Part A: Co-Occurrence Network of Words
How it relates to the text
Networks depend on co-occurrence units and word or edge selection, so check settings and originals.
Source checked: 2026-09-13 (includes AI cross-checking)

How to cite this page

EvalCompass Editorial Team. “Text mining implementation and validation worksheet”. EvalCompass, updated 2026-09-13. https://evalcompass.jun-nakatani.com/en/worksheets/text-mining/

Add the date you accessed the page. Text created by this site is available under CC BY 4.0. Quotations, external materials, and bundled fonts have their own terms of use. When reusing content, credit the source, link to the license, and indicate any changes.

Suggest a correction or improvement to this page

Change history

2026-09-13 — Checked sources for definitions and applicability conditions, and added fictional examples, reasons for judgments and examples for every worksheet field. Revised unverified claims about history and adoption. Also reviewed existing procedures and cautions, and adjusted fixed participant counts and thresholds and generalizations about institutions.