Content analysis codebook and reliability worksheet
Record definitions, included and excluded examples and ambiguous-case rules. Use reliability checks appropriate to the purpose and approach; do not certify quality solely from codebooks or agreement coefficients.
Content analysis codebook and reliability worksheet
Basic information
Record the analytical framework.
Completed example for this section (fictional)
This fictional example is for explanation. Adapt it to your own program. Your entries will not change.
- Program and analysis issue
- Application barriers
- Data type and count, such as 400 open responses
- 10 open responses (fictional)
- Coding unit: sentence / paragraph / whole document
- Whole response per person; multiple categories allowed
- Coders and roles
- A and B independently classify the same 10 responses.
- Reliability method, such as comparison after independent coding
- Compare binary classifications by unit.
- Implementation period
- July 2026 (fictional)
Reason for this entry: Count presence or absence separately for each category.
Writing tips and explanation for this section
About this section
Record the analytical framework.
Count presence or absence separately for each category.
View the method explanationWriting tips
Follow each field’s heading to record the program name, focus, period, and other details. Leave undecided matters clearly marked as unresolved.
Read the full fictional exampleCoding frame: codebook
Record each category's definition, criteria and typical examples.
Completed example for this section (fictional)
This fictional example is for explanation. Adapt it to your own program. Your entries will not change.
Scroll horizontally to view the table
| Category ID | Category name | Definition: 1–2 sentences | Typical example: quotation | Atypical or excluded examples | Ambiguous-case handling |
|---|---|---|---|---|---|
| C1 | Operational barrier | Experience of procedures not progressing due to screen or entry operations | The screen went back during entry (fictional). | The screen was easy to use (fictional). | Leave unclear reasons pending and recheck originals. |
Reason for this entry: Define categories through paired included and excluded examples.
Scroll horizontally to view the table
| Category ID | Category name | Definition: 1–2 sentences | Typical example: quotation | Atypical or excluded examples | Ambiguous-case handling |
|---|---|---|---|---|---|
Writing tips and explanation for this section
About this section
Record each category's definition, criteria and typical examples.
Define categories through paired included and excluded examples.
View the method explanationWriting tips
Keep the information in each row aligned. Scroll wide tables horizontally to see every column.
Read the full fictional exampleInter-coder reliability check
Record coding results and agreement for the reliability sample.
Completed example for this section (fictional)
This fictional example is for explanation. Adapt it to your own program. Your entries will not change.
Scroll horizontally to view the table
| Category name | Coder A count | Coder B count | Agreement count | kappa / alpha | Coefficient and disagreement interpretation and judgment rationale |
|---|---|---|---|---|---|
| Operational barrier | 4/10 positive | 4/10 positive | 8/10: 3 positive and 5 negative agreements | κ = (0.8−0.52)/(1−0.52) ≈ 0.583 | Reconsider. Do not set universal pass/fail criteria from this small fictional sample. |
Reason for this entry: Show agreement breakdown because marginal counts alone cannot calculate kappa.
Scroll horizontally to view the table
| Category name | Coder A count | Coder B count | Agreement count | kappa / alpha | Coefficient and disagreement interpretation and judgment rationale |
|---|---|---|---|---|---|
Writing tips and explanation for this section
About this section
Record coding results and agreement for the reliability sample.
Show agreement breakdown because marginal counts alone cannot calculate kappa.
View the method explanationWriting tips
Keep the information in each row aligned. Scroll wide tables horizontally to see every column.
Read the full fictional exampleDisagreement discussion record
Record discussions of disagreement and use them to revise the codebook.
Completed example for this section (fictional)
This fictional example is for explanation. Adapt it to your own program. Your entries will not change.
Reason for this entry: Examine concepts rather than mechanically agreeing to raise coefficients.
Writing tips and explanation for this section
About this section
Record discussions of disagreement and use them to revise the codebook.
Examine concepts rather than mechanically agreeing to raise coefficients.
View the method explanationWriting tips
Use the example as a reference and record what you have established about your own program. Distinguish assumptions and unchecked matters from confirmed facts.
Read the full fictional exampleCompletion check
Completed example for this section (fictional)
This fictional example is for explanation. Adapt it to your own program. Your entries will not change.
- ☑ Checked: Defined every category with included and excluded examples
- ☑ Checked: Conducted coder training
- ☑ Checked: Checked reliability appropriate to the approach and, when using coefficients, recorded conditions and interpretation
- ☑ Checked: Reported reliability evidence suited to the approach, including coder count, conditions and interpretation when coefficients are used
- ☑ Checked: Defined category exclusivity or multiple-coding rules
- ☑ Checked: Stored coding linked to source data
Reason for this entry: Kappa is approximately 0.583 in this fictional example. Report calculation conditions, agreement breakdown and disagreements, and review category definitions.
Writing tips and explanation for this section
About this section
Kappa is approximately 0.583 in this fictional example. Report calculation conditions, agreement breakdown and disagreements, and review category definitions.
View the method explanationWriting tips
Read each item and check those you have confirmed. Do not use the fictional example’s checkbox states as findings about your own program.
Read the full fictional exampleBefore writing, check when to use the worksheet and what completion involves
Content analysis codebook and reliability worksheet is a worksheet for applying Content Analysis in meetings or individual work. For the overall duration of the method, use “Planning example, not a universal standard duration: 1–3 weeks for frame development, coder training, implementation and reliability checks” as a reference, adjusting for scale and resources. Fill in what you know provisionally and leave unresolved matters as questions to check next.
When this worksheet is useful
- When systematically categorizing many open responses, documents or reports to examine frequencies and trends
- When several coders must share analysis and reproducibility is required
- When comparing policy documents, minutes or media reports over time or across actors
Participants and preparation
1 analysis lead and 1–3 coders; multiple coders are ideal for reliability checks. For large datasets, assign dedicated coder training staff.
Bring the materials used for decisions, existing data, and differing stakeholder views to help complete the worksheet.
Suggested order
- 1. Basic informationRecord the analytical framework.
- 2. Coding frame: codebookRecord each category's definition, criteria and typical examples.
- 3. Inter-coder reliability checkRecord coding results and agreement for the reliability sample.
- 4. Disagreement discussion recordRecord discussions of disagreement and use them to revise the codebook.
- 5. Completion check
Signs the worksheet is ready for review
- Codebook: definitions, criteria and typical examples
- Inter-coder reliability: Cohen's kappa / Krippendorff's alpha
- Category counts and percentages for quantitative content analysis
Cautions for review
- Arbitrary coding due to inadequate definitions. An ambiguous category such as statements about difficulties produces differing coder judgments. Specify criteria and typical and atypical examples.
- Not reporting reliability evidence. For independent agreement, state coefficient calculation conditions and interpretation. For dialogue or consensus analysis, document deliberation and judgments; multiple analysts alone do not demonstrate quality.
- Non-exclusive categories complicate aggregation when 1 text unit can receive multiple codes. Refine coding units and category structure beforehand.
Read a completed fictional example (your entries will not change)
A fictional illustrative example where two people classify 10 open responses about applying for a consultation event.
Basic information
- Program and analysis issue
- Application barriers
- Data type and count, such as 400 open responses
- 10 open responses (fictional)
- Coding unit: sentence / paragraph / whole document
- Whole response per person; multiple categories allowed
- Coders and roles
- A and B independently classify the same 10 responses.
- Reliability method, such as comparison after independent coding
- Compare binary classifications by unit.
- Implementation period
- July 2026 (fictional)
Reason for this entry: Count presence or absence separately for each category.
Coding frame: codebook
Scroll horizontally to view the table
| Category ID | Category name | Definition: 1–2 sentences | Typical example: quotation | Atypical or excluded examples | Ambiguous-case handling |
|---|---|---|---|---|---|
| C1 | Operational barrier | Experience of procedures not progressing due to screen or entry operations | The screen went back during entry (fictional). | The screen was easy to use (fictional). | Leave unclear reasons pending and recheck originals. |
Reason for this entry: Define categories through paired included and excluded examples.
Inter-coder reliability check
Scroll horizontally to view the table
| Category name | Coder A count | Coder B count | Agreement count | kappa / alpha | Coefficient and disagreement interpretation and judgment rationale |
|---|---|---|---|---|---|
| Operational barrier | 4/10 positive | 4/10 positive | 8/10: 3 positive and 5 negative agreements | κ = (0.8−0.52)/(1−0.52) ≈ 0.583 | Reconsider. Do not set universal pass/fail criteria from this small fictional sample. |
Reason for this entry: Show agreement breakdown because marginal counts alone cannot calculate kappa.
Disagreement discussion record
Reason for this entry: Examine concepts rather than mechanically agreeing to raise coefficients.
Completion check
- ☑ Checked: Defined every category with included and excluded examples
- ☑ Checked: Conducted coder training
- ☑ Checked: Checked reliability appropriate to the approach and, when using coefficients, recorded conditions and interpretation
- ☑ Checked: Reported reliability evidence suited to the approach, including coder count, conditions and interpretation when coefficients are used
- ☑ Checked: Defined category exclusivity or multiple-coding rules
- ☑ Checked: Stored coding linked to source data
Reason for this entry: Kappa is approximately 0.583 in this fictional example. Report calculation conditions, agreement breakdown and disagreements, and review category definitions.
Common errors in judgment
Assigning the same category whenever the same word appears
How to revise: Set contextual and exclusion rules.
Reason: The same word can occur in negative statements or with different meanings.
Assuming agreement because two coders' totals match
How to revise: Check whether identical units received identical codes.
Reason: Totals may match despite different cases being classified.
References and checked sections
- Content AnalysisColumbia University Mailman School of Public Health
- Section consulted
- Conceptual Analysis / Reliability and Validity
- How it relates to the text
- Distinguish concept presence from word occurrences, and check category assignment rules and context.
How to cite this page
Change history
- — Checked sources for definitions and applicability conditions, and added fictional examples, reasons for judgments and examples for every worksheet field. Revised unverified claims about history and adoption. Also reviewed existing procedures and cautions, and adjusted fixed participant counts and thresholds and generalizations about institutions. Preserved field positions, types and counts while revising fixed conditions and thresholds in the forms.
- — Correction based on independent content review (content-check-causal). Replaced mandatory reliability coefficients with reporting evidence appropriate to the analytical approach.