Cohen's Kappa Calculator

This calculator computes Cohen's kappa (κ), a statistic that measures inter-rater agreement for categorical items while accounting for agreement that would occur by chance. It is commonly used to assess reliability between two raters classifying items into binary categories.

Enter the 2×2 agreement table

Enter the counts for each combination of rater classifications

Both Positive (a) Rater 1: +, Rater 2: +
Rater 1 +, Rater 2 − (b) Disagreement
Rater 1 −, Rater 2 + (c) Disagreement
Both Negative (d) Rater 1: −, Rater 2: −

Formula

Cohen's Kappa

\( \kappa = \frac{P_o - P_e}{1 - P_e} \)

where \( P_o \) is observed agreement and \( P_e \) is expected agreement by chance.

For a 2×2 Table

\( P_o = \frac{a + d}{n} \)
\( P_e = \frac{(a+b)(a+c) + (c+d)(b+d)}{n^2} \)

Interpreting Kappa Values

The Landis and Koch (1977) interpretation scale is commonly used:

  • κ < 0 — Poor (less than chance agreement)
  • 0.00–0.20 — Slight agreement
  • 0.21–0.40 — Fair agreement
  • 0.41–0.60 — Moderate agreement
  • 0.61–0.80 — Substantial agreement
  • 0.81–1.00 — Almost perfect agreement

Note: These thresholds are guidelines, not strict cutoffs. Context matters.

When to Use Cohen's Kappa

Cohen's kappa is appropriate when:

  • Two raters independently classify the same set of items
  • Categories are mutually exclusive and exhaustive
  • Raters are fixed (the same two raters for all items)

For more than two raters, consider Fleiss' kappa. For ordinal categories, consider weighted kappa.

Assumptions & Limitations

  • Assumes independent ratings (raters do not influence each other)
  • Kappa can be paradoxically low when prevalence is very high or very low
  • Sensitive to marginal distributions (prevalence and bias)
  • Standard error assumes large sample approximation
  • Not appropriate for comparing raters with different marginal rates

Related Calculators

Simplify Your Entire Manuscript Workflow with Livewrite

Livewrite works directly inside Microsoft Word to support every stage of manuscript preparation—from first draft to final submission.

Why Livewrite

  • • Write and edit in Word with structured, journal-aware assistance that improves clarity, flow, and scientific rigor.
  • • Cite as you write by searching 240+ million publications that match any sentence or paragraph, previewing the most relevant citations, and inserting them instantly—fully compatible with Zotero and Mendeley.
  • • Instantly format your manuscript for any journal, including title pages, abstracts, headings, tables, figures, and references. (premium subscription)
  • • Automated peer review before submission to flag missing requirements, reporting guideline gaps, and common reviewer concerns—helping prevent avoidable delays and desk rejections.

Draft, edit, cite, review, and format—without leaving Word.

Try Livewrite - for free