Rater Agreement — Cohen's Kappa In Minutes

What's in the report?

Standard-library analysis: two raters, one categorical judgement per item — how much do they really agree? Map the two judgement columns and get the full confusion matrix as a heatmap, raw percent agreement, the agreement chance alone would produce, Cohen's kappa with its standard error and 95% confidence interval, a test of kappa against chance, linear- and quadratic-weighted kappa when the categories turn out to be ordered (detected from the labels, never assumed), a per-category breakdown of exactly where the raters diverge, and — when skewed marginals are depressing kappa — the kappa paradox explained with your own numbers rather than reported as a bad score.