Kappa Statistic is not Satisfactory for Assessing the Extent of Agreement Between Raters

By K. Gwet. 2002

281 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/7786ba48592a8b6ae773a8385a156154e02f4534

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Kevin A. Hallgren
2. Christin Hoffmann
3. S. Skippon
4. M. White
5. Susan J. Ball
6. C. Abraham
7. Klaus Krippendorff
8. Matthijs J. Warrens

Citations in the corpus

• Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial (2012, background): “Two well-documented effects can substantially cause Cohen’s kappa to misrepresent the IRR of a measure (Di Eugenio & Glass, 2004, Gwet, 2002), and two kappa variants have been developed to accommodate these effects.”
• What cognitive mechanisms predict travel mode choice? A systematic review with meta-analysis (2017, methodology): “The AC1 statistic (Gwet, 2002) was used to calculate the degree of agreement between two coders and a score of 0.93 indicated good inter-rater reliability.”
• Commentary: A Dissenting View on So-Called Paradoxes of Reliability Coefficients (2013, methodology): “Gwet’s (2002) AC1, intended to overcome the paradox of “high agreement but low reliability,” has to be disqualifi ed as well.”