Interobserver Agreement in Behavioral Research: Importance and Calculation

By Marley W. Watkins, Miriam Pacheco. 2000

241 citations recorded by the source corpus.

Subfields: Developmental and Educational Psychology, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/0c429d128574de889c72d2c206e09fd3bab827f4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Developmental and Educational Psychology

1. S. Methe
2. Randy G. Floyd
3. A. Bruhn
4. John M. Hintze
5. Shanna Eisner Hirsch
6. S. Zuniga
7. J. Herman
8. Kathleen Lynne Lane

In Management Science and Operations Research

1. S. Zuniga
2. J. Herman
3. Kathleen Lynne Lane
4. N. Webb
5. Randy G. Floyd
6. John M. Hintze

In Statistics and Probability

1. S. Methe
2. Randy G. Floyd
3. John M. Hintze
4. S. Zuniga
5. J. Herman
6. N. Webb
7. Theodore J. Christ
8. S. Ardoin

Citations in the corpus

• A Review of Tier 2 Interventions Conducted Within Multitiered Models of Behavioral Prevention (2014, background): “This strength is tempered, however, by only one study reporting a kappa value, which is a more conservative estimate accounting for chance agreement (Watkins & Pacheco, 2001).”
• Validation and Decision Accuracy of Early Numeracy Skill Indicators (2008, background): “To adjust for chance agreement in hit rates and to further examine general levels of agreement, kappa indices can be interpreted much like correlation coefficients (Watkins & Pacheco, 2000).”
• Measurement Issues in the Alignment of Standards and Assessments (2007, methodology): “To examine the consistency among raters for the categorical ratings (topic, dimensionality), we calculated kappa coefficients of agreement (Cohen, 1960; Fleiss, 1971; Watkins & Pacheco, 2001).”