The Restriction of Variance Hypothesis and Interrater Reliability and Agreement: Are Ratings from Multiple Sources Really Dissimilar?

By James M. LeBreton, Jennifer R. D. Burgess, R. Kaiser, Kate Atchley, Lawrence R. James, Ed Lawler. 2003

284 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

From orientation to behavior: The interplay between learning orientation, open-mindedness, and psychological safety in team learning (2019, background): “Indeed, our ICC(2) coefficients are influenced by the relatively small size of some of our participating teams and variance restriction (LeBreton et al., 2003; Shieh, 2016).”
Answers to 20 Questions About Interrater Reliability and Interrater Agreement (2008, methodology): “However, LeBreton et al. (2003) demonstrated how it is possible to have high levels of IRA yet low levels of IRR and IRR + IRA.”
The Sources of Four Commonly Reported Cutoff Criteria (2006, methodology): “Others have noted similar misinterpretations (e.g., A. Cohen et al., 2001; LeBreton et al., 2003; Wright et al., 2001).”
Multiple Informant Methodology: A Critical Review and Recommendations (2010, methodology): “…interrater reliability are not commonly used in multiple informant research (LeBreton and Senter 2008), we will present several measures for interrater agreement in the following while we refer to the extant literature for details on measures for interrater reliability (e.g., LeBreton et al. 2003).”