Interrater Agreement Reconsidered: An Alternative to the rwg Indices

By Reagan D. Brown, Neil M. A. Hauenstein. 2005

311 citations recorded by the source corpus.

Subfields: Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d0516bdcf80a17c5977300010ab36e33f4484eae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Within-group agreement: On the use (and misuse) of rWG and rWG(J) in leadership research and some best practice guidelines. (2012, background): “Thus, it has been shown that blindly using the rectangular (uniform) null distribution may obscure the true distribution of members' responses (Brown & Hauenstein, 2005; Lebreton, Burgess, Kaiser, Atchley, & James, 2003).”
Effects of guided reflexivity and team feedback on team performance improvement: The role of team regulatory processes and cognitive emergent states (2015, methodology): “On average, across measures, aWG(J) values ranged from .71 to .83, indicating that the analyses at a team level were appropriate (cf. Brown & Hauenstein, 2005).”
Diversity in goal orientation, team reflexivity, and team performance (2011, background): “Awg(1) index has been proposed to overcome several limitations attached to the Rwg index (Brown & Hauenstein, 2005).”
Whence Differences in Value Priorities? (2011, methodology): “However, Brown and Hauenstein (2005) show that rwg(j) overestimates agreement.”
A Twenty-First Century Assessment of Values Across the Global Workforce (2011, background): “…autonomy 0.65 (0.06) 0.44–0.79 28 52 18 2 0 Egalitarianism 0.65 (0.08) 0.48–0.80 22 64 14 2 0 Harmony 0.59 (0.08) 0.35–0.79 6 46 34 14 0 a IRA level categories per Brown and Hauenstein (2005) and LeBreton et al. (2008) Sagie (2000) with the exception that they found significant positive…”
Answers to 20 Questions About Interrater Reliability and Interrater Agreement (2008, methodology): “Brown and Hauenstein (2005) extended Cohen’s kappa to the single target situation.”
An Exploratory Study of Group Development and Team Learning (2015, background): “However, Brown & Hauenstein (2005) consider an Rwg score larger than .80 as high agreement between team members about the construct.”
The Role of Character Strengths for Task Performance, Job Dedication, Interpersonal Facilitation, and Organizational Support (2014, background): “…assumption of the uniform null distribution (without bias; answering options have the same probability to be selected due to random response pattern) as the referring distribution for random answers was questioned (e.g., Brown & Hauenstein, 2005; Kotzlowski & Hattrup, 1992; LeBreton et al., 2003).”
Justifying Aggregation With Consensus-Based Constructs (2015, methodology): “In addition to rwg(J) estimates, we calculated the awg(J) index for each construct (Brown & Haunstein, 2005).”
Reliability and agreement of student ratings of the classroom environment: A reanalysis of TIMSS data (2007, background): “Various inconsistencies are apparent and further developments in both the theoretical framework and the indexes themselves can certainly be expected ( Brown & Hauenstein, 2005; LeBreton et al., 2005; Lindell, Brandt, & Whitney, 1999).”
Supplier traits for better customer firm innovation performance (2010, background): “75 there is a moderate and acceptable agreement of the customer and supplier ratings on the evaluation of their homophiliy (Brown and Hauenstein, 2005).”
Work-Unit Absenteeism: Effects of Satisfaction, Commitment, Labor Market Conditions, and Time (2008, background): “These authors developed 𝑎𝑎𝑤𝑤𝑤𝑤to overcome limitations associated with other agreement indices such as 𝑟𝑟𝑤𝑤𝑤𝑤 (i.e., scale dependency, sample size dependency, and potentially biasing uniform distribution assumptions [see Brown & Hauenstein, 2005:170]).”