The Stability of Rater Severity in Large‐Scale Assessment Programs

By P. Congdon, Joy MeQueen. 2000

118 citations recorded by the source corpus.

Subfields: Developmental and Educational Psychology, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/3eba2e13f5e370938aada3d6c281a514b6595e76

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Developmental and Educational Psychology

1. Gad S. Lim
2. U. Knoch
3. Jeff Johnson
4. Jacquelynne S. Eccles
5. Ulrich Trautwein
6. C. Elder
7. George Leckie
8. J. Baird

In Management Science and Operations Research

1. Gad S. Lim
2. U. Knoch
3. Wolfgang Wagner
4. Jacquelynne S. Eccles
5. Ulrich Trautwein
6. C. Elder
7. George Leckie
8. Richard Göllner

Citations in the corpus

• Students’ Idiosyncratic Perceptions of Teaching Quality in Mathematics: A Result of Rater Tendency Alone or an Expression of Dyadic Effects Between Students and Teachers? (2018, background): “Most research on rating tendencies has focused on characteristics of raters, the rating task, or the person to be appraised (Congdon & McQueen, 2000; Freedman, 1979; Hoyt, 2000; Villanova, Bernardin, Dahmus, & Sims, 1993).”
• Diagnostic Writing Assessment: The Development and Validation of a Rating Scale (2009, background): “Weigle (2002) and other authors (e.g. Congdon & McQueen, 2000; Lumley & McNamara, 1995) also suggest regular re-standardisation sessions.”
• Rater types in writing performance assessments: A classification approach to rater variability (2008, unknown): “188, 58084 Hagen, Germany; email: thomas.eckes@testdaf.de Congdon & McQueen, 2000; Engelhard & Myford, 2003; Eckes, 2005b; Schoonen, 2005).”
• Measuring and promoting inter-rater agreement of teacher and principal performance ratings. (2012, background): “Researchers have found short training sessions to be ineffective at calibration (bringing evaluators in line with expert ratings or one another) and unlikely to produce consistent results (Barrett, 2001; Congdon & McQueen, 2000).”
• Rater Effects on Essay Scoring: A Multilevel Analysis of Severity Drift, Central Tendency, and Rater Experience (2011, methodology): “A related issue raised by the Congdon and McQueen (2000) study and by subsequent researchers (Harik et al., 2009; Hoskens & Wilson, 2001; Lamprianou, 2006) is whether rater severity effects are stable over time.”
• The influence of rater language background on writing performance assessment (2009, background): “…such possible sources of variation as the professional background of raters (Barnwell, 1989; Brown, 1995; Shohamy, Gordon, & Kraemer, 1992), their relative severity in rating (Englehard, 1994; Lumley & McNamara, 1995), and the stability of their ratings over time (Congdon & McQueen, 2000).”
• Evaluating rater responses to an online training program for L2 writing assessment (2007, background): “Indeed, in the absence of a controlled experiment, we cannot be certain that changes observed in the current study were not simply due to random variations in rater behaviour over time, of the kind documented by Congdon and McQueen (2000). Further research exploring the interactions between differen”