A Framework for Evaluation and Use of Automated Scoring

By David M. Williamson, Xiaoming Xi, F. Breyer. 2012

382 citations recorded by the source corpus.

Subfields: Management Science and Operations Research, Statistics and Probability, Surgery

View Paper: https://www.semanticscholar.org/paper/2ecb7975d72c5dd64ddf783b5eabaa1cea7db57a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

1. Benno Stein
2. Nitin Madnani
3. Iryna Gurevych
4. Anastassia Loukina
5. Ou Lydia Liu
6. Klaus Zechner
7. Alina von Davier
8. Marcia C. Linn

In Statistics and Probability

1. Benno Stein
2. John Blackmore
3. Nitin Madnani
4. Iryna Gurevych
5. C. Brew
6. Anastassia Loukina
7. Ou Lydia Liu
8. Steven Burrows

In Surgery

1. Nitin Madnani
2. Anastassia Loukina
3. Klaus Zechner

Citations in the corpus

• The Eras and Trends of Automatic Short Answer Grading (2015, background): “Yet another comparison is content versus expression and fluency (Williamson et al. 2012).”
• Automated Scoring of Constructed-Response Science Items: Prospects and Obstacles (2014, methodology): “Most previous studies used the following criteria to evaluate the accuracy of c-rater scoring: quadratic-weighted kappa (Fleiss & Cohen, 1973), Pearson correlation, degradation of human/machine agreement from human/human agreement, and standardized mean score differences between automated and human ”
• The many dimensions of algorithmic fairness in educational applications (2019, methodology): “Absolute values above 0.1 are considered an issue of concern and require further evaluation (Williamson et al., 2012).”
• Building Better Open-Source Tools to Support Fairness in Automated Scoring (2017, methodology): “The measurement guidelines currently implemented in RSMTool follow the psychometric framework suggested by Williamson et al. (2012).”