Automated Essay Scoring in the Presence of Biased Ratings

By Evelin Amorim, Márcia Cançado, Adriano Veloso. 2018

86 citations recorded by the source corpus.

Studies in Social Sciences have revealed that when people evaluate someone else, their evaluations often reflect their biases. As a result, rater bias may introduce highly subjective factors that make their evaluations inaccurate. This may affect automated essay scoring models in many ways, as these models are typically designed to model (potentially biased) essay raters. While there is sizeable literature on rater effects in general settings, it remains unknown how rater bias affects automated essay scoring. To this end, we present a new annotated corpus containing essays and their respective scores. Different from existing corpora, our corpus also contains comments provided by the raters in order to ground their scores. We present features to quantify rater bias based on their comments, and we found that rater bias plays an important role in automated essay scoring. We investigated the extent to which rater bias affects models based on hand-crafted features. Finally, we propose to rectify the training set by removing essays associated with potentially biased scores while learning the scoring model.

Subfields: Artificial Intelligence, General Social Sciences, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

Measuring Bias in Contextualized Word Representations (2019, background): “…resolution (Rudinger et al., 2018) and language identiﬁcation (Jurgens et al., 2017), to downstream systems such as automated essay scoring (Amorim et al., 2018), exhibit inherent social biases which are attributed to the datasets used to train the embeddings (Barocas and Selbst, 2016;…”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, background): “However, their team suspects that irrelevant factors may influence scorers’ judgments [6, 81].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “For example, automatic essay scoring has been found to be affected by rater bias (Amorim et al., 2018) and spurious correlations like essay length (Perelman, 2014), leading to problems with subpopulation shift; and these systems would also ideally generalize across different educational contexts,…”
Quantifying Social Biases in Contextual Word Representations (2019, background): “…resolution (Rudinger et al., 2018) and language identiﬁcation (Jurgens et al., 2017), to downstream systems such as automated essay scoring (Amorim et al., 2018), exhibit inherent social biases which are attributed to the datasets used to train the embeddings (Barocas and Selbst, 2016;…”
Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers (2021, background): “After the seminal work of Bolukbasi et al. (2016), bias has been found ubiquitous in word embedding models (Amorim et al., 2018; Brunet et al., 2018; Rudinger et al., 2018; Zhao et al., 2017; Costa-jussà et al., 2019; Silva et al., 2020).”