Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations

By Aida Mostafazadeh Davani, Mark Díaz, Vinodkumar Prabhakaran. 2021

490 citations recorded by the source corpus.

Majority voting and averaging are common approaches used to resolve annotator disagreements and derive single ground truth labels from multiple annotations. However, annotators may systematically disagree with one another, often reflecting their individual biases and values, especially in the case of subjective tasks such as detecting affect, aggression, and hate speech. Annotator disagreements may capture important nuances in such tasks that are often ignored while aggregating annotations to a single ground truth. In order to address this, we investigate the efficacy of multi-annotator models. In particular, our multi-task based approach treats predicting each annotators’ judgements as separate subtasks, while sharing a common learned representation of the task. We show that this approach yields same or better performance than aggregating labels in the data prior to training across seven different binary classification tasks. Our approach also provides a way to estimate uncertainty in predictions, which we demonstrate better correlate with annotation disagreements than traditional methods. Being able to model uncertainty is especially useful in deployment scenarios where knowing when not to make a prediction is important.

Subfields: Artificial Intelligence, Sociology and Political Science, Evaluation & Benchmarks, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Evaluation & Benchmarks

In AI Ethics, Fairness & Society

Citations in the corpus

On Releasing Annotator-Level Labels and Information in Datasets (2021, methodology): “Finally, Davani et al. (2021) introduce a multi-annotator architecture that models each annotators’ perspectives separately using a multi-task approach.”
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023, background): “It is commonly believed that annotator (Geva et al., 2019; Sap et al., 2019; Davani et al., 2022; Sap et al., 2022) and data bias (Park et al., 2018; Dixon et al., 2018; Dodge et al., 2021; Harris et al., 2022) are the cause of this impact, and some studies have investigated the connection between…”
Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? (2022, background): “This issue could be tackled by a multi-annotator architecture [Davani et al. 2022] that captures the differences between annotators’ perspectives.”
Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, unknown): “…(Kasirzadeh & Gabriel, 2023) — we choose to average survey responses across humans within a country, which a simplifying assumption, but it is unclear what to do when people within a country have dissenting opinions (Gabriel & Ghazavi, 2021; Davani et al., 2022; Gordon et al., 2021; 2022).”
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (2023, background): “The majority-rule problem with RLHF can also be improved by algorithms that explicitly model multiple evaluators [Gordon et al., 2021, Davani et al., 2022, Daniels-Koch and Freedman, 2022, Gordon et al., 2022, Barnett et al., 2023], that tune models to individuals [Kumar et al., 2021], or that use…”
Evaluation Gaps in Machine Learning Practice (2022, background): “An array of recent scholarship on subjectivity in ML has “embraced disagreement” through practices of explicitly modeling—in both the data model and the ML model—inter-subject variation in interpretations [7, 12, 45, 48, 55].”
Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning (2021, background): “Discrepancies between learning problem datasets may also arise from inherent contextual differences - data sourced from differing geographies or cultural context [29, 138], in addition to annotators with inherently differing viewpoints regarding ground truth [27, 48].”
Whose Opinions Do Language Models Reflect? (2023, background): “Prior works show that for many tasks—from toxicity detection (Gordon et al., 2021, 2022; Davani et al., 2022; Sap et al., 2022; Goyal et al., 2022), ethics judgements (Lourie et al., 2021), and inference (Pavlick & Kwiatkowski, 2019)—there is inherent variability in what different humans consider…”
The Ghost in the Machine has an American accent: value conflict in GPT-3 (2022, background): “Such quantified methods, however, may be less likely to handle the nuance of value conflicts[59].”
Challenges and Strategies in Cross-Cultural NLP (2022, background): “In the case of subjective tasks such as detecting affect, aggression, and hate speech, annotators may systematically disagree with one another due to cultural differences that are often reflected by their biases and values (Davani et al., 2021).”