Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis

By Björn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, Michael Wojatzki. 2016

434 citations recorded by the source corpus.

Some users of social media are spreading racist, sexist, and otherwise hateful content. For the purpose of training a hate speech detection system, the reliability of the annotations is crucial, but there is no universally agreed-upon definition. We collected potentially hateful messages and asked two groups of internet users to determine whether they were hate speech or not, whether they should be banned or not and to rate their degree of offensiveness. One of the groups was shown a definition prior to completing the survey. We aimed to assess whether hate speech can be annotated reliably, and the extent to which existing definitions are in accordance with subjective ratings. Our results indicate that showing users a definition caused them to partially align their own opinion with the definition but did not improve reliability, which was very low overall. We conclude that the presence of hate speech should perhaps not be considered a binary yes-or-no decision, and raters need more detailed instructions for the annotation.

Subfields: Artificial Intelligence, Communication, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Challenges in Detoxifying Language Models (2021, neutral): “The score indicates fair overall agreement, and is comparable to the level of agreement reported in prior work (Ross et al., 2016; Wulczyn et al., 2017).”
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, methodology): “Although widely used, the PERSPECTIVE API and other hate speech detection systems and corpora exhibit biases against minorities and suffer from low agreement in annotations (Waseem, 2016; Ross et al., 2017), partially due to annotator identity influencing their perception of hate speech (Cowan and K”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Existing datasets are often biased due to their curation process (Dixon et al., 2018; Wiegand et al., 2019; Geva et al., 2019; Sap et al., 2021; Kryscinski et al., 2020) and can have unreliable annotations (Ross et al., 2017; Field and Tsvetkov, 2020b; Pagnoni et al., 2021).”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, neutral): “8 We ﬁnd that annotators agreed moderately and are higher than or equal rates to prior work on hate speech annotation (Ross et al., 2017; Sap et al., 2020), with a Fleiss’ κ =0.46 (Fleiss, 1971) and Krippendorff’s α =0.64 (Krippen-8 Speciﬁcally, we take the max of the HARMFUL I F AI and HARMFUL I F…”
GeDi: Generative Discriminator Guided Sequence Generation (2020, background): “Toxicity detection is difficult as toxicity labelling is subjective and often has poor annotator agreement (Waseem, 2016; Ross et al., 2017).”
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter (2016, background): “Waseem and Hovy (2016) and Ross et al. (2016) focus on building corpora which they annotate for containing hate speech.”
A Survey on Hate Speech Detection using Natural Language Processing (2017, background): “Ross et al. (2016) point out that this is particularly a problem for hate speech detection.”
Hate Speech Dataset from a White Supremacy Forum (2018, background): “Ross et al. (2016) also studied the reliability of hate speech annotations and acknowledge the importance of having detailed instructions for the annotation of hate speech available.”
Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, background): “While there are a number of annotated public datasets in adjacent ﬁelds, such as hate speech (Ross et al., 2016; Gao and Huang, 2017), racism/sexism (Waseem, 2016; Waseem and Hovy, 2016) or harassment (Golbeck et al., 2017)”
Understanding Abuse: A Typology of Abusive Language Detection Subtasks (2017, background): “, 2017), others find that they do not improve the reliability of non-expert classifications (Ross et al., 2016).”
Challenges in Automated Debiasing for Toxic Language Detection (2021, neutral): “First, compared to these NLU tasks where there is one correct label, the toxicity of language is inherently more nuanced, subjective, and contextual, which causes toxic language datasets to have lower agreement in general (Ross et al., 2017).”
One-step and Two-step Classification for Abusive Language Detection on Twitter (2017, background): “Many have addressed the difficulty of the definition of abusive language while annotating the data, because they are often subjective to individuals (Ross et al. 2016) and lack of context (Waseem and Hovy, 2016; Schmidt & Wiegand, 2017).”
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2021, background): “Table 1: All 15 posts for our breadth-of-workers study. of toxicity or hate speech which previous work has shown still suffers from large annotator disagreement (Ross et al., 2017).”
Using Convolutional Neural Networks to Classify Hate-Speech (2017, background): “However, Ross et al. (2016) had a set of 541 German tweets annotated, in particular addressing the”
Resources and benchmark corpora for hate speech detection: a systematic review (2020, background): “Ross et al. (2017) (RRCCKW)—541 tweets in German, annotated with the labels ‘‘ HS/ no HS ’’ and with a discrete value for offensiveness on a 1–6 rating scale.”
Handling Bias in Toxic Speech Detection: A Survey (2022, background): “Meanwhile, existing datasets display a variety of biases via their collection pipeline [24, 113] and are subject to unreliable annotations [81].”