Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter

By Zeerak Talat. 2016

665 citations recorded by the source corpus.

Hate speech in the form of racism and sexism is commonplace on the internet (Waseem and Hovy, 2016). For this reason, there has been both an academic and an industry interest in detection of hate speech. The volume of data to be reviewed for creating data sets encourages a use of crowd sourcing for the annotation efforts. In this paper, we provide an examination of the inﬂuence of annotator knowledge of hate speech on classiﬁcation models by comparing classiﬁcation results obtained from training on expert and amateur annotations. We provide an evaluation on our own data set and run our models on the data set released by Waseem and Hovy (2016). We ﬁnd that amateur annotators are more likely than expert annotators to label items as hate speech, and that systems trained on expert annotations outperform systems trained on amateur annotations.

Subfields: Artificial Intelligence, Communication, Law, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Law

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, methodology): “Although widely used, the PERSPECTIVE API and other hate speech detection systems and corpora exhibit biases against minorities and suffer from low agreement in annotations (Waseem, 2016; Ross et al., 2017), partially due to annotator identity influencing their perception of hate speech (Cowan and K”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “, 2020), and are known to exhibit annotator biases (Waseem, 2016; Sap et al., 2019) as well as topic and author biases (Wiegand”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Annotators often have different perceptions of concepts like race or are influenced by their own lived experience [234] when categorizing protected categories [187].”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, background): “Examples are collected at scale us-ing keyword-based scraping approaches (Waseem, 2016; Davidson et al., 2017; Zampieri et al., 2019), the bootstrapped scraping approaches (Founta et al., 2018), and machine-vs-human adversarial data collection (Dinan et al., 2019; Vidgen et al., 2021), among others.”
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, background): “Several text-only hate speech datasets have been released, mostly based on Twitter [83, 82, 11, 24, 20], and various architectures have been proposed for classifiers [45, 54, 53].”
GeDi: Generative Discriminator Guided Sequence Generation (2020, background): “Toxicity detection is difficult as toxicity labelling is subjective and often has poor annotator agreement (Waseem, 2016; Ross et al., 2017).”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, methodology): “Benchmark datasets Several benchmark datasets have been put forward for online hate classification (Waseem and Hovy, 2016; Waseem, 2016; Davidson et al., 2017; Founta et al., 2018; Mandl et al., 2019; Zampieri et al., 2019, 2020; Vidgen et al., 2020, 2021).”
CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, positive): “Waseem [59] demonstrated that incorporating feminist and antiracist activists’ perspectives into hate speech annotations yielded better aligned models.”
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science (2018, methodology): “experts) on the effectiveness of models trained on the annotations, Waseem and Hovy (2016) performed a scrape of Twitter data using contentious terms and topics.”
Automated Hate Speech Detection and the Problem of Offensive Language (2017, neutral): “This ﬁnding is consistent with previous work that has found amateur coders to often be un-reliable at identifying abusive content (Nobata et al. 2016; Waseem 2016).”
Racial Bias in Hate Speech and Abusive Language Detection Datasets (2019, methodology): “We see similar results for Waseem and Hovy (2016) and Waseem (2016).”
Reducing Gender Bias in Abusive Language Detection (2018, background): “Waseem and Hovy (2016); Waseem (2016), the creators of the dataset, describe “sexist” and “racist” languages as specific subsets of abusive language.”
Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, methodology): “While there are a number of annotated public datasets in adjacent ﬁelds, such as hate speech (Ross et al., 2016; Gao and Huang, 2017), racism/sexism (Waseem, 2016; Waseem and Hovy, 2016) or harassment (Golbeck et al., 2017)”
Understanding Abuse: A Typology of Abusive Language Detection Subtasks (2017, background): “80 ) (Ross et al., 2016; Waseem, 2016a; Tulkens et al., 2016; Bretschneider and Peters, 2017) are obtained.”
One-step and Two-step Classification for Abusive Language Detection on Twitter (2017, methodology): “We used the two English Twitter Datasets (Waseem and Hovy, 2016; Waseem, 2016) published as unshared tasks for the 1 Workshop on Abusive Language Online(ALW1).”
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2021, methodology): “Following crowdsourcing setups in prior work (Waseem, 2016; Davidson et al., 2017; Wulczyn et al., 2017; Founta et al., 2018; Sap et al., 2019), we asked three fine-grained questions to annotators for each post in both our studies: • “How toxic/hateful/disrespectful or offensive does this post seem ”
Demoting Racial Bias in Hate Speech Detection (2020, background): “Waseem (2016) show that non-experts are more likely to label text as abusive than expert annotators, and Sap et al. (2019) show how lack of social context in annotation tasks further increases the risk of annotator bias, which can in turn lead to the marginalization of racial minorities.”
Using Convolutional Neural Networks to Classify Hate-Speech (2017, methodology): “all four Convolutional Neural Network (CNN) models are shown in Table 2, and compared to the Logistic Regression (LogReg) model used by Waseem and Hovy (2016).”
A Survey of Race, Racism, and Anti-Racism in NLP (2021, background): “• Annotator selection Waseem (2016) show that feminist/anti-racist activists assign different offensive language labels to tweets than figure-eight workers, demonstrating that annotators’ lived experiences affect data annotations.”