Automated Hate Speech Detection and the Problem of Offensive Language
By Thomas Davidson, Dana Warmsley, Michael W. Macy, Ingmar Weber. 2017
3,212 citations recorded by the source corpus.
A key challenge for automatic hate-speech detection on social media is the separation of hate speech from other instances of offensive language. Lexical detection methods tend to have low precision because they classify all messages containing particular terms as hate speech and previous work using supervised learning has failed to distinguish between the two categories. We used a crowd-sourced hate speech lexicon to collect tweets containing hate speech keywords. We use crowd-sourcing to label a sample of these tweets into three categories: those containing hate speech, only offensive language, and those with neither. We train a multi-class classifier to distinguish between these different categories. Close analysis of the predictions and the errors shows when we can reliably separate hate speech from other offensive language and when this differentiation is more difficult. We find that racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive. Tweets without explicit hate keywords are also more difficult to classify.
Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Noah A. Smith
- Jason Aaron Edward Weston
- Percy Liang
- Jianfeng Gao
- Luke S. Zettlemoyer
- Graham Neubig
In Information Systems
- Bertie Vidgen
- Isar Nejadgholi
- Svetlana Kiritchenko
- Jason Aaron Edward Weston
- Yejin Choi
- Leon Derczynski
- Xiang Ren
- Ziqi Zhang
In Natural Language Processing
- Yejin Choi
- Mohit Bansal
- Jason Aaron Edward Weston
- Noah A. Smith
- Yulia Tsvetkov
- Douwe Kiela
- Emily Dinan
- Graham Neubig
In Safety, Robustness & Interpretability
- Isar Nejadgholi
- Svetlana Kiritchenko
- Yulia Tsvetkov
- Bertie Vidgen
- Yejin Choi
- Noah A. Smith
- Valerio Basile
- Maarten Sap
In AI Ethics, Fairness & Society
- Bertie Vidgen
- Valerio Basile
- Leon Derczynski
- Marcos Zampieri
- Isar Nejadgholi
- Viviana Patti
- Björn Gambäck
- Mladen Karan
Citations in the corpus
- Challenges in Detoxifying Language Models (2021, background): “While detecting hate speech and offensive language (Warner and Hirschberg, 2012; Kwok and Wang, 2013; Davidson et al., 2017; Zampieri et al., 2019), mostly in the context of online community moderation, has long been a subject of research; the”
- HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “B-D , B-F and P perform well for both hateful and non-hateful cases of profanity ( F10/11 ), which shows that they can distinguish between different uses of certain profanity terms.”
- The Risk of Racial Bias in Hate Speech Detection (2019, methodology): “To understand the racial and dialectic bias in toxic language detection, we focus our analyses on two corpora of tweets (Davidson et al., 2017; Founta et al., 2018) that are widely used in hate speech detection (Park et al.”
- Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “as predicting toxicity (Nobata et al., 2016; Davidson et al., 2017b; Xiang et al., 2021), or the factual accuracy of the outputs (Kryscinski et al.”
- ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “Examples are collected at scale using keyword-based scraping approaches (Waseem, 2016; Davidson et al., 2017; Zampieri et al., 2019), the bootstrapped scraping approaches (Founta et al.”
- Understanding Dataset Difficulty with V-Usable Information (2021, positive): “DWMW17 (Davidson et the conditional entropy above 0.30 upon omission.”
- The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, background): “Several text-only hate speech datasets have been released, mostly based on Twitter [83, 82, 11, 24, 20], and various architectures have been proposed for classifiers [45, 54, 53].”
- Ethical Challenges in Data-Driven Dialogue Systems (2017, methodology): “ases in several commonly used dialogue datasets. We leverage the linguistic bias detection framework of (Hutto, Folds, and Appling 2015) and the hate speech and offensive language detection model of (Davidson et al. 2017) to gather bias metrics on several popular dialogue datasets: Twitter (Ritter, ”
- Bot-Adversarial Dialogue for Safe Conversational Agents (2021, methodology): “Using three annotations per utterance is a method that is widely adopted by prior work on offensive language crowdsourcing tasks (Davidson et al., 2017; Chen et al., 2012; Zampieri et al., 2019) and also aligned with empirical studies on the diminishing return of increasing annotation size (Wulczyn…”
- Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, methodology): “Benchmark datasets Several benchmark datasets have been put forward for online hate classification (Waseem and Hovy, 2016; Waseem, 2016; Davidson et al., 2017; Founta et al., 2018; Mandl et al., 2019; Zampieri et al., 2019, 2020; Vidgen et al., 2020, 2021).”
- Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “Most previous approaches to understanding the implied harm in statements have cast this task as a simple toxicity classifcation (e.g., Waseem and Hovy, 2016; Founta et al., 2018; Davidson et al., 2017).”
- Racial Bias in Hate Speech and Abusive Language Detection Datasets (2019, background): “The classifier trained on Davidson et al. (2017) shows the largest disparities in Experiment 1, with tweets in the black-aligned corpus classified as hate speech and offensive language at substantially higher rates than white-aligned tweets.”
- Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior (2018, methodology): “Another dataset that has already used in different studies (e.g., (Malmasi and Zampieri 2017), (Olteanu, Talamadupula, and Varshney 2017)) is the one presented in (Davidson et al. 2017).”
- Hate Speech Dataset from a White Supremacy Forum (2018, background): “Some studies make use of Hatebase to build a classifier for hate speech (Davidson et al., 2017; Serra et al., 2017; Nobata et al., 2016).”
- Challenges and frontiers in abusive content detection (2019, methodology): “Researchers have also addressed numerous tasks beyond binary abusive content classification, including identifying the target of abuse and its strength as well as automatically moderating content (Burnap & Williams, 2016; Davidson, Warmsley, Macy, & Weber, 2017; Santos, Melnyk, & Padhi, 2018).”