A Survey on Hate Speech Detection using Natural Language Processing

By Anna Schmidt, Michael Wiegand. 2017

1,390 citations recorded by the source corpus.

This paper presents a survey on hate speech detection. Given the steadily growing body of social media content, the amount of online hate speech is also increasing. Due to the massive scale of the web, methods that automatically detect hate speech are required. Our survey describes key areas that have been explored to automatically recognize these types of utterances using natural language processing. We also discuss limits of those approaches.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “While there is no single agreed definition of what constitutes hate speech or toxic speech (Fortuna and Nunes, 2018; Persily and Tucker, 2020; Schmidt and Wiegand, 2017), proposed definitions often include profanities, identity attacks, sleights, insults, threats, sexually explicit content,…”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “However, even state-of-the-art models exhibit substantial weaknesses (see Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018; Vidgen et al., 2019; Mishra et al., 2020, for reviews).”
The Risk of Racial Bias in Hate Speech Detection (2019, background): “A robust body of work has emerged trying to address the problem of hate speech and abusive language on social media (Schmidt and Wiegand, 2017).”
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “As the first concern, toxicity and unfair contents have been extensively studied, and relevant research like toxicity detection, detoxification, and bias mitigation have made considerable progress (Schmidt and Wiegand, 2017; Gehman et al., 2020; Welbl et al., 2021).”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “Several surveys cover approaches to this problem (Schmidt & Wiegand, 2017; Fortuna & Nunes, 2018; Vidgen et al., 2019), and there exist reviews of offensive language datasets (Fortuna et al., 2020; Vidgen & Derczynski, 2020).”
Recipes for Safety in Open-domain Chatbots (2020, background): “This narrow, Western-centric viewpoint will be insufficient for solving the issue in other languages and locales (Schmidt and Wiegand, 2017).”
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, background): “Hate speech There has been a lot of work in recent years on detecting hate speech in network science [61] and natural language processing [81, 65, 19].”
Bot-Adversarial Dialogue for Safe Conversational Agents (2021, background): “We also reiterate that the issue is further complicated by the fact that the very deﬁnition of “safe" is both contextually and culturally dependent (Schmidt and Wiegand, 2017).”
Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “Bias and toxicity detection Detection of hateful, abusive, or other toxic language has received increased attention recently (Schmidt and Wiegand, 2017), and most dataset creation work has cast this detection problem as binary classifcation (Waseem and Hovy, 2016; Davidson et al.”
Racial Bias in Hate Speech and Abusive Language Detection Datasets (2019, background): “While hate speech and abusive language detection has become an important area for natural language processing research (Schmidt and Wiegand, 2017; Waseem et al., 2017; Fortuna and Nunes, 2018), there has been little work addressing the potential for these systems to be biased.”
Hate Speech Dataset from a White Supremacy Forum (2018, background): “Schmidt and Wiegand (2017) acknowledge that whether a message contains hate speech or not can depend solely on the context, and thus encourage the inclusion of extra-linguistic features for annotation of hate speech.”
Challenges and frontiers in abusive content detection (2019, background): “Advances in machine learning and NLP have led to marked improvements in abusive content detection systems’ performance (Fortuna & Nunes, 2018; Schmidt & Wiegand, 2017).”
The Design and Implementation of XiaoIce, an Empathetic Social Chatbot (2018, methodology): “The neural response generator in XiaoIce follows the sequence-to-sequence (seq2seq) framework Sutskever et al. [2014], Cho et al.”
Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, methodology): “…of hate speech (Badjatiya et al., 2017; Bur-nap and Williams, 2016; Davidson et al., 2017; Gamb¨ack and Sikdar, 2017; Njagi et al., 2015; Schmidt and Wiegand, 2017; Vigna et al., 2017; Warner and Hirschberg, 2012), online harassment (Yin and Davison, 2009; Golbeck et al., 2017),…”
Understanding Abuse: A Typology of Abusive Language Detection Subtasks (2017, background): “There has been a surge in interest in the detection of abusive language, hate speech, cyberbullying, and trolling in the past several years (Schmidt and Wiegand, 2017).”
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus (2021, background): “several datasets and approaches being proposed in recent years (Schmidt and Wiegand, 2017; Mishra et al., 2019; Vidgen and Derczynski, 2020; Kiritchenko and Mohammad, 2018).”
Finding Microaggressions in the Wild: A Case for Locating Elusive Phenomena in Social Media Posts (2019, background): “profane, and hateful language (Schmidt and Wiegand, 2017), offensiveness spans a far larger spectrum that includes comments with more implicit and subtle signals that are no less offensive (Jurgens et al.”
One-step and Two-step Classification for Abusive Language Detection on Twitter (2017, background): “Many have addressed the difficulty of the definition of abusive language while annotating the data, because they are often subjective to individuals (Ross et al. 2016) and lack of context (Waseem and Hovy, 2016; Schmidt & Wiegand, 2017).”