Challenges in discriminating profanity from hate speech

By Shervin Malmasi, Marcos Zampieri. 2017

261 citations recorded by the source corpus.

ABSTRACT In this study, we approach the problem of distinguishing general profanity from hate speech in social media, something which has not been widely considered. Using a new dataset annotated specifically for this task, we employ supervised classification along with a set of features that includes -grams, skip-grams and clustering-based word representations. We apply approaches based on single classifiers as well as more advanced ensemble classifiers and stacked generalisation, achieving the best result of accuracy for this 3-class classification task. Analysis of the results reveals that discriminating hate speech and profanity is not a simple task, which may require features that capture a deeper understanding of the text not always possible with surface -grams. The variability of gold labels in the annotated data, due to differences in the subjective adjudications of the annotators, is also an issue. Other directions for future work are discussed.

Subfields: Artificial Intelligence, Communication, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “F11 – non-hateful uses of profanity : Oversensi-tiveness of hate speech detection models to profan-ity (Davidson et al., 2017; Malmasi and Zampieri, 2018; van Aken et al., 2018).”
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, background): “Several text-only hate speech datasets have been released, mostly based on Twitter [83, 82, 11, 24, 20], and various architectures have been proposed for classifiers [45, 54, 53].”
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval) (2019, background): “Examples of offensive content investigated in previous studies include hate speech (Davidson et al., 2017; Malmasi and Zampieri, 2017, 2018), cyber-bulling (Dinakar et al., 2011), and aggression (Kumar et al., 2018).”
Predicting the Type and Target of Offensive Posts in Social Media (2019, background): “…(Kumar et al., 2018), (cyber-)bullying (Xu et al., 2012; Dadvar et al., 2013), toxic comments , hate speech (Kwok and Wang, 2013; Djuric et al., 2015; Burnap and Williams, 2015; Davidson et al., 2017; Malmasi and Zampieri, 2017, 2018), and offensive language (Wiegand et al., 2018).”
Resources and benchmark corpora for hate speech detection: a systematic review (2020, background): “Partial attempts to precisely classify overlapping abusive phenomena are found in the literature, such as Malmasi and Zampieri (2018) exploring the distinction between HS and profanity.”
Aggression Identification Using Deep Learning and Data Augmentation (2018, background): “hate speech is given by Malmasi and Zampieri (2018). The results demonstrate that it can be hard to distinguish between overt and covert aggression in social media.”
Aggression Detection in Social Media: Using Deep Neural Networks, Data Augmentation, and Pseudo Labeling (2018, background): “In (Malmasi and Zampieri, 2018), it was concluded that subtle distinction between types of anti-social behaviour: profanity and hate speech is a difficult task for machine learning classifiers.”
A Unified Deep Learning Architecture for Abuse Detection (2018, background): “Hate speech detection [2, 12, 25, 40, 42], cyberbullying identification [6, 11, 16], and the detection of abusive [7, 10, 27] or offensive language [8, 26, 43], are some of the facets of this problem.”
Misogyny Detection in Twitter: a Multilingual and Cross-Domain Study (2020, unknown): “However, they could also lead to false positives when the swear word is used in a casual, not offensive context Malmasi and Zampieri (2018); Nobata, Tetreault, Thomas, Mehdad, and Chang (2016); Van Hee et al. (2018).”
Directions in Abusive Language Training Data: Garbage In, Garbage Out (2020, background): “However, creating training datasets is often a laborious and non-trivial task – and creating datasets which are non-biased, large and theoretically-informed is even more difficult ([71] p. 189).”
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media (2019, methodology): “ such as bag of words, word-level and character-level n-grams, etc. have proven to be the most predictive features [11,13,22]. Apart from features, dierent algorithms such as Support Vector Machines [10], Naive Baye [16], and Logistic Regression [3,22], etc. have been applied for classication purpos”
Benchmarking Aggression Identification in Social Media (2018, background): “…and Varma, 2017; Burnap and Williams, 2014; Davidson et al., 2017; Vigna et al., 2017; Djuric et al., 2015; Fortana, 2017; Gitari et al., 2015; Malmasi and Zampieri, 2018; Waseem and Hovy, 2016; Schmidt and Wie-gand, 2017), radicalization (Agarwal and Sureka, 2015; Agarwal and Sureka,…”
HateBERT: Retraining BERT for Abusive Language Detection in English (2020, unknown): “(2020) introduce a graphical visualisation (Figure 1) of the connections among abusive language phenomena according to the definitions in previous work (Waseem and Hovy, 2016; Fortuna and Nunes, 2018; Malmasi and Zampieri, 2018; Basile et al., 2019; Zampieri et al., 2019).”