Abusive Language Detection in Online User Content

By Chikashi Nobata, Joel R. Tetreault, Achint Oommen Thomas, Yashar Mehdad, Yi Chang. 2016

1,243 citations recorded by the source corpus.

Detection of abusive language in user generated online content has become an issue of increasing importance in recent years. Most current commercial methods make use of blacklists and regular expressions, however these measures fall short when contending with more subtle, less ham-fisted examples of hate speech. In this work, we develop a machine learning based method to detect hate speech on online user comments from two domains which outperforms a state-of-the-art deep learning approach. We also develop a corpus of user comments annotated for abusive language, the first of its kind. Finally, we use our detection tool to analyze abusive language over time and in different settings to further enhance our knowledge of this behavior.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, neutral): “F29 – leet speak : Resembles “obfuscations” (Nobata et al., 2016; van Aken et al., 2018).”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “…like lexicons (such as hatebase) (Xiang et al., 2012; Dadvar et al., 2012; Burnap and Williams, 2015; Liu and Forss, 2015), n-grams, capitalization/punctuation details (Chen et al., 2012; Waseem and Hovy, 2016; Nobata et al., 2016), among others (Xu et al., 2012; Burnap and Williams, 2016).”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, background): “Models trained on historical data may also not be effective for present-day hate classification models given how quickly online conversations evolve (Nobata et al., 2016).”
Ex Machina: Personal Attacks Seen at Scale (2016, background): “The Wikimedia Foundation found that 54% of those who had experienced online harassment expressed decreased participation in the project where they experienced the harassment [23].”
Automated Hate Speech Detection and the Problem of Offensive Language (2017, background): “While the classiﬁer performs well at prevalent forms of hate speech, particularly anti-black racism and homophobia, but is less reliable at detecting types of hate speech that occur infrequently, a problem noted by Nobata et al. (2016).”
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter (2016, methodology): “We follow Nobata et al. (2016), in their use of the length of comments in tokens, and the average length of the words in a tweet.”
A Survey on Hate Speech Detection using Natural Language Processing (2017, methodology): “Nobata et al. (2016) use a similar feature.”
Hate Speech Dataset from a White Supremacy Forum (2018, methodology): “Some studies make use of Hatebase to build a classifier for hate speech (Davidson et al., 2017; Serra et al., 2017; Nobata et al., 2016).”
Challenges and frontiers in abusive content detection (2019, background): “Supposedly humorous, ironic or sarcastic abusive content is often viewed as a source of classification error (Nobata, Thomas, Mehdad, Chang, & Tetreault, 2016; van Aken, Risch, Krestel, & Löser, 2018).”
Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, background): “…by either meth-ods including manual feature engineering (Burnap and Williams, 2015; Mehdad and Tetreault, 2016; Waseem, 2016; Davidson et al., 2017; Nobata et al., 2016; Kennedy et al., 2017; Samghabadi et al., 2017; Robinson et al., 2018) or the use of (deep) neural networks (Ptaszynski et al.,…”
Understanding Abuse: A Typology of Abusive Language Detection Subtasks (2017, methodology): “Additionally, character n-grams have been shown to be apt for abusive language tasks due to their ability to capture variation of words associated with abuse (Nobata et al., 2016; Waseem, 2016a).”
Predicting the Type and Target of Offensive Posts in Social Media (2019, background): “Most prior work focuses on a different aspect of offensive language such as abusive language (Nobata et al., 2016; Mubarak et al., 2017), (cyber-)aggression (Kumar et al., 2018), (cyber-)bullying (Xu et al., 2012; Dadvar et al., 2013), toxic comments , hate speech (Kwok and Wang, 2013; Djuric et…”
Revealing Persona Biases in Dialogue Systems (2021, neutral): “Offensiveness Offensiveness overlaps with concepts of abusive language (Nobata et al. 2016), toxicity (Dixon et al. 2018), hate speech (Warner and Hirschberg 2012), and conversational agent safety (Dinan et al. 2019).”
One-step and Two-step Classification for Abusive Language Detection on Twitter (2017, neutral): “Nobata et al. (2016) combined pre-defined language elements and word embedding to train a regression model.”
Using Convolutional Neural Networks to Classify Hate-Speech (2017, background): “Looking at another set of AMT-annotated Yahoo! news posts, Nobata et al. (2016) experimented with several different word-internal, n-gram-based, syntactic, and distributional semantic features, concluding that character n-grams alone contribute sufﬁciently strongly for an online gradient descent…”
All You Need is "Love": Evading Hate Speech Detection (2018, background): “As stand-alone features, their performance is weak in comparison to n-grams [20, 24].”
Resources and benchmark corpora for hate speech detection: a systematic review (2020, background): “Nobata et al. (2016), for example, use news comments reported as ‘‘abusive’’ by users, but rely as well on both expert judges and crowdsourcing for validation.”
Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer (2018, methodology): “…work in the past have attacked this problem by using different machine learning models to detect abusive behavior (Xiang et al., 2012; Warner and Hirschberg, 2012; Kwok and Wang, 2013; Wang et al., 2014; Nobata et al., 2016; Burnap and Williams, 2015; Davidson et al., 2017; Founta et al., 2018).”
Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language (2018, background): “With regard to traditional feature engineering, the features found effective very much reflect the insights of recent research on English data, particulary the extensive study presented in Nobata et al. (2016).”