Do Characters Abuse More Than Words?

By Yashar Mehdad, Joel R. Tetreault. 2016

190 citations recorded by the source corpus.

Although word and character n-grams have been used as features in different NLP applications, no systematic comparison or analysis has shown the power of character-based features for detecting abusive language . In this study, we investigate the effectiveness of such features for abusive language detection in user-generated online comments

Subfields: Artificial Intelligence, Communication

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

Citations in the corpus

Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, methodology): “…Wiegand, 2017; Vigna et al., 2017; Warner and Hirschberg, 2012), online harassment (Yin and Davison, 2009; Golbeck et al., 2017), abusive language (Mehdad and Tetreault, 2016; Park and Fung, 2017), cyberbullying (Dad-var et al., 2013; Dinakar et al., 2012; Hee et al., 2015; Zhong et al., 2016)…”
All You Need is "Love": Evading Hate Speech Detection (2018, neutral): “larities between different spelling variants [16].”
Studying Generalisability across Abusive Language Detection Datasets (2019, background): “…to their ability to perform better in an online setting where spelling errors and adversarial behaviour are commonplace, character-level features have been endorsed (Mehdad and Tetreault, 2016), and also shown to often be superior to word-level information for this task (Meyer and Gambäck, 2019).”
Tackling Online Abuse: A Survey of Automated Abuse Detection Methods (2019, methodology): “Experimenting with the DATA - YAHOO - FIN - DJ dataset, Mehdad and Tetreault (2016) investigated whether character-level features are more indicative of abuse than word-level ones.”
The Linguistic Ideologies of Deep Abusive Language Classification (2018, result): “…ideology, they are “implicitly” offensive), the majority of the errors are due to unconventional ways of spelling, a known problem already being tack-led by previous researchers who convincingly argue for character-level as opposed to word-level approaches (Mehdad and Tetreault, 2016).”
Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media (2018, background): “Hate speech detection has been supplemented by a variety of features including lexical properties such as n-gram features (Nobata et al. 2016), character n-gram features (Mehdad and Tetreault 2016), average word embeddings, and paragraph embeddings (No-bata et al. 2016; Djuric et al. 2015).”
CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech (2019, background): “, 2012), or to utilize deep neural networks (Mehdad and Tetreault, 2016; Badjatiya et al., 2017).”
Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network (2018, methodology): “These methods can be divided into two categories: one relies on manual feature engineering that are then consumed by algorithms such as SVM, Naive Bayes, and Logistic Regression [4,7,9,16,24,26,25,27] (classic methods); the other represents the more recent deep learning paradigm that employs neural ”
DeepHate: Hate Speech Detection via Multi-Faceted Text Representations (2020, methodology): “These methods can be broadly groupedintotwocategories: (i)methodsthatadoptedclassicmachine-learning strategies [9–11, 30, 41, 42], and more recently, (ii) deep learning-based methods [2, 3, 12, 15, 17, 21, 28, 32, 47].”
Detect All Abuse! Toward Universal Abusive Language Detection Models (2020, methodology): “Although some of the baseline models such as OTH, MFR also combine two embedding approaches (Chars2vec and Glove) to get more information, they still just consider the general information of the texts rather than extract information in a targeted fashion from various aspects.”
A Unified Deep Learning Architecture for Abuse Detection (2018, methodology): “Hate speech detection [2, 12, 25, 40, 42], cyberbullying identification [6, 11, 16], and the detection of abusive [7, 10, 27] or offensive language [8, 26, 43], are some of the facets of this problem.”
Challenges and frontiers in abusive content detection (2019, background): “A more promising way of addressing this is to model language at the character or subword level (Devlin, Chang, Lee, & Toutanova, 2018; Mehdad & Tetreault, 2016).”
Cross-Domain Detection of Abusive Language Online (2018, methodology): “Some approaches (Badjatiya et al., 2017; Park and Fung, 2017; Mehdad and Tetreault, 2016) also rely on combining different types of models.”
A Survey on Hate Speech Detection using Natural Language Processing (2017, methodology): “Among the more recent methods, deep learning with Recurrent Neural Network Language Models has been employed in Mehdad and Tetreault (2016).”
Comparative Studies of Detecting Abusive Language on Twitter (2018, background): “Other researchers have investigated characterlevel representations and their effectiveness compared to word-level representations (Mehdad and Tetreault, 2016; Park and Fung, 2017).”
A Survey on Automatic Detection of Hate Speech in Text (2018, background): “Character N-gram features proved to be more predictive than token N-gram features, for the specific problem of abusive language detection [56].”
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media (2019, methodology): “ detect online hate speech, a large number of scientic studies have been dedicated by using Natural Language Processing (NLP) in combination with Machine Learning (ML) and Deep Learning (DL) methods [1,8,11,13,22,25]. Although supervised machine learning-based approaches have used dierent text minin”
Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter (2018, methodology): “Simple surface features such as bag of words, word and character n-grams have been used as fundamental features in hate speech detection [2, 3, 9, 16, 20, 37–40], as well as other related tasks such as the detection of offensive and abusive content [5, 24, 27], discrimination [42], and cyberbullying”