Detecting Offensive Language in Social Media to Protect Adolescent Online Safety

By Ying Chen, Yilu Zhou, Sencun Zhu, Heng Xu. 2012

679 citations recorded by the source corpus.

Since the textual contents on online social media are highly unstructured, informal, and often misspelled, existing research on message-level offensive language detection cannot accurately detect offensive content. Meanwhile, user-level offensiveness detection seems a more feasible approach but it is an under researched area. To bridge this gap, we propose the Lexical Syntactic Feature (LSF) architecture to detect offensive content and identify potential offensive users in social media. We distinguish the contribution of pejoratives/profanities and obscenities in determining offensive content, and introduce hand-authoring syntactic rules in identifying name-calling harassments. In particular, we incorporate a user's writing style, structure and specific cyber bullying content as features to predict the user's potentiality to send out offensive content. Results from experiments showed that our LSF framework performed significantly better than existing methods in offensive content detection. It achieves precision of 98.24% and recall of 94.34% in sentence offensive detection, as well as precision of 77.9% and recall of 77.8% in user offensive detection. Meanwhile, the processing speed of LSF is approximately 10msec per sentence, suggesting the potential for effective deployment in social media.

Subfields: Artificial Intelligence, Communication, Information Systems, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Information Systems

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Bot-Adversarial Dialogue for Safe Conversational Agents (2021, methodology): “…three annotations per utterance is a method that is widely adopted by prior work on offensive language crowdsourcing tasks (Davidson et al., 2017; Chen et al., 2012; Zampieri et al., 2019) and also aligned with empirical studies on the diminishing return of increasing annotation size (Wulczyn et…”
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter (2016, methodology): “Chen et al. (2012a) address this by using lexical and syntactical features along with automatically generated black lists.”
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter (2016, background): “, 2012; Chen et al., 2012). These methods traditionally suffer from a poor recall and do not address hate speech. While Sood et al. (2012) incorporate edit distances to find variants of slurs, they are not able to find terms that do not occur in these lists.”
A Survey on Hate Speech Detection using Natural Language Processing (2017, methodology): “Taking into account deeper syntactic information as a feature, Chen et al. (2012) employ typed”
Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, background): “…online harassment (Yin and Davison, 2009; Golbeck et al., 2017), abusive language (Mehdad and Tetreault, 2016; Park and Fung, 2017), cyberbullying (Dad-var et al., 2013; Dinakar et al., 2012; Hee et al., 2015; Zhong et al., 2016) and offensive language (Chen et al., 2012; Xiang et al., 2012).”
Cyber Hate Speech on Twitter: An Application of Machine Classification and Statistical Modeling for Policy and Decision Making (2015, background): “Chen et al. (2012) identified offensive content by using profanities, obscenities, and pejorative terms as features, weighted accordingly based on the associated strength of the term, as well as references to people.”
Using Convolutional Neural Networks to Classify Hate-Speech (2017, background): “Moving away from features based solely on the language used in online messages, Chen et al. (2012) proposed a model also taking into account the posting patterns of the users in order to single out persons exhibiting abusive behaviour.”
All You Need is "Love": Evading Hate Speech Detection (2018, background): “Of linguistic features applied to hate speech classification, the most common have been part-of-speech tags and dependency relations [5, 6, 20].”
Exploring the Role of Grammar and Word Choice in Bias Toward African American English (AAE) in Hate Speech Classification (2022, background): “These include, simple surface features (such as word and character n-grams) [4, 5, 9, 27, 40, 42], word generalization [11, 27], sentiment analysis [40], lexical resources”
Detecting Offensive Content in Open-domain Conversations using Two Stage Semi-supervision (2018, neutral): “There has also been a line of work on extracting user level features to identify people exhibiting abusive behavior (Chen et al., 2012; Buckels et al., 2014; Papegnies et al., 2017).”
Hunting for Troll Comments in News Community Forums (2016, methodology): “, 2014), and lexico-syntactic features about user’s writing style (Chen et al., 2012).”
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021, background): “Offensive Language Detection Ad hominems occur in many forms and are related to different types of offensive language, including abusive language (Yin et al., 2009; Chen et al., 2012; Nobata et al., 2016), hate speech (Warner and Hirschberg, 2012; Kwok and Wang, 2013; Djuric et al.”
Mean Birds: Detecting Aggression and Bullying on Twitter (2017, background): “n of such behavior. Detection. Over the past couple of years, a few techniques have been proposed that aim to detect offensive or abusive content/behavior in platforms such as Instagram [16], YouTube [6], Yahoo Finance [10], and Yahoo Answers [18]. More speciﬁcally, Chen et al. [6] use both textual ”
Characterizing Linguistic Attributes for Automatic Classification of Intent Based Racist/Radicalized Posts on Tumblr Micro-Blogging Website (2017, methodology): “We remove all punctuations and special characters (tagged as PUNC) from semantically tagged content and decode all remaining terms with their respective labels in tags’ hierarchy 13 .”
Large Language Model Alignment: A Survey (2023, background): “Offensive language pertains to the deployment of injurious articulates in a sacrilegious, extremely discourteous, impolite, or crude fashion, aiming to derogate the specified individual or group (Chen et al., 2012; Razavi et al., 2010).”