Predicting the Type and Target of Offensive Posts in Social Media

By Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, Ritesh Kumar. 2019

905 citations recorded by the source corpus.

As offensive content has become pervasive in social media, there has been much research in identifying potentially offensive messages. However, previous work on this topic did not consider the problem as a whole, but rather focused on detecting very specific types of offensive content, e.g., hate speech, cyberbulling, or cyber-aggression. In contrast, here we target several different kinds of offensive content. In particular, we model the task hierarchically, identifying the type and the target of offensive messages in social media. For this purpose, we complied the Offensive Language Identification Dataset (OLID), a new dataset with tweets annotated for offensive content using a fine-grained three-layer annotation scheme, which we make publicly available. We discuss the main similarities and differences between OLID and pre-existing datasets for hate speech identification, aggression detection, and similar tasks. We further experiment with and we compare the performance of different machine learning models on OLID.

Subfields: Artificial Intelligence, Social Psychology, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “F10 – hate expressed using profanity : Reﬁnes more general “insult” categories (Davidson et al., 2017; Zampieri et al., 2019).”
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “While most research has concentrated on single-sentence toxic content, including explicit insults (Wulczyn et al., 2017; David-son et al., 2017; Zampieri et al., 2019; Rosen-thal et al., 2021) or more covert offenses (Wang and Potts, 2019; Breitfeller et al., 2019; Han and Tsvetkov, 2020; Price et…”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “Examples are collected at scale us-ing keyword-based scraping approaches (Waseem, 2016; Davidson et al., 2017; Zampieri et al., 2019), the bootstrapped scraping approaches (Founta et al., 2018), and machine-vs-human adversarial data collection (Dinan et al., 2019; Vidgen et al., 2021), among others.”
Bot-Adversarial Dialogue for Safe Conversational Agents (2021, methodology): “…per utterance is a method that is widely adopted by prior work on offensive language crowdsourcing tasks (Davidson et al., 2017; Chen et al., 2012; Zampieri et al., 2019) and also aligned with empirical studies on the diminishing return of increasing annotation size (Wulczyn et al., 2017).”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, background): “The taxonomy for the type of hate draws on and extends previous work, including Waseem and Hovy (2016); Vidgen et al. (2019a); Zampieri et al. (2019).”
Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “Similarly, Zampieri et al. (2019) hierarchically annotate a dataset of tweets with offensiveness and whether a group or individual is targeted.”
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval) (2019, methodology): “A detailed description of the data collection process and annotation is presented in Zampieri et al. (2019).”
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2021, background): “Yet, most previous research has treated this detection as a simple classification with one correct label, obtained by averaging judgments by a small set of human raters per post (Waseem and Hovy, 2016; Wulczyn et al., 2017; Davidson et al., 2017; Founta et al., 2018; Zampieri et al., 2019).”
Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology (2020, methodology): “…Dataset (HSDD) [12] and the Multilingual Detection of Hate Speech (MDHS) task at SemEval 2019 [5] that support the development of methods for detecting hate speech; and the Offensive Language Identification Dataset (OLID) [61] that can be used to develop methods that classify offensive posts.”
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) (2020, methodology): “The second place team, PGSG, used a complex teacher-student architecture built on top of a BERT-LSTM model, which was ﬁne-tuned on the SOLID dataset in an unsupervised way, i.e., optimizing for the MLM objective.”
Resources and benchmark corpora for hate speech detection: a systematic review (2020, methodology): “Zampieri et al. (2019a) (ZMNRFK)—14,100 tweets in English, annotated with crowdsourcing using a multi-level scheme.”
XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages (2020, background): “Another typology focusing more on the nature of targets of abusive texts was proposed by Zampieri et al. (2019).”
Joint Modelling of Emotion and Abusive Language Detection (2020, methodology): “OffensEval 2019 (OffensEval) This dataset is from SemEval 2019 - Task 6: OffensEval 2019 - Identifying and Categorizing Offensive Language in Social Media (Zampieri et al., 2019a,b).”
Red Alarm for Pre-trained Models: Universal Vulnerabilities by Neuron-Level Backdoor Attacks (2021, methodology): “For the evaluation of NLP PTMs, we use SST-2 (Socher et al., 2013), OLID (Zampieri et al., 2019), and Enron (Metsis et al., 2006).”
ToxCCIn: Toxic Content Classification with Interpretability (2021, methodology): “More recently, large-scale pre-trained language models such as Bidirectional Encoder Representation from Transformer (BERT) (Devlin et al., 2019) have shown great advantages in toxicity detection (Zampieri et al., 2019b, 2020), by learning contextual word embeddings instead of static embeddings.”
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases (2021, background): “Furthermore, most of the prior work relies on finetuning (Sap et al., 2020; Mandl et al., 2019; Zampieri et al., 2019) neural architectures which is costly in case of large LMs (Strubell et al.”
Introducing CAD: the Contextual Abuse Dataset (2021, methodology): “Zampieri et al. (2019) use a hierarchical three-level approach to annotation, separating (a) offensive from not-offensive tweets, (b) offensive into targeted and untargeted statements and (c) for targeted statements, identiﬁcation of what is attacked (group, individual or other).”
A taxonomy, data set, and benchmark for detecting and classifying malevolent dialogue responses (2021, methodology): “The offensive language identification data set (OLID) data set released by Zampieri et al. (2019) has two categories, “ offensive ” and “ not offensive ” ; it contains 13,240 tweets, 3,942 of which are “ offensive.”