Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach

By Jan Kocoń, Alicja Figas, Marcin Gruza, Daria Puchalska, Tomasz Kajdanowicz, Przemyslaw Kazienko. 2021

114 citations recorded by the source corpus.

Abstract Analysis of subjective texts like offensive content or hate speech is a great challenge, especially regarding annotation process. Most of current annotation procedures are aimed at achieving a high level of agreement in order to generate a high quality reference source. However, the annotation guidelines for subjective content may restrict the annotators’ freedom of decision making. Motivated by a moderate annotation agreement in offensive content datasets, we hypothesize that personalized approaches to offensive content identification should be in place. Thus, we propose two novel perspectives of perception: group-based and individual. Using demographics of annotators as well as embeddings of their previous decisions (annotated texts), we are able to train multimodal models (including transformer-based) adjusted to personal or community profiles. Based on the agreement of individuals and groups, we experimentally showed that annotator group agreeability strongly correlates with offensive content recognition quality. The proposed personalized approaches enabled us to create models adaptable to personal user beliefs rather than to agreed offensiveness understanding. Overall, our individualized approaches to offensive content classification outperform classic data-centric methods that generalize offensiveness perception and it refers to all six tested models. Additionally, we developed requirements for annotation procedures, personalization and content processing to make the solutions human-centered.

Subfields: Artificial Intelligence, Social Psychology, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “Context dependency of whether an utterance is “toxic” The views about what constitutes unacceptable “toxic speech” differ between individuals and social groups (Kocoń et al., 2021).”
Taxonomy of Risks posed by Language Models (2022, background): “Mitigation is further complicated by the context dependency of what constitutes profoundly offensive speech [80, 92, 105].”
Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation (2022, background): “This paper continues to build upon research in this space of creating groups of annotators based on some differentiating factor(s) [3, 16, 22, 41].”