Time of Your Hate: The Challenge of Time in Hate Speech Detection on Social Media

By Komal Florio, Valerio Basile, Marco Polignano, Pierpaolo Basile, Viviana Patti. 2020

104 citations recorded by the source corpus.

The availability of large annotated corpora from social media and the development of powerful classification approaches have contributed in an unprecedented way to tackle the challenge of monitoring users’ opinions and sentiments in online social platforms across time. Such linguistic data are strongly affected by events and topic discourse, and this aspect is crucial when detecting phenomena such as hate speech, especially from a diachronic perspective. We address this challenge by focusing on a real case study: the “Contro l’odio” platform for monitoring hate speech against immigrants in the Italian Twittersphere. We explored the temporal robustness of a BERT model for Italian (AlBERTo), the current benchmark on non-diachronic detection settings. We tested different training strategies to evaluate how the classification performance is affected by adding more data temporally distant from the test set and hence potentially different in terms of topic and language use. Our analysis points out the limits that a supervised classification model encounters on data that are heavily influenced by events. Our results show how AlBERTo is highly sensitive to the temporal distance of the fine-tuning set. However, with an adequate time window, the performance increases, while requiring less annotated data than a traditional classifier.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. William W. Cohen
2. Bhuwan Dhingra
3. Valerio Basile
4. Viviana Patti
5. Jacob Eisenstein
6. Paul Röttger
7. Janet B. Pierrehumbert
8. Julian Martin Eisenschlos

In Information Systems

1. Julian Martin Eisenschlos
2. Jacob Eisenstein
3. Viviana Patti
4. D. Gillick
5. Valerio Basile
6. Bhuwan Dhingra
7. William W. Cohen
8. Peter Christen

In Sociology and Political Science

1. Jacob Eisenstein
2. Viviana Patti
3. Janet B. Pierrehumbert
4. Paul Röttger
5. Valerio Basile
6. William W. Cohen

In Natural Language Processing

1. William W. Cohen
2. Valerio Basile
3. Bhuwan Dhingra
4. Viviana Patti
5. Paul Röttger
6. Jacob Eisenstein
7. Janet B. Pierrehumbert
8. Julian Martin Eisenschlos

In Safety, Robustness & Interpretability

1. Valerio Basile
2. Viviana Patti
3. Jacob Eisenstein
4. Manuela Sanguinetti
5. Paul Röttger
6. Fabio Poletto
7. Cristina Bosco
8. Bhuwan Dhingra

Citations in the corpus

• Time-Aware Language Models as Temporal Knowledge Bases (2021, background): “…have largely replaced non-contextual word embed-dings (Peters et al., 2018; Devlin et al., 2019), the main application of diachronic word embeddings is to detect and model lexical semantic changes (e.g., Frermann and Lapata, 2016), rather than to improve temporal awareness on downstream tasks.”
• Resources and benchmark corpora for hate speech detection: a systematic review (2020, background): “2019), or in restricted time spans (Florio et al. 2020).”
• Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media (2021, methodology): “, 2021) and other document classification tasks (e.g. Huang and Paul, 2018; Florio et al., 2020).”