Directions in abusive language training data, a systematic review: Garbage in, garbage out

By Bertie Vidgen, Leon Derczynski. 2020

357 citations recorded by the source corpus.

Data-driven and machine learning based approaches for detecting, categorising and measuring abusive content such as hate speech and harassment have gained traction due to their scalability, robustness and increasingly high performance. Making effective detection systems for abusive content relies on having the right training datasets, reflecting a widely accepted mantra in computer science: Garbage In, Garbage Out. However, creating training datasets which are large, varied, theoretically-informed and that minimize biases is difficult, laborious and requires deep expertise. This paper systematically reviews 63 publicly available training datasets which have been created to train abusive language classifiers. It also reports on creation of a dedicated website for cataloguing abusive language data hatespeechdata.com. We discuss the challenges and opportunities of open science in this field, and argue that although more dataset sharing would bring many benefits it also poses social and ethical risks which need careful consideration. Finally, we provide evidence-based recommendations for practitioners creating new abusive content training datasets.

Subfields: Artificial Intelligence, Communication, General Health Professions, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In General Health Professions

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “speech datasets vary in data source, sampling strategy and annotation process (Vidgen and Derczynski, 2020; Poletto et al., 2020), and are known to exhibit annotator biases (Waseem, 2016; Sap et al.”
A Holistic Approach to Undesired Content Detection in the Real World (2022, neutral): “For instance, examples from our production traffic are usually much longer and contain few-shot prompts, whereas existing public NLP datasets are usually shorter and often crawled from Wikipedia, Twitter, etc. (Vidgen and Derczynski 2020).”
Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering (2022, background): “, 2021) and of problems such as bias replication (Binns et al., 2017; Davidson et al., 2019; Vidgen and Derczynski, 2020; Sap et al., 2019; Tsvetkov, 2020).”
COLD: A Benchmark for Chinese Offensive Language Detection (2022, methodology): “Following the annotation suggestions provided by Vidgen and Derczynski (2020), we iteratively develop guidelines and train annotators to ensure the quality of annotation.”
Predicting Anti-Asian Hateful Users on Twitter during COVID-19 (2021, methodology): “While the keyword-based method can result in a skewed dataset due to the oversampling of certain keywords (e.g., n-word) (Vidgen and Derczynski, 2020), since we collect all tweets containing Asian slurs to study users, not tweets, we believe the sampling bias would not be a critical issue in our…”
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective (2020, methodology): “Vidgen and Derczynski (2020) enumerated the sources of inconsistencies in creating abusive language datasets, highlighted the barriers to data sharing and the lack of infrastructure needed for open-source research, and recommended a set of best practices for data sampling and annotation to improve…”