Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior

By Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, Nicolas Kourtellis. 2018

824 citations recorded by the source corpus.

In recent years online social networks have suffered an increase in sexism, racism, and other types of aggressive and cyberbullying behavior, often manifesting itself through offensive, abusive, or hateful language. Past scientific work focused on studying these forms of abusive activity in popular online social networks, such as Facebook and Twitter. Building on such work, we present an eight month study of the various forms of abusive behavior on Twitter, in a holistic fashion. Departing from past work, we examine a wide variety of labeling schemes, which cover different forms of abusive behavior. We propose an incremental and iterative methodology that leverages the power of crowdsourcing to annotate a large collection of tweets with a set of abuse-related labels. By applying our methodology and performing statistical analysis for label merging or elimination, we identify a reduced but robust set of labels to characterize abuse-related tweets. Finally, we offer a characterization of our annotated dataset of 80 thousand tweets, which we make publicly available for further scientific exploration.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, background): “…in prompted generations with two metrics: 1) the expected maxi- 10Oversampling toxicity is necessary since it is a relatively rare phenomenon online (Founta et al., 2018). mum toxicity over k = 25 generations, which we estimate with a mean and standard deviation; and 2) the empirical probability…”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “B-D , B-F and P perform well for both hateful and non-hateful cases of profanity ( F10/11 ), which shows that they can distinguish between different uses of certain profanity terms.”
The Risk of Racial Bias in Hate Speech Detection (2019, methodology): “To understand the racial and dialectic bias in toxic language detection, we focus our analyses on two corpora of tweets (Davidson et al., 2017; Founta et al., 2018) that are widely used in hate speech detection (Park et al.”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “Examples are collected at scale us-ing keyword-based scraping approaches (Waseem, 2016; Davidson et al., 2017; Zampieri et al., 2019), the bootstrapped scraping approaches (Founta et al., 2018), and machine-vs-human adversarial data collection (Dinan et al., 2019; Vidgen et al., 2021), among others.”
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, background): “Several text-only hate speech datasets have been released, mostly based on Twitter [83, 82, 11, 24, 20], and various architectures have been proposed for classifiers [45, 54, 53].”
Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face (2023, background): “This is especially problematic in light of the potential real-world harms that ensue [8, 9, 13, 31].”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, methodology): “Benchmark datasets Several benchmark datasets have been put forward for online hate classification (Waseem and Hovy, 2016; Waseem, 2016; Davidson et al., 2017; Founta et al., 2018; Mandl et al., 2019; Zampieri et al., 2019, 2020; Vidgen et al., 2020, 2021).”
Reducing Gender Bias in Abusive Language Detection (2018, background): “Hosseini et al. (2017) have shown how to easily cause false predictions with adversarial examples in Google’s API, and Dixon et al.”
Challenges and frontiers in abusive content detection (2019, background): “For instance, Founta et al. shared a dataset of 80,000 tweets but soon after this was reduced to 70,000 (Founta et al., 2018; Lee et al., 2018).”
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus (2021, methodology): “Most of this research is carried out on data extracted from social media sources such as Twitter (Founta et al., 2018; Basile et al., 2019; Waseem and Hovy, 2016) and Reddit (Tadesse et al., 2019; Farrell et al., 2019), with both ML-based (Badjatiya et al., 2017) and count-based approaches…”
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection (2020, methodology): “We collect our dataset from sources where previous studies on hate speech have been conducted: Twitter (Davidson in our paper as they are arguably very similar (Founta et al. 2018). et al. 2017; Fortuna and Nunes 2018) and Gab (Lima et al. 2018; Mathew et al. 2020; Zannettou et al. 2018).”
Challenges in Automated Debiasing for Toxic Language Detection (2021, methodology): “Next, as per our heuristic, we only relabel toxic AAE tweets whose WAE translation is predicted as non-toxic by either our vanilla classiﬁer trained on the original Founta et al. (2018) dataset, or an identical classiﬁer trained on the WAE translated tweets.”
Detection of Abusive Language: the Problem of Biased Datasets (2019, methodology): “For the creation of manually annotated datasets, randomly sampling microposts from large social media platforms typically results in a too small proportion of abusive comments (Wulczyn et al., 2017; Founta et al., 2018).”
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2021, methodology): “First, we select posts that are automatically detected as AAE and/or vulgar from Founta et al. (2018), using the lexical detector of AAE by (Blodgett et al., 2016) and the vulgar wordlist by Zhou et al. (2021).”
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021, background): “The majority are skewed towards explicitly abusive text since they were originally seeded with hate lexicons (Basile et al., 2019; Founta et al., 2018; Davidson et al., 2017; Waseem and Hovy, 2016), racial identiﬁers (Warner and Hirschberg, 2012), or explicitly hateful phrases such as “I hate…”