The Gab Hate Corpus: A collection of 27k posts annotated for hate speech

By Brendan Kennedy, Mohammad Atari, Aida Mostafazadeh Davani, Leigh Yeh, Ali Omrani, Y. Kim, K. Coombs, Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez and 11 more. 2018

82 citations recorded by the source corpus.

The growing prominence of online hate speech is a threat to a safe and just society. This endangering phenomenon requires collaboration across the sciences in order to generate evidence-based knowledge of, and policies for, the dissemination of hatred in online spaces. To foster such collaborations, here we present the Gab Hate Corpus (GHC), consisting of 27,665 posts from the social network service gab.ai, each annotated by a minimum of three trained annotators. Annotators were trained to label posts according to a coding typology derived from a synthesis of hate speech definitions across legal, computational, psychological, and sociological research. We detail the development of the corpus, describe the resulting distributions of hate-based rhetoric, target group, and rhetorical framing labels, and establish baseline classification performance for each using standard natural language processing methods. The GHC, which is the largest theoretically-justified, annotated corpus of hate speech to date, provides opportunities for training and evaluating hate speech classifiers and for scientific inquiries into the linguistic and network components of hate speech.

Subfields: Artificial Intelligence, Communication, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Refining Language Models with Compositional Explanations (2021, methodology): “4, we also conduct experiment on Stormfront → GHC by varying the number of explanations.”
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021, methodology): “Inferiority language is also related to assaults on human dignity (Kennedy et al., 2018), dominance (Saha et al., 2018), and declarations of superiority of the in-group (Fortuna and Nunes, 2018).”
Detecting Hate Speech with GPT-3 (2021, background): “One definition is ‘the intentional verbalization of prejudice against a social group’ (Kennedy et al., 2018).”
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021, methodology): “We regularize importance score on 25 group identiﬁers in (Kennedy et al., 2018) for GHC and Stormfront.”
Handling Bias in Toxic Speech Detection: A Survey (2022, methodology): “The BERT models trained on the GHC [48]and Stormfront [22] datasets with regularised loss showed a reduction in FPR on the test datasets.”
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations (2021, methodology): “In other words, ﬁne-tuning each annotator The average and standard deviation of precision, recall, and f-score of model predictions on the GHC dataset, evaluated during 5 iterations of 5-fold stratiﬁed cross validation.”
A New Generation of Perspective API: Efficient Multilingual Character-level Transformers (2022, background): “It is also worth noting the progress towards a more comprehensive taxonomy of abusive content has drawn interdisciplinary attention, and systemic annotation efforts [14] which also inform our work.”