Latent Hatred: A Benchmark for Understanding Implicit Hate Speech

By Mai Elsherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, M. de Choudhury, Diyi Yang. 2021

377 citations recorded by the source corpus.

Hate speech has grown significantly on social media, causing serious consequences for victims of all demographics. Despite much attention being paid to characterize and detect discriminatory speech, most work has focused on explicit or overt hate speech, failing to address a more pervasive form based on coded or indirect language. To fill this gap, this work introduces a theoretically-justified taxonomy of implicit hate speech and a benchmark corpus with fine-grained labels for each message and its implication. We present systematic analyses of our dataset using contemporary baselines to detect and explain implicit hate speech, and we discuss key features that challenge existing models. This dataset will continue to serve as a useful benchmark for understanding this multifaceted issue.

Subfields: Artificial Intelligence, Communication, Language and Linguistics, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Language and Linguistics

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

ChatGPT outperforms crowd workers for text-annotation tasks (2023, background): “Such data may be available for high-level tasks such as semantic evaluation(1) and hate speech (ElSherief et al., 2021), as well as, sometimes, more specific tasks such as party ideology (Herrmann and Döring, 2023).”
Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech (2023, methodology): “We call these implied statements the human-generated NLEs as explained in [8].”
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models (2023, methodology): “In addition to Regard classifier, we utilized HateBERT(ElSherief et al., 2021) and RoBERTa-ToxiGen (Hartvigsen et al., 2022) classifiers.”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “…of the out-of-the-box models to those ﬁne-tuned on T OXI G EN on three publicly available human-written datasets (I MPLICIT H ATE C ORPUS (ElSherief et al., 2021), the S OCIAL B IAS F RAMES test set (Sap et al., 2020), and D YNA H ATE (Vidgen et al., 2021)) as well as the evaluation…”
Explaining Toxic Text via Knowledge Enhanced Text Generation (2022, background): “Results on the implicit hate corpus (ElSherief et al., 2021) are given in Table 4.”
Chain of Explanation: New Prompting Method to Generate Quality Natural Language Explanation for Implicit Hate Speech (2022, methodology): “Thus, more recent works were proposed to apply the Generative Pre-trained models to create hate explanations [7].”
Large Language Model Alignment: A Survey (2023, background): “ImplicitHateCorpus (ElSherief et al., 2021) introduces a groundbreaking benchmark corpus for implicit hate speech on Twitter.”
Handling Bias in Toxic Speech Detection: A Survey (2022, background): “Recent studies in implicit hate speech [27, 42] attempt to bridge this gap; however, such practice in computational methods for toxicity detection is not yet well adopted.”
SafeText: A Benchmark for Exploring Physical Safety in Language Models (2022, background): “Other research focuses on speciﬁc aspects of safety such as hate speech, gang violence, and online van-dalism (ElSherief et al., 2021; Chang et al., 2018; Wang and McKeown, 2010).”
Can Large Language Models Transform Computational Social Science? (2023, methodology): “Latent Hatred Context #example input " : j e w i s h h a r v a r d p r o f e s s o r n o e l i g n a t i e v wants t o a b o l i s h t h e w h i t e r a c e v i a #wr " " " Prompt #query ChatGPT Which of t h e f o l l o w i n g c a t e g o r i e s o f h a t e s pe e ch b e s t d e s c r i b e s t h ”