CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech

By Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroğlu, Marco Guerini. 2019

251 citations recorded by the source corpus.

Although there is an unprecedented effort to provide adequate responses in terms of laws and policies to hate content on social media platforms, dealing with hatred online is still a tough problem. Tackling hate speech in the standard way of content deletion or user suspension may be charged with censorship and overblocking. One alternate strategy, that has received little attention so far by the research community, is to actually oppose hate content with counter-narratives (i.e. informed textual responses). In this paper, we describe the creation of the first large-scale, multilingual, expert-based dataset of hate-speech/counter-narrative pairs. This dataset has been built with the effort of more than 100 operators from three different NGOs that applied their training and expertise to the task. Together with the collected data we also provide additional annotations about expert demographics, hate and response type, and data augmentation through translation and paraphrasing. Finally, we provide initial experiments to assess the quality of our data.

Subfields: Artificial Intelligence, Communication, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “, 2011) or counterspeech (Chung et al., 2019; Qian et al., 2019).”
Resources and benchmark corpora for hate speech detection: a systematic review (2020, background): “Chung et al. (2019) (CKTG)—15,024 short text in English, French and Italian, consisting of HS–counterspeech (CS) pairs created ad hoc by experts.”
Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering (2022, background): “datasets and systems (Mathew et al., 2019; Qian et al., 2019; Chung et al., 2019; Fanton et al., 2021), up to now only datasets containing 2-turn interactions have been proposed (i.”
TweetBLM: A Hate Speech Dataset and Analysis of Black Lives Matter-related Microblogs on Twitter (2020, background): “Multilingual (Ousidhoum et al., 2019), (Chung et al., 2019) and multimodal (Gomez et al., 2020) hate speech datasets also played a very crucial role in understanding the diversity and extent of hate speech across different languages and modality.”
Low-Resource Languages Jailbreak GPT-4 (2023, background): “[11] Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini.”
Misogyny Detection in Twitter: a Multilingual and Cross-Domain Study (2020, unknown): “…(2019), and cyberbullying and trolling Cambria, Chandra, Sharma, and Hussain (2010); Menini et al. (2019), or on hate speech counter-narratives Chung, Kuzmenko, Tekiroglu, and Guerini (2019), with focus on generating positive responses, after tackling with detection of abusive content…”
Directions in Abusive Language Training Data: Garbage In, Garbage Out (2020, background): “had lost more than 60% of its content [26, 73].”
Generating Counter Narratives against Online Hate Speech: Data and Strategies (2020, background): “As a necessary ﬁrst step, diverse CN collection strategies have been proposed, each of which has its advantages and shortcomings (Mathew et al., 2018; Qian et al., 2019; Chung et al., 2019).”
SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification (2020, background): “Hate speech identiﬁcation is by far the most studied abusive language detection task (Ousidhoum et al., 2019; Chung et al., 2019; Mathew et al., 2021).”
Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis (2020, background): “Although multiple researchers have addressed the topic of hate speech (e.g. Waseem and Hovy (2016), Chung et al. (2019), Basile et al. (2019)), we focus the literature review on previous work related to toxic comments detection, the topic of our paper.”
Towards a Comprehensive Taxonomy and Large-Scale Annotated Corpus for Online Slur Usage (2020, background): “It now includes resources in Italian, Indonesian, Hindi-English, Tunisian, etc. (Sanguinetti et al., 2018; Ibrohim and Budi, 2018; Kumar et al., 2018; Bohra et al., 2018; Haddad et al., 2019; Mulki et al., 2019; Chung et al., 2019).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Dialogue Classification Chung et al. (2019) Dialogue NLI A NLI dataset with sentences annotated with entailment (E), neutral (N), or contradiction (C).”
Directions in abusive language training data, a systematic review: Garbage in, garbage out (2020, background): “provide a dataset for Islamophobia which also contains annotations for themes, including culture, economy and criminality [72].”