XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages

By Goran Glavas, Mladen Karan, Ivan Vulic. 2020

69 citations recorded by the source corpus.

We present XHate-999, a multi-domain and multilingual evaluation data set for abusive language detection. By aligning test instances across six typologically diverse languages, XHate-999 for the first time allows for disentanglement of the domain transfer and language transfer effects in abusive language detection. We conduct a series of domain- and language-transfer experiments with state-of-the-art monolingual and multilingual transformer models, setting strong baseline results and profiling XHate-999 as a comprehensive evaluation resource for abusive language detection. Finally, we show that domain- and language-adaption, via intermediate masked language modeling on abusive corpora in the target language, can lead to substantially improved abusive language detection in the target language in the zero-shot transfer setups.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, neutral): “, 2019; 2020), Facebook (Glavaš et al., 2020; Zampieri et al., 2020), or Reddit (Han & Tsvetkov, 2020; Zampieri et al.”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “, 2020), Facebook (Glavaš et al., 2020; Zampieri et al., 2020), or Reddit (Han and Tsvetkov, 2020; Zampieri et al.”
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022, background): “, 2019, 2020), Facebook (Glavaš et al., 2020; Zampieri et al., 2020), or Reddit”
Towards multidomain and multilingual abusive language detection: a survey (2021, methodology): “The most recent study by Glavas et al. [48] shows that ROBERTa outperformed other models such as BERT in the cross-domain setting of the hate speech detection task.”
ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI (2021, methodology): “An assistant created by Rasa 3 and, hosted on Facebook Messenger.”
Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? (2022, background): “A vast body of work to date that describes methodologies to tackle, abusive, offensive, hateful [19], toxic [22], stereotypical [32] or otherwise biased content [13] come from NLP.”