Hybrid Emoji-Based Masked Language Models for Zero-Shot Abusive Language Detection
By Michele Corazza, Stefano Menini, Elena Cabrio, Sara Tonelli, Serena Villata. 2020
18 citations recorded by the source corpus.
Recent studies have demonstrated the effectiveness of cross-lingual language model pre-training on different NLP tasks, such as natural language inference and machine translation. In our work, we test this approach on social media data, which are particularly challenging to process within this framework, since the limited length of the textual messages and the irregularity of the language make it harder to learn meaningful encodings. More specifically, we propose a hybrid emoji-based Masked Language Model (MLM) to leverage the common information conveyed by emojis across different languages and improve the learned cross-lingual representation of short text messages, with the goal to perform zero- shot abusive language detection. We compare the results obtained with the original MLM to the ones obtained by our method, showing improved performance on German, Italian and Spanish.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Quoc V. Le
- Mohit Bansal
- Percy Liang
- Hannaneh Hajishirzi
- Graham Neubig
- Jianfeng Gao
- Jason Wei
In Natural Language Processing
- Andrew M. Dai
- Quoc V. Le
- Kelvin Guu
- Adams Wei Yu
- Jason Wei
- Endang Wahyu Pamungkas
- Mohit Bansal
- Viviana Patti
Citations in the corpus
- Finetuned Language Models Are Zero-Shot Learners (2021, background): “, 2020), as well as various NLP applications (Liu et al., 2019a; Corazza et al., 2020; Wang et al., 2021; Reif et al., 2021).”
- Towards multidomain and multilingual abusive language detection: a survey (2021, background): “Several studies also presented the interesting finding that infusing language-agnostic features extracted from hate-specific lexicons HurtLex, in particular) [29, 94] and emoji-based features [28] could improve abusive language detection systems in a multilingual setting.”