On Cross-Dataset Generalization in Automatic Detection of Online Abuse

By Isar Nejadgholi, Svetlana Kiritchenko. 2020

32 citations recorded by the source corpus.

NLP research has attained high performances in abusive language detection as a supervised classification task. While in research settings, training and test datasets are usually obtained from similar data samples, in practice systems are often applied on data that are different from the training set in topic and class distributions. Also, the ambiguity in class definitions inherited in this task aggravates the discrepancies between source and target datasets. We explore the topic bias and the task formulation bias in cross-dataset generalization. We show that the benign examples in the Wikipedia Detox dataset are biased towards platform-specific topics. We identify these examples using unsupervised topic modeling and manual inspection of topics’ keywords. Removing these topics increases cross-dataset generalization, without reducing in-domain classification performance. For a robust dataset design, we suggest applying inexpensive unsupervised methods to inspect the collected data and downsize the non-generalizable content before manually annotating for class labels.

Subfields: Artificial Intelligence, Information Systems, Molecular Biology, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Molecular Biology

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “This demonstrates the importance of training set composition, which previous hate speech research has similarly emphasised (Wiegand et al., 2019; Nejadgholi and Kiritchenko, 2020).”
ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI (2021, background): “Following initial hyperparameter optimization experiments, we use the following systems and settings: • Support Vector Machine : SVMs have been used is previous work on abuse detection in Twitter data, (e.g. Davidson et al., 2017), and have been shown to outperform neural systems (Niemann et al., 20”
Towards multidomain and multilingual abusive language detection: a survey (2021, methodology): “Transformer based Experimented with a BERT-based classifier and topic modeling approach, which show that removing domain-specific instances improve the model’s out-domain performance 2020 [83]”
Towards Ethics by Design in Online Abusive Content Detection (2020, background): “Nejadgholi and Kiritchenko (2020) demonstrated that platform-speciﬁc topics can negatively impact the generalizability of the trained classiﬁers.”
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective (2020, background): “Nejadgholi and Kiritchenko (2020) explored multiple types of selection bias and demonstrated that the ratio of offensive versus normal examples leads to a trade-off between False Positive and False Negative error rates.”
Auditing large language models: a three-layered approach (2023, background): “It is well-known that training data gaps or biases create models that perform poorly on different datasets [205].”