Directions in Abusive Language Training Data: Garbage In, Garbage Out

By Bertie Vidgen, Leon Derczynski. 2020

75 citations recorded by the source corpus.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “Several surveys cover approaches to this problem (Schmidt & Wiegand, 2017; Fortuna & Nunes, 2018; Vidgen et al., 2019), and there exist reviews of offensive language datasets (Fortuna et al., 2020; Vidgen & Derczynski, 2020).”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, methodology): “Vidgen and Derczynski (2020) examined 63 open-source abusive language datasets and found that 27 (43”
Dynabench: Rethinking Benchmarking in NLP (2021, background): “Few high-quality, varied and large training datasets are available for training hate detection systems (Vidgen and Derczynski, 2020; Poletto et al., 2020; Vidgen et al., 2019).”
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus (2021, background): “several datasets and approaches being proposed in recent years (Schmidt and Wiegand, 2017; Mishra et al., 2019; Vidgen and Derczynski, 2020; Kiritchenko and Mohammad, 2018).”
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) (2020, neutral): “This strategy is in line with current best practices in abusive language data collection (Vidgen and Derczynski, 2020).”
Implicitly Abusive Language – What does it actually look like and why are we not getting there? (2021, background): “Since a survey would be beyond the scope of this section, we refer the reader to Poletto et al. (2020) and Vidgen and Derczynski (2020).”
On Cross-Dataset Generalization in Automatic Detection of Online Abuse (2020, methodology): “Several publicly available datasets have been created for the task (Mishra et al., 2019; Vidgen and Derczynski, 2020).”
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective (2020, methodology): “Vidgen and Derczynski (2020) enumerated the sources of inconsistencies in creating abusive language datasets, highlighted the barriers to data sharing and the lack of infrastructure needed for open-source research, and recommended a set of best practices for data sampling and annotation to improve…”
Introducing CAD: the Contextual Abuse Dataset (2021, background): “Trained experts with clear guidelines are often preferable for ensuring consistency (Vidgen and Derczynski, 2020).”
XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages (2020, unknown): “Recently, it has been pointed out that existing abusive language data sets are biased towards certain types of abuse (Jurgens et al., 2019; Vidgen and Derczynski, 2020) and domains/topics (Wiegand et al., 2019).”
A Unified Taxonomy of Harmful Content (2020, background): “While the number of research datasets continues to grow (Vidgen and Derczynski, 2020), the research community has not reached a consensus on how common abuse types are deﬁned.”
Towards multidomain and multilingual abusive language detection: a survey (2021, methodology): “First, Vidgen and Derczynski [136] provided the analysis of available training data for abusive language detection tasks and proposed best practices in creating training data of abusive language based on existing studies.”
Abusive Language Detection using Syntactic Dependency Graphs (2020, background): “…and annotator bias: Recent works have highlighted that majority of the abusive language datasets suffer from poor quality (Wie-gand et al., 2019; Vidgen and Derczynski, 2020) or show evidence that annotator decisions were inap-propriately affected by surface markers of speaker race (Sap et al.,…”
Recalibrating classifiers for interpretable abusive content detection (2020, background): “Equally, many classiﬁers struggle with contextual statements, irony, humour and content that is non-abusive but ‘incivil’ (Vidgen and Derczynski, 2020).”