Towards Ethics by Design in Online Abusive Content Detection

By Svetlana Kiritchenko, Isar Nejadgholi. 2020

14 citations recorded by the source corpus.

To support safety and inclusion in online communications, significant efforts in NLP research have been put towards addressing the problem of abusive content detection, commonly defined as a supervised classification task. The research effort has spread out across several closely related sub-areas, such as detection of hate speech, toxicity, cyberbullying, etc. There is a pressing need to consolidate the field under a common framework for task formulation, dataset design and performance evaluation. Further, despite current technologies achieving high classification accuracies, several ethical issues have been revealed. We bring ethical issues to forefront and propose a unified framework as a two-step process. First, online content is categorized around personal and identity-related subject matters. Second, severity of abuse is identified through comparative annotation within each category. The novel framework is guided by the Ethics by Design principle and is a step towards building more accurate and trusted models.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “Ultimately, whether or not something is offensive is subjective, and several authors emphasize that any decisions (e.g., on classiﬁcation or mitigation strategies) should respect community norms and language practices (Jurgens et al., 2019; Sap et al., 2019; Kiritchenko & Nejadgholi, 2020).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “This makes it especially important to consider what community norms are applicable when deploying a model (Jurgens et al., 2019; Sap et al., 2019; Kiritchenko and Nejadgholi, 2020; Liang et al., 2022),”
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022, background): “, on classification or mitigation strategies) should respect community norms and language practices (Jurgens et al., 2019; Sap et al., 2019; Kiritchenko and Nejadgholi, 2020).”