A Just and Comprehensive Strategy for Using NLP to Address Online Abuse

By David Jurgens, Eshwar Chandrasekharan, Libby Hemphill. 2019

153 citations recorded by the source corpus.

Online abusive behavior affects millions and the NLP community has attempted to mitigate this problem by developing technologies to detect abuse. However, current methods have largely focused on a narrow definition of abuse to detriment of victims who seek both validation and solutions. In this position paper, we argue that the community needs to make three substantive changes: (1) expanding our scope of problems to tackle both more subtle and more serious forms of abuse, (2) developing proactive technologies that counter or inhibit abuse before it harms, and (3) reframing our effort within a framework of justice to promote healthy communities.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, background): “We acknowledge that the binary distinction between recommended and non-recommended is only the coarsest-grained view of complex and multi-dimensional social norms, however more input from impacted communities is required before attempting more sophisticated distinctions (Jurgens et al., 2019).”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “Ultimately, whether or not something is offensive is subjective, and several authors emphasize that any decisions (e.g., on classiﬁcation or mitigation strategies) should respect community norms and language practices (Jurgens et al., 2019; Sap et al., 2019; Kiritchenko & Nejadgholi, 2020).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “This makes it especially important to consider what community norms are applicable when deploying a model (Jurgens et al., 2019; Sap et al., 2019; Kir-itchenko and Nejadgholi, 2020; Liang et al., 2022), and whether the use of labels might not be a risk in itself (Thylstrup and Waseem, 2020).”
Finding Microaggressions in the Wild: A Case for Locating Elusive Phenomena in Social Media Posts (2019, background): “profane, and hateful language (Schmidt and Wiegand, 2017), offensiveness spans a far larger spectrum that includes comments with more implicit and subtle signals that are no less offensive (Jurgens et al., 2019).”
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021, background): “…on overt abuse or explicit hate speech (Schmidt and Wiegand, 2017), recent works have started to highlight the diverse range of implicitly hateful messages that have previously gone unnoticed by moderators and researchers alike (Jurgens et al., 2019; Waseem et al., 2017; Qian et al., 2019).”
Resources and benchmark corpora for hate speech detection: a systematic review (2020, background): “More boldly, Jurgens et al. (2019) call for a paradigm shift in the use of NLP technologies to address abusive language.”
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022, background): “In addition to overtly offensive language, several works highlight the importance of including more subtle forms of abuse, such as implicit abuse and micro-aggressions (e.g., Jurgens et al., 2019; Caselli et al., 2020; Han and Tsvetkov, 2020).”
Leveraging Bias in Pre-Trained Word Embeddings for Unsupervised Microaggression Detection (2021, unknown): “Most previous work has focused on few, easy to model phenomena, ignoring more subtle and complex ones, such as microaggressions (Jurgens et al., 2019).”
Directions in Abusive Language Training Data: Garbage In, Garbage Out (2020, background): “elled data to learn these representations, it may be difficult for researchers to come up with models of syntactic structure that can help to identify implicit abuse.âĂŹ ([110], p. 81) •Jurgens et al.[63] discuss also conduct a critical review of hate speech detection, and note that ‘ïż£labelled gro”
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective (2020, background): “Jurgens et al. (2019) argue that the NLP community has thus far focused on a too-narrow definition of online abuse, and should branch out to both more subtle forms of abuse, such as microaggressions, and more severe threats to safety and personal privacy, such as doxxing.”
I Feel Offended, Don’t Be Abusive! Implicit/Explicit Messages in Offensive and Abusive Language (2020, background): “Social media platforms have been promoted as (virtual) places where many people from different parts of the world can engage in productive discussions and share opinions (Jurgens et al., 2019).”
Towards Ethics by Design in Online Abusive Content Detection (2020, background): “Moreover, the scope of studied abusive behaviors has been limited (Jurgens et al., 2019).”
The Psychological Well-Being of Content Moderators: The Emotional Labor of Commercial Moderation and Avenues for Improving Support (2021, background): “Various algorithms have been proposed to automatically detect problematic content in an effort to minimize the amount of human moderation work required [42, 67, 78, 92, 123, 136, 156].”
Introducing CAD: the Contextual Abuse Dataset (2021, background): “As Jurgens et al. (2019) argued in a recent review, research has ‘struggled to move beyond the most obvious tasks in abuse detection.’”
HateBERT: Retraining BERT for Abusive Language Detection in English (2020, unknown): “, microaggression, stereotyping, offense, abuse, hate speech, threats, and doxxing (Jurgens et al., 2019).”
Directions in abusive language training data, a systematic review: Garbage in, garbage out (2020, background): “3661) They argue that the field needs to ‘address the data scarcity faced by abuse detection research’ in order to better address more complex research issues and pressing social challenges, such as ‘develop[ing] proactive technologies that counter or inhibit abuse before it harms’ ([23], pp.”
Misogyny Detection in Twitter: a Multilingual and Cross-Domain Study (2020, unknown): “…and Guerini (2019), with focus on generating positive responses, after tackling with detection of abusive content published online, encouraging the community to adopt a proactive approach to transform the toxic environments into positive ones Jurgens, Chandrasekharan, and Hemphill (2019).”
Unsupervised Discovery of Implicit Gender Bias (2020, background): “This task is notoriously difficult because words like “pretty” often register as positive content, rather than as indicative of bias (Breitfeller et al., 2019; Jurgens et al., 2019).”
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations (2021, background): “Researchers have also expanded the focus to more subtle forms of abuse such as condescension and microaggres-sions (Breitfeller et al., 2019; Jurgens et al., 2019).”