Preemptive Toxic Language Detection in Wikipedia Comments Using Thread-Level Context

By Mladen Karan, Jan Snajder. 2019

26 citations recorded by the source corpus.

We address the task of automatically detecting toxic content in user generated texts. We fo cus on exploring the potential for preemptive moderation, i.e., predicting whether a particular conversation thread will, in the future, incite a toxic comment. Moreover, we perform preliminary investigation of whether a model that jointly considers all comments in a conversation thread outperforms a model that considers only individual comments. Using an existing dataset of conversations among Wikipedia contributors as a starting point, we compile a new large-scale dataset for this task consisting of labeled comments and comments from their conversation threads.

Subfields: Artificial Intelligence, Communication, Molecular Biology, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Molecular Biology

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, background): “The creation of REALTOXICITYPROMPTS was partly inspired by work in detecting conversational patterns that can cause derailment into antisocial behavior in online conversations (Zhang et al., 2018; Stoop et al., 2019; Karan and Šnajder, 2019).”
Auditing large language models: a three-layered approach (2023, background): “For example, quantitative assessments linking prompts with outputs can give a sense of what kinds of language an LLM is propagating and how appropriate that communication style and content is in different settings [233], [234].”