Deep Learning for User Comment Moderation

By John Pavlopoulos, Prodromos Malakasiotis, Ion Androutsopoulos. 2017

137 citations recorded by the source corpus.

Experimenting with a new dataset of 1.6M user comments from a Greek news portal and existing datasets of EnglishWikipedia comments, we show that an RNN outperforms the previous state of the art in moderation. A deep, classification-specific attention mechanism improves further the overall performance of the RNN. We also compare against a CNN and a word-list baseline, considering both fully automatic and semi-automatic moderation.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Since language is inherently a tool of power—the primary means by *Equal contribution 1While the majority of these models are trained on English language datasets, recent studies have also obtained similar gains on other languages (Lin et al., 2021c; Shliazhko et al., 2022) which people and societie”
Detecting Online Hate Speech Using Context Aware Models (2017, unknown): “sitional meanings of text, but they have not been well explored for online hate speech detection until recently Pavlopoulos et al. (2017). We experiment with neural net models containing separate learning components that model compositional meanings of context information.”
Deeper Attention to Abusive User Content Moderation (2017, background): “Experiments we reported elsewhere (Pavlopoulos et al., 2017) show that results on the ‘attacks’ and ‘tox-icity’ datasets are very similar; we do not include results on the latter in this paper to save space.”
Crossmod: A Cross-Community Learning-based System to Assist Reddit Moderators (2019, background): “Prior work on ML-based approaches to detect online misbehavior (e.g., abuse [19, 49], toxicity [70], hate speech [17], violations [51]) focus mainly on the detection-side of online moderation as if it is the ultimate step.”
Conversations Gone Awry: Detecting Early Signs of Conversational Failure (2018, background): “…antisocial online behavior: trolling (Cheng et al., 2015, 2017), hate speech (Warner and Hirschberg, 2012; Davidson et al., 2017), harassment (Yin et al., 2009), personal attacks (Wulczyn et al., 2017) or, more generally, toxicity (Chandrasekha-ran et al., 2017; Pavlopoulos et al., 2017b).”
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation (2017, background): “While newer machine learning techniques have been applied lately (see [26]), many relevant issues such as contextualisation are a bottleneck more widely across machine learning research [28].”
Preemptive Toxic Language Detection in Wikipedia Comments Using Thread-Level Context (2019, background): “…most often attained by deep learning approaches, such as convolutional neural networks (CNN) (Gamb¨ack and Sikdar, 2017; Potapova and Gordeev, 2016; Pavlopoulos et al., 2017) and variants of recurrent neural networks (RNN) (Pavlopou-los et al., 2017; Gao and Huang, 2017b; Pitsilis et al., 2018;…”
Neural Character-based Composition Models for Abuse Detection (2018, methodology): “As our ﬁrst baseline, we adopt the “ RNN ” method of Pavlopoulos et al. (2017) since it produces state of the art results on the Wikipedia datasets.”
The Linguistic Ideologies of Deep Abusive Language Classification (2018, background): “RNN Model Recurrent neural networks are widely used in NLP tasks (e.g. Pavlopoulos et al. (2017)) because they are good at capturing longer-term sequential patterns in text.”
Tackling Online Abuse: A Survey of Automated Abuse Detection Methods (2019, methodology): “Pavlopoulos et al. (2017a; 2017b) applied deep learning to the DATA - WIKI - ATT , DATA - WIKI - TOX , and DATA - GAZZETTA datasets.”
Toxicity Detection can be Sensitive to the Conversational Context (2021, methodology): “Most of the published toxicity datasets contain posts in English, but datasets in other languages also exist, such as Greek (Pavlopoulos et al., 2017a), Arabic (Mubarak et al., 2017), French (Chiril et al., 2020), Indonesian (Ibrohim and Budi, 2018) and German (Ross et al., 2016; Wiegand et al.,…”
Joint Modelling of Emotion and Abusive Language Detection (2020, background): “as recurrent and convolutional neural networks (Pavlopoulos et al., 2017; Park and Fung, 2017; Wang, 2018), character-based models (Nobata et al.”
Machine Learning Suites for Online Toxicity Detection (2018, background): “Pavlopoulos, et al. (2017) examined deep learning methods with word embeddings on a variety of online commentaries (both Greek and English Wikipedia).”
Cross-Domain Detection of Abusive Language Online (2018, background): “…often attained by deep learning models, the most popular being convolutional neural networks (Gamb¨ack and Sikdar, 2017; Potapova and Gordeev, 2016; Pavlopoulos et al., 2017) and variants of recurrent neural networks (Pavlopoulos et al., 2017; Gao and Huang, 2017; Pitsilis et al., 2018; Zhang et…”
Context Sensitivity Estimation in Toxicity Detection (2021, background): “These posts are called toxic (Borkan et al., 2019) or abusive (Thylstrup and Waseem, 2020), and systems detecting them (Waseem and Hovy, 2016; Pavlopoulos et al., 2017b; Badjatiya et al., 2017) are called toxicity (or abusive language) detection systems.”