Deeper Attention to Abusive User Content Moderation

By John Pavlopoulos, Prodromos Malakasiotis, Ion Androutsopoulos. 2017

162 citations recorded by the source corpus.

Experimenting with a new dataset of 1.6M user comments from a news portal and an existing dataset of 115K Wikipedia talk page comments, we show that an RNN operating on word embeddings outpeforms the previous state of the art in moderation, which used logistic regression or an MLP classifier with character or word n-grams. We also compare against a CNN operating on word embeddings, and a word-list baseline. A novel, deep, classificationspecific attention mechanism improves the performance of the RNN further, and can also highlight suspicious words for free, without including highlighted words in the training data. We consider both fully automatic and semi-automatic moderation.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

What do we need to build explainable AI systems for the medical domain? (2017, background): “While this second network is not guaranteed to provide reasons correlated to those used in the original network, it seems promising to use neural attention mechanisms to be able to trace which part of the input contributed most to which part of the output, see e.g. (Pavlopoulos et al., 2017).”
Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, background): “, 2018) or the use of (deep) neural networks (Ptaszynski et al., 2017; Pavlopoulos et al., 2017; Badjatiya et al., 2017; Vigna et al., 2017; Park and Fung, 2017; Gambäck and Sikdar, 2017).”
Developing an online hate classifier for multiple social media platforms (2020, background): “These architectures, including variants of recurrent neural networks (RNN) [14, 24, 57], convolutional neural networks (CNN) [58], or their combination, produce state-of-the-art results.”
An Attentive Survey of Attention Models (2019, background): “Recommender System User Profiling Qiannan et al. [2019], Shuai Yu [2019], He et al. [2018], Zhou et al. [2018], Kang and McAuley [2018], Ying et al.”
Tackling Online Abuse: A Survey of Automated Abuse Detection Methods (2019, methodology): “Pavlopoulos et al. (2017a; 2017b) applied deep learning to the DATA - WIKI - ATT , DATA - WIKI - TOX , and DATA - GAZZETTA datasets.”
Directions in Abusive Language Training Data: Garbage In, Garbage Out (2020, background): “Content which is considered to be incivil, rude, inappropriate, offensive or disre-spectful [46, 69, 83].”
Reducing Gender Bias in Abusive Language Detection (2018, methodology): “We explore three neural models used in previous works on abusive language classi-ﬁcation: Convolutional Neural Network (CNN) (Cho et al., 2014), and Bidirectional GRU with self-attention ( α -GRU) (Pavlopoulos et al., 2017), but with a simpler mechanism used in Felbo et al. (2017).”
Multilingual Sentiment Analysis and Toxicity Detection for Text Messages in Russian (2021, background): “In [18], authors review a possibility for semi-automatic moderation where in addition to automatic classiﬁcation of messages into insulting or neutral a moderator is also presented with the original message with suspicious words highlighted.”
Toxicity Detection can be Sensitive to the Conversational Context (2021, methodology): “Most of the published toxicity datasets contain posts in English, but datasets in other languages also exist, such as Greek (Pavlopoulos et al., 2017a), Arabic (Mubarak et al., 2017), French (Chiril et al., 2020), Indonesian (Ibrohim and Budi, 2018) and German (Ross et al., 2016; Wiegand et al.,…”
Attention in Natural Language Processing (2019, background): “exploited it in different fashions [36], [41], [54], [116].”
Conversations Gone Awry: Detecting Early Signs of Conversational Failure (2018, background): “…antisocial online behavior: trolling (Cheng et al., 2015, 2017), hate speech (Warner and Hirschberg, 2012; Davidson et al., 2017), harassment (Yin et al., 2009), personal attacks (Wulczyn et al., 2017) or, more generally, toxicity (Chandrasekha-ran et al., 2017; Pavlopoulos et al., 2017b).”
Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, background): “…shows that its strategy may involve shifting the original intent, since “republican” is not a non-offensive synonym of “moron”. et al., 2017; Pavlopoulos et al., 2017; Wulczyn et al., 2017; Gamb¨ack and Sikdar, 2017; Fortuna and Nunes, 2018; Zhang et al., 2018a; Van Hee et al., 2018;…”
Policy Compliance Detection via Expression Tree Inference (2022, background): “statutory law (Holzenberger and Van Durme, 2021) to the issue of content moderation on the social network websites (Pavlopoulos et al., 2017).”
Directions in abusive language training data, a systematic review: Garbage in, garbage out (2020, background): “Professional moderators offer a standardized approach to content annotation [41, 42].”
Context Sensitivity Estimation in Toxicity Detection (2021, background): “These posts are called toxic (Borkan et al., 2019) or abusive (Thylstrup and Waseem, 2020), and systems detecting them (Waseem and Hovy, 2016; Pavlopoulos et al., 2017b; Badjatiya et al., 2017) are called toxicity (or abusive language) detection systems.”
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases (2021, methodology): “Supervised training based methods rely on large labeled datasets to train models (Badjatiya et al., 2017; Pavlopoulos et al., 2017; Zhang et al., 2018; D’sa et al., 2019; Caselli et al., 2020; Silva et al., 2020).”