Hunting for Troll Comments in News Community Forums

By Todor Mihaylov, Preslav Nakov. 2016

93 citations recorded by the source corpus.

There are different definitions of what a troll is. Certainly, a troll can be somebody who teases people to make them angry, or somebody who offends people, or somebody who wants to dominate any single discussion, or somebody who tries to manipulate people’s opinion (sometimes for money), etc. The last definition is the one that dominates the public discourse in Bulgaria and Eastern Europe, and this is our focus in this paper. In our work, we examine two types of opinion manipulation trolls: paid trolls that have been revealed from leaked “reputation management contracts” and “mentioned trolls” that have been called such by several different people. We show that these definitions are sensible: we build two classifiers that can distinguish a post by such a paid troll from one by a non-troll with 81-82% accuracy; the same classifier achieves 81-82% accuracy on so called mentioned troll vs. non-troll posts.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, background): “Unfortunately, such exchanges in the wild will not always involve human utterances that are benign or of high quality, and will include a mixture of engaged users (dubbed helpers) and unengaged or even malicious users (dubbed trolls, following the term used elsewhere (Shachaf and Hara, 2010; Mihaylo”
Predicting Factuality of Reporting and Bias of News Media Sources (2018, background): “, 2015b) or just perceived (Mihaylov et al., 2015a; Mihaylov and Nakov, 2016; Mihaylov et al., 2018; Mihaylova et al., 2018), sockpuppets (Maity et al.”
SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums (2019, background): “In social media, there has been research targeting the user, e.g., finding malicious users (Mihaylov and Nakov, 2016; Mihaylova et al., 2018; Mihaylov et al., 2018), sockpuppets (Maity et al., 2017), Internet water army (Chen et al., 2013), and seminar users (Darwish et al., 2017).”
SemanticZ at SemEval-2016 Task 3: Ranking Relevant Answers in Community Question Answering Using Semantic Similarity Based on Fine-tuned Word Embeddings (2016, unknown): “, 2016), including troll user features as inspired by (Mihaylov et al., 2015a; Mihaylov et al., 2015b; Mihaylov and Nakov, 2016), and PMIbased goodness polarity lexicons as in the PMI-cool system (Balchev et al.”
Deep Learning for User Comment Moderation (2017, background): “Mihaylov and Nakov (2016) identify comments posted by opinion manipulation trolls.”
Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls (2022, background): “…work has centered on detecting undesirable behavior such as toxic language (Xu et al., 2020; Dinan et al., 2021), trolling (Tomaiuolo et al., 2020; Mihaylov and Nakov, 2019) and bias (Dinan et al., 2019a), less work has studied robust learning from organic conversations with potentially…”
SemEval-2016 Task 3: Community Question Answering (2019, background): “Some exotic features by the SUper Team modeled readability, credibility, sentiment analysis, trollness (Mihaylov et al., 2015a; Mihaylov et al., 2015b; Mihaylov and Nakov, 2016a), and goodness polarity, e.”