Common Sense Reasoning for Detection, Prevention, and Mitigation of Cyberbullying

By Karthik Dinakar, Rosalind W. Picard, Henry Lieberman. 2012

440 citations recorded by the source corpus.

Cyberbullying (harassment on social networks) is widely recognized as a serious social problem, especially for adolescents. It is as much a threat to the viability of online social networks for youth today as spam once was to email in the early days of the Internet. Current work to tackle this problem has involved social and psychological studies on its prevalence as well as its negative effects on adolescents. While true solutions rest on teaching youth to have healthy personal relationships, few have considered innovative design of social network software as a tool for mitigating this problem. Mitigating cyberbullying involves two key components: robust techniques for effective detection and reflective user interfaces that encourage users to reflect upon their behavior and their choices. Spam filters have been successful by applying statistical approaches like Bayesian networks and hidden Markov models. They can, like Google’s GMail, aggregate human spam judgments because spam is sent nearly identically to many people. Bullying is more personalized, varied, and contextual. In this work, we present an approach for bullying detection based on state-of-the-art natural language processing and a common sense knowledge base, which permits recognition over a broad spectrum of topics in everyday life. We analyze a more narrow range of particular subject matter associated with bullying (e.g. appearance, intelligence, racial and ethnic slurs, social acceptance, and rejection), and construct BullySpace, a common sense knowledge base that encodes particular knowledge about bullying situations. We then perform joint reasoning with common sense knowledge about a wide range of everyday life topics. We analyze messages using our novel AnalogySpace common sense reasoning technique. We also take into account social network analysis and other factors. We evaluate the model on real-world instances that have been reported by users on Formspring, a social networking website that is popular with teenagers. On the intervention side, we explore a set of reflective user-interaction paradigms with the goal of promoting empathy among social network participants. We propose an “air traffic control”-like dashboard, which alerts moderators to large-scale outbreaks that appear to be escalating or spreading and helps them prioritize the current deluge of user complaints. For potential victims, we provide educational material that informs them about how to cope with the situation, and connects them with emotional support from others. A user evaluation shows that in-context, targeted, and dynamic help during cyberbullying situations fosters end-user reflection that promotes better coping strategies.

Subfields: Artificial Intelligence, Social Psychology, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Signal Processing

Citations in the corpus

Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, background): “, 2017), abusive language (Mehdad and Tetreault, 2016; Park and Fung, 2017), cyberbullying (Dadvar et al., 2013; Dinakar et al., 2012; Hee et al., 2015; Zhong et al., 2016) and offensive language (Chen et al.”
Cyber Hate Speech on Twitter: An Application of Machine Classification and Statistical Modeling for Policy and Decision Making (2015, methodology): “Burnap et al. (2013) developed a rule-based approach to classifying antagonistic content on Twitter and, similarly to Dinakar et al. (2012), they used associational terms as features.”
All You Need is "Love": Evading Hate Speech Detection (2018, background): “help in detecting particular paerns related to hate speech [8], but do not scale beyond those paerns.”
Detection of Cyberbullying Incidents on the Instagram Social Network (2015, background): “Prior works that investigated cyberbullying (Ptaszynski et al. 2010),(Dadvar et al. 2012),(K. Reynolds and Edwards 2011),(Dinakar et al. 2012),(H. Sanchez 2012),(Kontostathis et al. 2013), (Xu et al. 2012),(Nahar et al. 2014),(Na-har, Li, and Pang 2013),(Dinakar, Reichart, and Lieberman…”
Detecting Nastiness in Social Media (2017, background): “Dinakar et al. (2012) concentrate on sexuality, race and culture, and intelligence as the primary categories related to cyberbullying.”
Crowdsourcing Subjective Tasks: The Case Study of Understanding Toxicity in Online Discussions (2019, neutral): “For example, [47] investigates personal attacks, [25] aggression, [7, 13, 45] hate speech, [29, 49] harassment, and [11] cyberbullying.”
Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media (2018, background): “Other work has lever-aged sentiment markers, speciﬁcally negative polarity and sentiment strength in preprocessing (Dinakar et al. 2012; Sood, Churchill, and Antin 2012; Gitari et al. 2015) and as features for hate speech classiﬁcation (Van Hee et al. 2015; Burnap et al. 2015).”
Algorithmic content moderation: Technical and political challenges in the automation of platform governance (2020, methodology): “Other approaches rely on domain-speciﬁc ontologies, e.g. of terms implicated in LGBT hate speech, to attempt to infer whether a statement might be hateful given certain domain knowledge (such as the gender of the recipient; Dinakar et al., 2012).”
Benchmarking Aggression Identification in Social Media (2018, background): “…several related behaviour such as trolling (Cambria et al., 2010; Kumar et al., 2014; Mojica, 2016; Mihaylov et al., 2015) , cyberbullying (Dinakar et al., 2012; Nitta et al., 2013; Dadvar et al., 2013; Dadvar et al., 2014; Hee et al., 2015), ﬂaming / insults (Sax, 2016; Nitin et al.,…”
A Framework for the Computational Linguistic Analysis of Dehumanization (2020, background): “Despite the American public’s increasing acceptance of LGBTQ people and recent legal successes, LGBTQ individuals frequently remain the targets of hate and violence (Dinakar et al., 2012; Silva et al., 2016; Gallup, 2019).”
A Survey on Hate Speech Detection using Natural Language Processing (2017, methodology): “More recently, many authors have shifted to employing the term cyberbullying (Xu et al., 2012; Hosseinmardi et al., 2015; Zhong et al., 2016; Van Hee et al., 2015; Dadvar et al., 2013; Dinakar et al., 2012).”
“A Stalker's Paradise”: How Intimate Partner Abusers Exploit Technology (2018, background): “For example, online bullying, which clearly occurs in IPV contexts, may also occur when people who don’t know each other post abusive content on-line [2, 14].”
CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech (2019, background): “, 2016) and knowledge base (Dinakar et al., 2012), or to utilize deep neural networks (Mehdad and Tetreault, 2016; Badjatiya et al.”
ETHOS: a multi-label hate speech detection dataset (2020, background): “Because of the social costs arising out of HS, several countries consider it an illegal act, particularly when violence or hatred is encouraged [9].”
Learning Deep on Cyberbullying is Always Better Than Brute Force (2017, methodology): “[Dinakar et al., 2012] focused on overlapping concepts like sexual or racial harassment.”
Toward Multimodal Cyberbullying Detection (2017, methodology): “…have worked on the effects of cy-berbullying on teenagers [25, 50] and also tried to identify automated methods for cyberbullying detection [13, 12, 40], such approaches are yet to consider the dramatically changed social media landscape that the teenagers are dealing with now compared to…”