Crowdsourcing Subjective Tasks: The Case Study of Understanding Toxicity in Online Discussions

By Lora Aroyo, Lucas Dixon, Nithum Thain, Olivia Redfield, Rachel Rosen. 2019

85 citations recorded by the source corpus.

Discussing things you care about can be difficult, especially via online platforms, where sharing your opinion leaves you open to the real and immediate threats of abuse and harassment. Due to these threats, people stop expressing themselves and give up on seeking different opinions. Recent research efforts focus on examining the strengths and weaknesses (e.g. potential unintended biases) of using machine learning as a support tool to facilitate safe space for online discussions; for example, through detecting various types of negative online behaviors such as hate speech, online harassment, or cyberbullying. Typically, these efforts build upon sentiment analysis or spam detection in text. However, the toxicity of the language could be a strong indicator for the intensity of the negative behavior. In this paper, we study the topic of toxicity in online conversations by addressing the problems of subjectivity, bias, and ambiguity inherent in this task. We start with an analysis of the characteristics of subjective assessment tasks (e.g. relevance judgment, toxicity judgment, sentiment assessment, etc). Whether we perceive something as relevant or as toxic can be influenced by almost infinite amounts of prior or current context, e.g. culture, background, experiences, education, etc. We survey recent work that tries to understand this phenomenon, and we outline a number of open questions and challenges which shape the research perspectives in this multi-disciplinary field.

Subfields: Artificial Intelligence, Information Systems, Information Extraction, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In AI Ethics, Fairness & Society

Citations in the corpus

CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, methodology): “To align with existing research in the area, we’ve chosen to give annotators an existing definition of toxicity, as “rude, disrespectful or otherwise likely to make someone leave a discussion” [3].”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “This deﬁnition is operationalized by asking humans to annotate if a given text is toxic [6].”
Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application (2020, unknown): “There is a growing body of related literature showing that a single "true" labeled response is often unrealistic (Aroyo et al., 2019; Palomaki et al., 2018; Geva et al., 2019; Pavlick and Kwiatkowski, 2019).”
Quick, Community-Specific Learning: How Distinctive Toxicity Norms Are Maintained in Political Subreddits (2020, background): “While colloquially and in popular media, many online discussion forums are often railed against for being toxic12, systematically classifying comments as toxic or not is a complex problem because of the inherent subjectivity of the task (Aroyo et al. 2019).”
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators’ Disagreement (2021, background): “In these tasks – which include offensive language detection among others – subjectivity, bias and text ambiguity play an important role (Aroyo et al., 2019), and being an inherent component of the task they should be measured and analysed rather than discarded (Klen-ner et al., 2020; Basile, 2020).”
Developing an online hate classifier for multiple social media platforms (2020, background): “Online hate is a rampant problem, with the negative consequence of prohibiting user participation in online discussions [100] and causing cognitive harm to individuals [101].”
Towards Ethics by Design in Online Abusive Content Detection (2020, background): “1 It has been studied under a plethora of names, such as detection of ﬂaming (Spertus, 1997), cyber-bullying (Dadvar et al., 2013), online harassment (Golbeck et al., 2017), hate speech (Djuric et al., 2015; Davidson et al., 2017), toxicity (Dixon et al., 2018; Aroyo et al., 2019), and others.”