A Unified Taxonomy of Harmful Content

By Michele Banko, Brendon MacKeen, Laurie Ray. 2020

99 citations recorded by the source corpus.

The ability to recognize harmful content within online communities has come into focus for researchers, engineers and policy makers seeking to protect users from abuse. While the number of datasets aiming to capture forms of abuse has grown in recent years, the community has not standardized around how various harmful behaviors are defined, creating challenges for reliable moderation, modeling and evaluation. As a step towards attaining shared understanding of how online abuse may be modeled, we synthesize the most common types of abuse described by industry, policy, community and health experts into a unified typology of harmful content, with detailed criteria and exceptions for each type of abuse.

Subfields: Artificial Intelligence, Communication, Information Systems, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Information Systems

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “B References for Functional Tests F1 – strong negative emotions explicitly expressed about a protected group or its members : Resembles “expressed hatred” (Davidson et al., 2017) and “identity attack” (Banko et al., 2020).”
Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “We base our rules on the taxonomy presented in Banko et al. (2020), though we do not include doxing as it is unclear how to formulate into a rule.”
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective (2020, background): “Banko, MacKeen, and Ray (2020) assessed the most common definitions of abuse used in the domain of online content moderation technologies across industry, government policies, online communities, and the health sector.”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “Taxonomies have been developed related to online content [19, 193, 227], social media [157, 217, 218], users “at-risk" for experiencing online abuse [216, 234], and mali-”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “Taxonomies have been developed related to online content [19, 170, 201], social media [138, 192, 193], and malicious uses of algorithmic systems [39], including cyber attacks [3] and cyberbullying [12].”