Disproportionate Removals and Differing Content Moderation Experiences for Conservative, Transgender, and Black Social Media Users: Marginalization and Moderation Gray Areas

By Oliver L. Haimson, Dan Delmonaco, Peipei Nie, Andrea Wegner. 2021

293 citations recorded by the source corpus.

Social media sites use content moderation to attempt to cultivate safe spaces with accurate information for their users. However, content moderation decisions may not be applied equally for all types of users, and may lead to disproportionate censorship related to people's genders, races, or political orientations. We conducted a mixed methods study involving qualitative and quantitative analysis of survey data to understand which types of social media users have content and accounts removed more frequently than others, what types of content and accounts are removed, and how content removed may differ between groups. We found that three groups of social media users in our dataset experienced content and account removals more often than others: political conservatives, transgender people, and Black people. However, the types of content removed from each group varied substantially. Conservative participants' removed content included content that was offensive or allegedly so, misinformation, Covid-related, adult, or hate speech. Transgender participants' content was often removed as adult despite following site guidelines, critical of a dominant group (e.g., men, white people), or specifically related to transgender or queer issues. Black participants' removed content was frequently related to racial justice or racism. More broadly, conservative participants' removals often involved harmful content removed according to site guidelines to create safe spaces with accurate information, while transgender and Black participants' removals often involved content related to expressing their marginalized identities that was removed despite following site policies or fell into content moderation gray areas. We discuss potential ways forward to make content moderation more equitable for marginalized social media users, such as embracing and designing specifically for content moderation gray areas.

Subfields: Artificial Intelligence, Communication, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Sociology and Political Science

In AI Ethics, Fairness & Society

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Biases in a system’s development process and safety provisions for generative AI systems, such as content moderation, can lead to community erasure [97].”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Lessons from social media platform content moderation highlight harms such as community erasure, especially among marginalized groups [39].”
Decolonizing Content Moderation: Does Uniform Global Community Standard Resemble Utopian Equality or Western Power Hegemony? (2023, background): “While a growing scholarship has started examining the inequities and biases in content moderation, to date, most research is heavily skewed towards and informed by the experiences of users in the West [64, 110, 147, 151].”