Content moderation, AI, and the question of scale

By Tarleton Gillespie. 2020

422 citations recorded by the source corpus.

AI seems like the perfect response to the growing challenges of content moderation on social media platforms: the immense scale of the data, the relentlessness of the violations, and the need for human judgments without wanting humans to have to make them. The push toward automated content moderation is often justified as a necessary response to the scale: the enormity of social media platforms like Facebook and YouTube stands as the reason why AI approaches are desirable, even inevitable. But even if we could effectively automate content moderation, it is not clear that we should.

Subfields: Artificial Intelligence, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yejin Choi
2. Mohit Bansal
3. Rowan Zellers
4. Ali Farhadi
5. Jianfeng Gao
6. Ximing Lu
7. Jack Hessel
8. Hannaneh Hajishirzi

In Sociology and Political Science

1. Yejin Choi
2. Rowan Zellers
3. Ali Farhadi
4. Aditya Vashistha
5. F. Delgado
6. Stephen Yang
7. Qian Yang
8. Michael Madaio

In AI Ethics, Fairness & Society

1. Ali Farhadi
2. Aditya Vashistha
3. Yejin Choi
4. Farhana Shahid
5. Rowan Zellers

Citations in the corpus

• MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, background): “Additionally, it should be mentioned that the content moderation on YouTube disproportionately filters out minoritized voices [43].”
• Content Moderation As a Political Issue: The Twitter Discourse Around Trump's Ban (2022, background): “Facebook/Meta CEO Mark Zuckerberg repeatedly said that the company should not be seen or indeed become an “arbiter of truth”.1 However, content moderation decisions are part and parcel of social media’s business operations—and are far from neutral (Gillespie, 2020, 2018; Gerrard and Thornham, 2020).”
• Decolonizing Content Moderation: Does Uniform Global Community Standard Resemble Utopian Equality or Western Power Hegemony? (2023, background): “However, the opaqueness of the moderation processes often make them seem biased and inconsistent [55, 64, 72, 147, 151].”
• The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice (2023, background): “These shifts raise questions about whether meaningful human participation is even feasible at the scales at which many AI systems operate [21, 83, 92, 128, 192, 211]—including both the spatial scales of collection of large-scale datasets and deployment of AI systems across geographic contexts…”