Auditing radicalization pathways on YouTube

By Manoel Horta Ribeiro, Raphael Ottoni, Robert West, Virgílio A. F. Almeida, Wagner Meira Jr. 2019

478 citations recorded by the source corpus.

Non-profits, as well as the media, have hypothesized the existence of a radicalization pipeline on YouTube, claiming that users systematically progress towards more extreme content on the platform. Yet, there is to date no substantial quantitative evidence of this alleged pipeline. To close this gap, we conduct a large-scale audit of user radicalization on YouTube. We analyze 330,925 videos posted on 349 channels, which we broadly classified into four types: Media, the Alt-lite, the Intellectual Dark Web (I.D.W.), and the Alt-right. According to the aforementioned radicalization hypothesis, channels in the I.D.W. and the Alt-lite serve as gateways to fringe far-right ideology, here represented by Alt-right channels. Processing 72M+ comments, we show that the three channel types indeed increasingly share the same user base; that users consistently migrate from milder to more extreme content; and that a large percentage of users who consume Alt-right content now consumed Alt-lite and I.D.W. content in the past. We also probe YouTube's recommendation algorithm, looking at more than 2M video and channel recommendations between May/July 2019. We find that Alt-lite content is easily reachable from I.D.W. channels, while Alt-right videos are reachable only through channel recommendations. Overall, we paint a comprehensive picture of user radicalization on YouTube.

Subfields: Artificial Intelligence, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In AI Ethics, Fairness & Society

Citations in the corpus

The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, background): “Optimizing for misspeciﬁed proxies led YouTube to overemphasize watch-time and harm user satisfaction (Stray, 2020), as well as to recommended extreme political content to users (Ribeiro et al., 2020).”
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, background): “Though the content moderation might perhaps reduce overtly ‘toxic’ content, social media platforms like YouTube still contain harmful microagressions [15], and alt-lite to alt-right content [94].”
MERLOT: Multimodal Neural Script Knowledge Models (2021, background): “YouTube also has problems with hate, including radical alt-right and ‘alt-lite’ content [90].”
Optimizing Long-term Social Welfare in Recommender Systems: A Constrained Matching Approach (2020, background): “Some work has looked at the impact of RS policies on provider welfare (Singh & Joachims, 2018), on long-term user and provider metrics (e.g., using RL (Chen et al., 2018; Ie et al., 2019b) or assessing various other phenomena (Ribeiro et al., 2020; Celma, 2010)).”
Harms from Increasingly Agentic Algorithmic Systems (2023, background): “Several landmark works in the FATE community have involved audits [Raji et al. 2020] of algorithmic systems [Black et al. 2022; Buolamwini and Gebru 2018; Obermeyer and Mullainathan 2019; Ribeiro et al. 2020].”