Participation Is not a Design Fix for Machine Learning

By Mona Sloane, E. Moss, O. Awomolo, Laura Forlano. 2020

361 citations recorded by the source corpus.

This paper critiques popular modes of participation in design practice and machine learning. It examines three existing kinds of participation in design practice and machine learning participation as work, participation as consultation, and as participation as justice – to argue that the machine learning community must become attuned to possibly exploitative and extractive forms of community involvement and shift away from the prerogatives of context independent scalability. Cautioning against “participation washing”, it argues that the notion of “participation” should be expanded to acknowledge more subtle, and possibly exploitative, forms of community involvement in participatory machine learning design. Specifically, it suggests that it is imperative to recognize design participation as work; to ensure that participation as consultation is context-specific; and that participation as justice must be genuine and long term. The paper argues that such a development can only be scaffolded by a new epistemology around design harms, including, but not limited to, in machine learning. To facilitate such a development, the paper suggests developing we argue that developing a cross-sectoral database of design participation failures that is cross-referenced with socio-structural dimensions and highlights “edge cases” that can and must be learned from.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “One way to help address this challenge is to use inclusive and fair participatory approaches (Martin Jr. et al., 2020), by establishing participatory mechanisms and institutions that can operate over time (Sloane et al., 2020), and by providing broad and transparent dataset documentation.”
Improving alignment of dialogue agents via targeted human judgements (2022, background): “Participatory approaches are not a panacea, however, and their successful deployment turns on a set of technical and ethical considerations that have been well documented in prior research on sociotechnical ML (Birhane et al., 2022; Sloane et al., 2020).”
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset (2023, background): “…calling to involve participants as full stake-holders of the entire research life-cycle rather to catering their role to post hoc model evaluation (Sloane et al., 2020; Caselli et al., 2021; Bondi et al., 2021), as exemplified by an organization like Maskhane (Nekoto et al., 2020) that brings…”
A Survey of Race, Racism, and Anti-Racism in NLP (2021, background): “Sloane et al. (2020) provide a discussion of how participatory design can collapse to ‘participation-washing’ and how such work must be context-specific, long-term, and genuine.”
Pitfalls in Machine Learning Research: Reexamining the Development Cycle (2020, neutral): “While the recent paper “Par-ticipation is not a Design Fix for Machine Learning” [44] raises important points about the limits of how participatory design is often done, that does not mean that participatory design is not a crucial component of the ML development cycle.”
Addressing Privacy Threats from Machine Learning (2021, background): “Although not a panacea (Sloan et al., 2020), participatory approaches can help ensure that the design of anti-surveillance technologies is led by those who are disproportionately targeted by surveillance.”
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice (2023, background): “However, it is critical to acknowledge that some participation may not be better than no participation, in situations where participation is extractive (and thus harms participants [65, 203]), tokenistic in ways that reify or amplify existing power structures within participating communities or betw”
Participatory provenance as representational auditing for AI-mediated public consultation (2026, unknown): “The participatory AI literature has highlighted power dynamics in system design [Sloane et al., 2022, Birhane et al., 2022, Delgado et al., 2023] and has called for meaningful involvement of affected communities [Young, 2000, Fung, 2006], but has not produced formal metrics for auditing live…”