Toward situated interventions for algorithmic equity: lessons from the field

By Michael A. Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Binz, Daniella Raz, P. Krafft. 2020

165 citations recorded by the source corpus.

Research to date aimed at the fairness, accountability, and transparency of algorithmic systems has largely focused on topics such as identifying failures of current systems and on technical interventions intended to reduce bias in computational processes. Researchers have given less attention to methods that account for the social and political contexts of specific, situated technical systems at their points of use. Co-developing algorithmic accountability interventions in communities supports outcomes that are more likely to address problems in their situated context and re-center power with those most disparately affected by the harms of algorithmic systems. In this paper we report on our experiences using participatory and co-design methods for algorithmic accountability in a project called the Algorithmic Equity Toolkit. The main insights we gleaned from our experiences were: (i) many meaningful interventions toward equitable algorithmic systems are non-technical; (ii) community organizations derive the most value from localized materials as opposed to what is "scalable" beyond a particular policy context; (iii) framing harms around algorithmic bias suggests that more accurate data is the solution, at the risk of missing deeper questions about whether some technologies should be used at all. More broadly, we found that community-based methods are important inroads to addressing algorithmic harms in their situated contexts.

Subfields: Computer Science Applications, Safety Research, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “whether they should be built at all, and (b) keeping development and deployment decisions in the hands of technologists (Bennett and Keyes, 2019; Cifor et al., 2019; Green, 2019; Katell et al., 2020).”
Re-imagining Algorithmic Fairness in India and Beyond (2021, background): “Since outcomes of ML deployments can be difficult to predict, proposals for accountability include participatory design [109] and participatory problem formulation [128], sharing the responsibility for design-”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, methodology): “…2013), using frameworks like the IEEE Ethically Aligned Design (IEEE Global Initiative 2016), technology policy design frameworks like Diverse Voices (Young et al. 2019) and mechanisms for the co-development of algorithmic accountability through participatory action research (Katell et al. 2020).”
Power to the People? Opportunities and Challenges for Participatory AI (2022, background): “A central tension of this this recent wave of “participatory” is whether these mechanisms should merely serve to aid in the refinement of relevant machine learning sytem or rather emphasize lived experience as a critical form of knowledge and employ experiential learning as a force for community emp”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “justice and thereby justify the collection and use of this sensitive data [6, 40, 50, 73], understanding how practitioners in industry”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “These harms focus on the domain of government, and focus on how algorithmic systems govern through individualized nudges or micro-directives [165], that may destabilize governance systems, erode human rights, be used as weapons of war [166], and enact surveillant regimes that disproportionately targ”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “…and focus on how algorithmic systems govern through individualized nudges or micro-directives [187], that may destabi-lize governance systems, erode human rights, be used as weapons of war [188], and enact surveillant regimes that disproportionately target and harm people of color [120].”
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice (2023, background): “(2020) [110] Academia Public Policy Re-center power with those”
How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India (2022, methodology): “We build on the work of Katell et al.[74], and propose a situated approach to algorithmic accountability.”
A Hierarchy of Limitations in Machine Learning (2020, background): “But the biggest point of implementation is not in software or even computational infrastructure; it is in institutions and power dynamics, for which nontechnical parts of interventions supersede the importance of technical parts (Katell et al., 2020).”
Expanding Explainability: Towards Social Transparency in AI systems (2021, background): “In a similar vein, field work on algorithmic fairness often found that meaningful interventions toward usable and ethical algorithmic systems are non-technical, and that user community derive most value from localized, as opposed to “scalable” solutions [49, 54].”
Epistemic values in feature importance methods: Lessons from feminist epistemology (2021, background): “places at particular points in time, rather than trying to study a system or question with an abstract approach removed from social context” [40].”
The ethics of algorithms: key problems and solutions (2020, unknown): “While this approach is useful for abstracting and defining analytical processes, it tends to ignore the social complexity of the real world (Katell et al. 2020).”