The Fallacy of AI Functionality

By Inioluwa Deborah Raji, Indra Elizabeth Kumar, Aaron Horowitz, Andrew D. Selbst. 2022

292 citations recorded by the source corpus.

Deployed AI systems often do not work. They can be constructed haphazardly, deployed indiscriminately, and promoted deceptively. However, despite this reality, scholars, the press, and policymakers pay too little attention to functionality. This leads to technical and policy solutions focused on “ethical” or value-aligned deployments, often skipping over the prior question of whether a given system functions, or provides any benefits at all. To describe the harms of various types of functionality failures, we analyze a set of case studies to create a taxonomy of known AI functionality issues. We then point to policy and organizational responses that are often overlooked and become more readily available once functionality is drawn into focus. We argue that functionality is a meaningful AI policy challenge, operating as a necessary first step towards protecting affected communities from algorithmic harm.

Subfields: Health Informatics, Information Systems, Safety Research, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Informatics

In Information Systems

In Safety Research

In Safety, Robustness & Interpretability

In AI Ethics, Fairness & Society

Citations in the corpus

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “, 2022), leading to misaligned research and policy priorities (Raji et al., 2022) and potentially dire consequences in e.”
Auditing large language models: a three-layered approach (2023, background): “The second check within functionality audits also aim to address the risks stemming from developers overstating or misrepresenting a specific application’s capabilities [223].”
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (2024, unknown): “If only well-heeled actors can even evaluate the risks of AI systems, claims of functionality (Raji et al., 2022) and that AI can improve access to justice may be quite overstated (Bommasani et al., 2022; Chien et al., 2024; Perlman, 2023; Tan et al., 2023).”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “Scholars have developed harm taxonomies related to system misbehaviors and failures [18, 198], particularly to aid algorithmic auditing [177].”
Harms from Increasingly Agentic Algorithmic Systems (2023, background): “[153] highlight could disincentivize adoption of increasingly agentic systems, even if they were”