Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing

By Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, Parker Barnes. 2020

1,364 citations recorded by the source corpus.

Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms. However, it remains challenging for practitioners to identify the harmful repercussions of their own systems prior to deployment, and, once deployed, emergent issues can become difficult or impossible to trace back to their source. In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development life-cycle. Each stage of the audit yields a set of documents that together form an overall audit report, drawing on an organization's values or principles to assess the fit of decisions made throughout the process. The proposed auditing framework is intended to contribute to closing the accountability gap in the development and deployment of large-scale artificial intelligence systems by embedding a robust process to ensure audit integrity.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “Risk assessment frameworks and tools Analysing and evaluating a LM regarding the above risks of harm requires innovation in risk assessment tools, benchmarks and frameworks (Raji et al., 2020; Tamkin et al., 2021).”
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs (2021, background): “[69], which proposes the “Scoping, Mapping, Artifact Collection, Testing, and Reflection” (SMACTR) framework for conducting comprehensive internal audits of AI systems via a rigorous, multi-stage process.”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “, [190]) are powerful tools that necessarily depend on the sector of deployment.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “In order to mitigate harms, we first need to be able to detect problematic, incorrect, and unreliable model outputs (Raji et al., 2020).”
Predictability and Surprise in Large Generative Models (2022, methodology): “Along with innovations in governance of AI organizations, and work on best practices, we also believe governments should invest in better methods to assure the benefits of systems being deployed - specifically, governments should support efforts to measure and monitor the capabilities (both harmful ”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Auditing must be considered both pre- and post-deployment as impacts from a system may not be detectable pre-deployment and when deployed, impacts may be difficult to trace back to a specific system [74].”
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, background): “or opportunistically. From an observer’s perspective, these practices replace abductive and a posteriori reasoning about dataset provenance with careful documentation, facilitating reviews and audits [114]. They help us to avoid the trap of being blindly “data-driven” by poorly-understood datasets— ”
Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward (2020, background): “Raji et al. (2020) suggest that a process of algorithmic auditing within the software-development company could help in tackling some of the ethical issues raised.”
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (2020, background): “[36] for a discussion of some lessons for AI from auditing in other industries.”
Taxonomy of Risks posed by Language Models (2022, background): “ing the methodological toolkit for LM analysis beyond benchmarks [151, 152, 180].”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “Model or algorithmic audits [264] can be used to assess and document such crucial accountability considerations.”
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models (2023, background): “• Accountability: It is essential for developers and stakeholders to establish a clear framework for accountability, which may include monitoring the performance of AI models, addressing biases and errors, and responding to the concerns of users and affected communities [94, 95, 96].”
What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research (2021, background): “With increasing use of artificial systems, accountability gaps might emerge [134, 135].”
Bridging the Gap Between Ethics and Practice (2020, background): “However, they can produce important internal efforts such as Google’s internal review and algorithmic auditing framework [92] (see Section 3.”
Education for AI, not AI for Education: The Role of Education and Ethics in National AI Policy Strategies (2021, methodology): “This could include, for example, requiring educational institutions that use algorithms to provide publicly accessible explanations for any decisions made, document these decisions, and establish venues to contest potentially problematic results or processes (Raji et al. 2020).”
The uselessness of AI ethics (2022, background): “The article thus concludes by highlighting alternatives: the first thinks more broadly about AI justice, considering sociopolitical dynamics and systems of oppression [14, 46]; the second thinks more narrowly, focusing on concrete issues like accuracy, auditing, and governance [25, 67].”