AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias

By R. Bellamy, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John T. Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, Yunfeng Zhang and 8 more. 2019

755 citations recorded by the source corpus.

Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This article introduces a new open-source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license ( https://github.com/ibm/aif360 ). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms for use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience that provides a gentle introduction to the concepts and capabilities for line-of-business users, researchers, and developers to extend the toolkit with their new algorithms and improvements and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

In AI Ethics, Fairness & Society

Citations in the corpus

Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “Researchers have advocated for increased transparency of datasets [27, 35], including developing tools to steer researcher intervention [8, 70].”
Data collection and quality challenges in deep learning: a data-centric AI perspective (2021, background): “Although there are many ways to measure fairness, one would ultimately like to perform unfairness mitigation [26, 28].”
How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis (2020, background): “Machine learning researchers have proposed numerous statistical approaches (e.g., [35]) and toolkits (e.g., [17]) for mitigating bias.”
Bias in artificial intelligence algorithms and recommendations for mitigation (2023, neutral): “An open-source toolkit, the AI Fairness 360, is available to help AI researchers examine, report, and mitigate discrimination and bias in machine learning models throughout the AI development lifecycle [52].”
Trustworthy Artificial Intelligence: A Review (2022, background): “[14] developed an open source toolkit to use fairness algorithms in an industrial setting.”
Your fairness may vary: Pretrained language model fairness in toxic text classification (2021, methodology): “A summary and implementation of group bias mitigation approaches are discussed in Bellamy et al. (2019).”
Algorithmic Bias in Education (2021, background): “Efforts are already underway to create domain-general packages for algorithmic bias/fairness metrics, and their developers welcome external contributions (Bellamy et al., 2019).”
FairBatch: Batch Selection for Model Fairness (2020, methodology): “We use the same pre-processing as in IBM’s AI Fairness 360 (Bellamy et al., 2019) and use GENDER as the sensitive attribute.”
Bridging the Gap Between Ethics and Practice (2020, background): “Beyond detecting biases, researchers offer fairness-enhancing interventions [38] and companies have developed commercial grade toolkits for detecting and mitigating algorithmic bias [7].”
Addressing bias in big data and AI for health care: A call for open science (2021, neutral): “36 Human bias .”
Trustworthy artificial intelligence (2020, background): “Next to the limited availability of high-quality training data, training data that are already available are routinely biased toward certain groups of people, which in the past has led to the implicit discrimination of those groups of people that are underrepresented in the training data (Bellamy et”