AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias

By R. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, P. Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic and 8 more. 2018

944 citations recorded by the source corpus.

Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This paper introduces a new open source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license {this https URL). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms to use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience (this https URL) that provides a gentle introduction to the concepts and capabilities for line-of-business users, as well as extensive documentation, usage guidance, and industry-specific tutorials to enable data scientists and practitioners to incorporate the most appropriate tool for their problem into their work products. The architecture of the package has been engineered to conform to a standard paradigm used in data science, thereby further improving usability for practitioners. Such architectural design and abstractions enable researchers and developers to extend the toolkit with their new algorithms and improvements, and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.

Subfields: Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “Bias calculation and mitigation for group fairness have accumulated extensive literature in recent years, along with many open-source benchmarks [5, 8, 21, 43].”
Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “Researchers have advocated for increased transparency of datasets [26, 35], including developing tools to steer researcher intervention [8, 71].”
Fairwashing: the risk of rationalization (2019, background): “Possible exceptions include AI Fairness 360 that provides visualizations of attribute bias and feature bias localization (Bellamy et al., 2018), DCUBE-”
Stable and Fair Classification (2019, background): “Consequently, a host of fair classification algorithms have been proposed; see (Bellamy et al., 2018).”
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes (2019, background): “Bellamy et al. [5] provide additional discussion on this topic.”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, methodology): “Using open-access, technical toolkits as an example, the fairness methods in Fairlearn (Microsoft) [59], AI Fairness 360 (IBM) [10], What-if tool (Google) [83], LiFT (LinkedIn) [80], Aequitas [70], and FAT-forensics [76] all require access to demographic attribute data.”
Data Cleaning for Accurate, Fair, and Robust Models: A Big Data - AI Integration Approach (2019, methodology): “More recently, there are mitigation techniques for fixing unfairness, which can be done before (pre-processing), during (in-processing), or after (post-processing) model training [2].”
Trustworthy AI: From Principles to Practices (2021, background): “Development Functional Testing [164, 235, 381] Performance Benchmarking [40, 56, 88, 93, 123, 127, 153, 238, 248, 307, 327]”
FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2023, neutral): “When possible—that is, the individuals’ and data attributes are available—bias detection methods should be applied by using fairness metrics such as true positive rates, statistical parity, group fairness, and equalised odds.”
Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics (2021, background): “These recent studies have led researchers to propose new evaluation tools and datasets [5, 14, 15, 16, 17, 18], to identify potential error rate gaps among different groups in machine learning.”
AUC Maximization in the Era of Big Data and AI: A Survey (2022, background): “Two issues are of foremost importance, namely fairness [7, 10, 20, 113] and interpretability [3, 24, 64].”
Education for AI, not AI for Education: The Role of Education and Ethics in National AI Policy Strategies (2021, methodology): “…(Mitchell et al. 2019) can help to promote transparency and address explicability concerns, while suites of tools such as IBM’s AI Fairness 360 (Bellamy et al. 2018) or the Scoping, Mapping, Artifact Collection, Testing, and Reflection (SMACTR) audit framework proposed by Raji et al. (2020)…”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, neutral): “For example, for the many algorithms that only work in this contrived setup, IBM’s AI Fairness 360 tool [10] formulates the binary attribute as White and Non-White, a trend sometimes shared by the social sciences that may conceptualize of social categories as dichotomous, e.”
FactSheets: Increasing trust in AI services through supplier's declarations of conformity (2019, methodology): “Unwanted biases in training data, because of prejudice in labels undersampling or oversampling, lead to unfairness and can be checked using statistical tests on datasets or models [41, 42].”
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (2023, background): “A critical step of RLHF is to train a reward model (RM) to predict human preferences between model responses.”