Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure

By Alexander Amini, Ava P. Soleimany, Wilko Schwarting, Sangeeta N. Bhatia, Daniela Rus. 2019

216 citations recorded by the source corpus.

Recent research has highlighted the vulnerabilities of modern machine learning based systems to bias, especially towards segments of society that are under-represented in training data. In this work, we develop a novel, tunable algorithm for mitigating the hidden, and potentially unknown, biases within training data. Our algorithm fuses the original learning task with a variational autoencoder to learn the latent structure within the dataset and then adaptively uses the learned latent distributions to re-weight the importance of certain data points while training. While our method is generalizable across various data modalities and learning tasks, in this work we use our algorithm to address the issue of racial and gender bias in facial detection systems. We evaluate our algorithm on the Pilot Parliaments Benchmark (PPB), a dataset specifically designed to evaluate biases in computer vision systems, and demonstrate increased overall performance as well as decreased categorical bias with our debiasing approach.

Subfields: Representation & Self-Supervised Learning, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In AI Ethics, Fairness & Society

Citations in the corpus

A Survey on Bias and Fairness in Machine Learning (2019, methodology): “In [4] authors also propose a debiased VAE architecture called DB-VAE which learns sensitive latent variables that can bias the model (e.”
Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes (2022, methodology): “We define Implementation harms as those arising through code, algorithm or data problems that can be fixed without changing the intent, or use, of the software, for example through the use of “de-biasing” techniques to reduce bias in algorithms or training data [9, 18, 78].”
Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network (2018, unknown): “Although some studies, e.g. unequal-training [9] and suppressing attributes [8, 43, 44, 42], have made effort to mitigate racial and gender bias in several computer vision tasks, this study remains to be vacant in FR.”
Fair Generative Modeling via Weak Supervision (2019, background): “Another relevant work is DB-VAE (Amini et al., 2019), which utilizes a VAE to learn the latent structure of sensitive attributes, and in turn employs importance weighting based on this structure to mitigate bias in downstream classifiers.”
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare (2020, methodology): “Recent developments in bias detection and mitigation also include methods such as adopting re-sampling 118 , adversarial learning 119 , and open-source toolkits such as IBM AI Fairness 360 (AIF360) (aif360.mybluemix.net) and Aequitas (dsapp.uchicago.edu/projects/aequitas).”
Face Recognition: Too Bias, or Not Too Bias? (2020, background): “In essence, the source varies, whether from humans [40], data or label types [35], ML models [3, 19], or evaluation protocols [34].”
Deep Evidential Regression (2019, methodology): “Systematic methods for calibrated uncertainty estimation under these conditions are needed, especially as these systems are deployed in safety critical domains, such for autonomous vehicle control [29], medical diagnosis [43], or in settings with large dataset imbalances and bias such as crime forec”