Feature-Wise Bias Amplification

By Klas Leino, Matt Fredrikson, Emily Black, Shayak Sen, Anupam Datta. 2018

52 citations recorded by the source corpus.

We study the phenomenon of bias amplification in classifiers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth. We demonstrate that bias amplification can arise via an inductive bias in gradient descent methods that results in the overestimation of the importance of moderately-predictive "weak" features if insufficient training data is available. This overestimation gives rise to feature-wise bias amplification -- a previously unreported form of bias that can be traced back to the features of a trained model. Through analysis and experiments, we show that while some bias cannot be mitigated without sacrificing accuracy, feature-wise bias amplification can be mitigated through targeted feature selection. We present two new feature selection algorithms for mitigating bias amplification in linear models, and show how they can be adapted to convolutional neural networks efficiently. Our experiments on synthetic and real data demonstrate that these algorithms consistently lead to reduced bias without harming accuracy, in some cases eliminating predictive bias altogether while providing modest gains in accuracy.

Subfields: Artificial Intelligence, Optimization & Training, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

In AI Ethics, Fairness & Society

Citations in the corpus

Measuring Bias in Contextualized Word Representations (2019, background): “These embeddings are then used in a plethora of downstream applications, which perpetuate and further amplify stereotypes (Zhao et al., 2017; Leino et al., 2019).”
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation (2019, background): “Classical literature on mitigating bias generally operates on simpler (often linear) models [11, 50, 28], which are easier to understand and control; only recently have researchers begun looking at mitigating bias in end-to-end trained deep learning models [16, 2, 40, 18, 48, 25, 30, 36, 47, 17].”
Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021, background): “Among the most harmful issues are their lack of interpretability (Lip-ton, 2018; Guidotti et al., 2018), and the tendency to learn spurious correlations, in addition to the true signals of the task (Leino et al., 2019; Sagawa et al., 2020b).”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, unknown): “amplification bias Arises when the distribution over prediction outputs is skewed in comparison to the prior distribution of the prediction target [271].”
Quantifying Social Biases in Contextual Word Representations (2019, background): “These embeddings are then used in a plethora of downstream applications, which perpetuate and further amplify stereotypes (Zhao et al., 2017; Leino et al., 2019).”
A Systematic Study of Bias Amplification (2022, methodology): “Prior work has measured bias amplification using generative adversarial networks [8, 22], by considering binary classifications without attributes [28], and by measuring correlations in model predictions [23, 47].”
Data Feedback Loops: Model-driven Amplification of Dataset Biases (2022, background): “Machine learning models have a tendency to amplify at test-time biases that exist in their training data, a problem known as bias ampliﬁcation [12, 38, 22].”
Directional Bias Amplification (2021, methodology): “Bias amplification has been measured by looking at binary classifications without attributes (Leino et al., 2019), GANs (Jain et al.”
Topics to Avoid: Demoting Latent Confounds in Text Classification (2019, background): “Text classification systems based on neural networks are biased towards learning frequent spurious correlations in the training data that may be confounds in the actual classification task (Leino et al., 2019).”
Gender bias and stereotypes in Large Language Models (2023, background): “…effect for women, such that stereotypically male occupations were chosen less frequently than expected and stereotypically female occupations were chosen more frequently than expected — that is, the model amplifies stereotypical biases about women’s occupations [8, 10, 37, 39, 54, 88, 101, 103].”