Directional Bias Amplification

By Angelina Wang, Olga Russakovsky. 2021

85 citations recorded by the source corpus.

Mitigating bias in machine learning systems requires refining our understanding of bias propagation pathways: from societal structures to large-scale data to trained models to impact on society. In this work, we focus on one aspect of the problem, namely bias amplification: the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. (2017); however, as we demonstrate, this metric suffers from a number of shortcomings including conflating different types of bias amplification and failing to account for varying base rates of protected attributes. We introduce and analyze a new, decoupled metric for measuring bias amplification, $\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification). We thoroughly analyze and discuss both the technical assumptions and normative implications of this metric. We provide suggestions about its measurement by cautioning against predicting sensitive attributes, encouraging the use of confidence intervals due to fluctuations in the fairness of models across runs, and discussing the limitations of what this metric captures. Throughout this paper, we work to provide an interrogative look at the technical measurement of bias amplification, guided by our normative ideas of what we want it to encompass. Code is located at https://github.com/princetonvisualai/directional-bias-amp

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “While in principle, LMs are optimised to represent language with high fidelity, they can also overrepresent small biases present in the training data, a phenomenon referred to as ‘bias amplification’ (Wang and Russakovsky, 2021; Zhao et al., 2017).”
Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “One important theme is bias amplification [80, 72, 73], or social biases in the data getting amplified in the trained models.”
Quantifying Societal Bias Amplification in Image Captioning (2022, methodology): “To solve this and other issues, Wang and Russakovsky [38] proposed an extension called directional bias amplification.”
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging (2023, background): “A final approach compares the distribution of tags applied to images that depict different social groups in a image tagging system’s training dataset to the distribution of tags applied by the system to images that depict these social groups (Wang et al. 2019, 2020; Zhao et al. 2017; Kay, Matuszek, ”
Fair Attribute Classification through Latent Space De-biasing (2020, methodology): “As our second fairness metric, we use the bias amplification (BA) metric proposed by Wang and Russakovsky [39].”
A Systematic Study of Bias Amplification (2022, methodology): “Moreover, the model developer may not always be able to measure the difficulty of recognizing group membership empirically as doing so may involve developing a model that predicts sensitive attributes—something that model developers may want to avoid [24, 27, 39, 43].”