Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints

By Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang. 2017

1,100 citations recorded by the source corpus.

Language is increasingly being used to de-fine rich visual recognition problems with supporting image collections sourced from the web. Structured prediction models are used in these tasks to take advantage of correlations between co-occurring labels and visual input but risk inadvertently encoding social biases found in web corpora. In this work, we study data and models associated with multilabel object classification and visual semantic role labeling. We find that (a) datasets for these tasks contain significant gender bias and (b) models trained on these datasets further amplify existing bias. For example, the activity cooking is over 33% more likely to involve females than males in a training set, and a trained model further amplifies the disparity to 68% at test time. We propose to inject corpus-level constraints for calibrating existing structured prediction models and design an algorithm based on Lagrangian relaxation for collective inference. Our method results in almost no performance loss for the underlying recognition task but decreases the magnitude of bias amplification by 47.5% and 40.5% for multilabel classification and visual semantic role labeling, respectively。

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, background): “, 2018) as well as in image recognition models (Zhao et al., 2017).”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “Stereotyping Bolukbasi et al. (2016a,b); Caliskan et al. (2017); McCurdy and Serbetçi (2017); Rudinger et al. (2017); Zhao et al. (2017); Curry and Rieser (2018); Díaz et al. (2018); Santana et al. (2018); Sutton et al. (2018); Zhao et al. (2018a,b); Agarwal et al. (2019); Basta et al. (2019);…”
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, background): “, 2018), visual semantic role labeling (Zhao et al., 2017), image captioning (Hendricks et al.”
Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function (2019, methodology): “Bolukbasi et al. (2016) and Zhao et al. (2017) deﬁned gender bias as a projection of gender-neutral words on a gender subspace and removed bias by minimizing this projection.”
Gender Bias in Coreference Resolution (2018, methodology): “Finally, we note independent but closely related work by Zhao et al. (2018), published concurrently with this paper.”
Ethical and social risks of harm from Language Models (2021, background): “While in principle, LMs are optimised to represent language with high fidelity, they can also overrepresent small biases present in the training data, a phenomenon referred to as ‘bias amplification’ (Wang and Russakovsky, 2021; Zhao et al., 2017).”
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021, background): “Text representations have been known to learn and exacerbate skewed associations and social biases from underlying data (Zhao et al., 2017; Bender et al., 2021; Dev, 2020), thus propagating representational harm.”
Shortcut learning in deep neural networks (2020, background): “When human biases are not only replicated, but worsened by a machine, this is referred to as bias amplification [72].”
On Measuring and Mitigating Biased Inferences of Word Embeddings (2019, methodology): “To discover biases, several lines of existing work (Bolukbasi et al. 2016; Caliskan, Bryson, and Narayanan 2017; Zhao et al. 2017; Dev and Phillips 2019) employ measurements intrinsic to the vector representations, which despite their utility, have two key problems.”
Evaluating Gender Bias in Machine Translation (2019, background): “Notable examples include gender biases in visual SRL (cooking is stereotypically done by women, construction workers are stereotypically men; Zhao et al., 2017), lexical semantics (“man is to computer programmer as woman is to homemaker”; Bolukbasi et al.”
On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “Machine learning models are shown to manifest and escalate historical biases present in their training data [1, 4, 16, 56].”
Re-contextualizing Fairness in NLP: The Case of India (2022, background): “While the majority of NLP fairness research focuses on gender (Bolukbasi et al., 2016; Sun et al., 2019; Zhao et al., 2017)”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “(2021), can make matters worse by overamplifying existing data bias (Zhao et al., 2017; Shah et al., 2020).”
Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models (2019, methodology): “We create such a dataset by ﬁrst computing the gender bias [Zhao et al., 2017] in the training data for each word.”