Women also Snowboard: Overcoming Bias in Captioning Models

By Kaylee Burns, Lisa Anne Hendricks, Trevor Darrell, Anna Rohrbach. 2018

525 citations recorded by the source corpus.

Most machine learning methods are known to capture and exploit biases of the training data. While some biases are beneficial for learning, others are harmful. Specifically, image captioning models tend to exaggerate biases present in training data (e.g., if a word is present in 60% of training sentences, it might be predicted in 70% of sentences at test time). This can lead to incorrect captions in domains where unbiased captions are desired, or required, due to over-reliance on the learned prior and image context. In this work we investigate generation of gender-specific caption words (e.g. man, woman) based on the person's appearance or the image context. We introduce a new Equalizer model that ensures equal gender probability when gender evidence is occluded in a scene and confident predictions when gender evidence is present. The resulting model is forced to look at a person rather than use contextual cues to make a gender-specific predictions. The losses that comprise our model, the Appearance Confusion Loss and the Confident Loss, are general, and can be added to any description model in order to mitigate impacts of unwanted bias in a description dataset. Our proposed model has lower error than prior work when describing images with people and mentioning their gender and more closely matches the ground truth ratio of sentences including women to sentences including men. We also show that unlike other approaches, our model is indeed more often looking at people when predicting their gender.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, methodology): “, 2017), image captioning (Hendricks et al., 2018), and demographic biases in language generation (Sheng et al.”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, neutral): “…gender in words, stands in contrast to other works which additionally use paired word lists (e.g., king-queen) to measure and debias (Bolukbasi et al. 2016; Zhao et al. 2017; Hendricks et al. 2018, among others, but see Gonen and Goldberg 2019; May et al. 2019 for shortcomings of such approaches).”
Towards Understanding and Mitigating Social Biases in Language Models (2021, background): “These approaches have also been applied to image captioning (Hendricks et al., 2018), image retrieval (Otterbacher, 2018), and dialog (Liu et al.”
Flamingo: a Visual Language Model for Few-Shot Learning (2022, background): “Its ability to additionally handle visual inputs poses specific risks such as gender and racial biases relating to the contents of the input images, similar to a number of visual recognition systems [12, 21, 37, 97, 147].”
Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “Recent work has discovered biases in these systems, but often with respect to gender [31, 12, 64]; the study of racial biases in captioning has been limited to analyzing bias in the manual captions [54, 65].”
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts (2021, background): “Less speciﬁc to the Web data are the unwanted dataset biases [13, 68, 75] that are prone to ampliﬁcation by machine learning models [11, 82].”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, background): “[8, 9, 71]), there has been comparatively less work on social bias evaluation methods for text-to-image models, with the recent exception of [10].”
Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy (2019, methodology): “Rather than addressing the full complexity of this question, we follow prior work [10, 11, 41, 65, 83] and use a set of discrete categories: Male , Female , and Unsure , in which Unsure is used to both handle ambiguous visual cues as well as to include people with diverse gender expression.”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, methodology): “stance, an image captioning model is worse at recognizing a woman than a man when they are each sitting in front of a computer (Burns et al., 2018), that is a clear indicator of recognition bias.”
Assessing Social and Intersectional Biases in Contextualized Word Representations (2019, neutral): “35]. Gender bias has also been demonstrated downstream on several applications of natural language processing, including sentiment analysis [18, 30], abusive language detection [23], image captioning [16] and text classiﬁcation [11]. These models not only reﬂect the bias in training data, but also a”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, background): “…relating to race, gender, (dis)ability, and more [Garg et al., 2018, Hutchinson et al., 2020] and correlations between gender and activities depicted in computer vision datasets have been shown to reﬂect common gender stereotypes [Zhao et al., 2017, Burns et al., 2018, van Miltenburg, 2016].”
Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development (2021, neutral): “Stereotype aligned correlations between gender and the activities being depicted in images have also been identified in several computer vision datasets (e.g., overrepresenting women in images depicting cooking and shopping [55, 123]).”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Likewise, a related objective of XAI is highlighting bias in the data a model was exposed to [168,169] .”
Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, background): “Some recent work proposes forcing models to attend to regions of the input which are known to be important (Burns et al., 2018; Mitsuhara et al., 2019), although it is important to note that attention is often not the same as explanation (Jain & Wallace, 2019).”
Diversity in Faces (2019, methodology): “age prediction in coding scheme 7, where we bin the age values into seven groups: [0, 1, 2, 3],[4, 5, 6, 7, 8, 9, 10, 11, 12],[13, 14, 15, 16, 17, 18, 19],[20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30],[31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45],[46, 47, 48, 49, 50, 51, 52, 53, 54, 55”