Understanding and Evaluating Racial Biases in Image Captioning

By Dora Zhao, Angelina Wang, Olga Russakovsky. 2021

172 citations recorded by the source corpus.

Image captioning is an important task for benchmarking visual reasoning and for enabling accessibility for people with vision impairments. However, as in many machine learning settings, social biases can influence image captioning in undesirable ways. In this work, we study bias propagation pathways within image captioning, focusing specifically on the COCO dataset. Prior work has analyzed gender bias in captions using automatically-derived gender labels; here we examine racial and intersectional biases using manual annotations. Our first contribution is in annotating the perceived gender and skin color of 28,315 of the depicted people after obtaining IRB approval. Using these annotations, we compare racial biases present in both manual and automatically-generated image captions. We demonstrate differences in caption performance, sentiment, and word choice between images of lighter versus darker-skinned people. Further, we find the magnitude of these differences to be greater in modern captioning systems compared to older ones, thus leading to concerns that without proper consideration and mitigation these differences will only become increasingly prevalent. Code and data is available at https://princetonvisualai.github.io/imagecaptioning-bias/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In AI Ethics, Fairness & Society

Citations in the corpus

Flamingo: a Visual Language Model for Few-Shot Learning (2022, background): “Its ability to additionally handle visual inputs poses specific risks such as gender and racial biases relating to the contents of the input images, similar to a number of visual recognition systems [12, 21, 37, 97, 147].”
A Systematic Study of Bias Amplification (2022, background): “Bias amplification is concerning as it can foster the proliferation of undesired stereotypes [12, 41, 46, 47] or lead to unjustifiable differences in model accuracy between subgroups of users [7, 11].”
DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (2022, unknown): “[9, 8, 76, 12, 92, 38, 80, 37] examine social biases in image-text datasets.”
Stable Bias: Evaluating Societal Representations in Diffusion Models (2023, unknown): “For instance, in the case of image captioning, models are much more likely to generate content that stereotypes the people depicted, such as by associating women with shopping and men with snowboarding [70, 28], or producing lower-quality captions for images of people with darker skin [69].”
Quantifying Societal Bias Amplification in Image Captioning (2022, methodology): “[46] studied racial bias from several perspectives, including visual representation, sentiment analysis, and semantics.”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, methodology): “Skin tone labels: Our skin tone annotations for COCO come from (D. Zhao et al., 2021), and are numbered 1-6 according to the Fitzpatrick scale (Fitzpatrick, 1988), where 1 is the lightest and 6 is the darkest.”