Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases

By Ryan Steed, Aylin Caliskan. 2020

186 citations recorded by the source corpus.

Recent advances in machine learning leverage massive datasets of unlabeled images from the web to learn general-purpose image representations for tasks from image classification to face recognition. But do unsupervised computer vision models automatically learn implicit patterns and embed social biases that could have harmful downstream effects? We develop a novel method for quantifying biased associations between representations of social concepts and attributes in images. We find that state-of-the-art unsupervised models trained on ImageNet, a popular benchmark image dataset curated from internet images, automatically learn racial, gender, and intersectional biases. We replicate 8 previously documented human biases from social psychology, from the innocuous, as with insects and flowers, to the potentially harmful, as with race and gender. Our results closely match three hypotheses about intersectional bias from social psychology. For the first time in unsupervised computer vision, we also quantify implicit human biases about weight, disabilities, and several ethnicities. When compared with statistical patterns in online image datasets, our findings suggest that machine learning models can automatically learn bias from the way people are stereotypically portrayed on the web.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Vector-quantized Image Modeling with Improved VQGAN (2021, methodology): “…image generation models, as demonstrated with biases found in the PULSE model (Menon et al., 2020) (see Section 6) and in model correlations with human biases found in social psychology (Steed & Caliskan, 2021), as well as with possible uses of such models to create fake media (Westerlund, 2019).”
LAION-5B: An open large-scale dataset for training next generation image-text models (2022, background): “Furthermore, these images and texts could amplify the social bias of machine learning models, especially ones trained with no or weak supervision [76].”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, background): “[8, 9, 71]), there has been comparatively less work on social bias evaluation methods for text-to-image models, with the recent exception of [10].”
Video Diffusion Models (2022, neutral): “In particular, our text-to-video models inherit the challenges faced by prior work on text-to-image models, and our future work will involve auditing for forms of social bias, similar to [6, 7, 50, 12] for image-to-text and image labeling models.”
DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (2022, background): “While much research has been done on evaluating common social biases in image-only [65, 56] and text-only [74, 10] models, recent research work conduct such studies in multimodal models and datasets.”
Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? (2022, background): “…for language models, for instance, models producing stereotypical and derogatory content [Bender et al. 2021], and for vision model respectively CV datasets highlighting, e.g., gender and racial biases [Denton et al. 2021; Larrazabal et al. 2020; Steed and Caliskan 2021; Wang et al. 2020].”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, background): “Algorithmic fairness methods have begun to consider intersectional attributes beyond just one axis of identity [81, 117, 118, 124].”
Muse: Text-To-Image Generation via Masked Generative Transformers (2023, background): “However, it is well known that they can also be leveraged for misinformation, harassment and various types of social and cultural biases (Franks & Waldman, 2018; Whittaker et al., 2020; Srinivasan & Uchino, 2021; Steed & Caliskan, 2021).”
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging (2023, background): “, similarly composed photos of CEOs of different genders), again testing for any differences in tagging behaviors (Otterbacher 2018; Steed and Caliskan 2021; Stock and Cisse 2018).”
Improved Transformer for High-Resolution GANs (2021, background): “For example, there are known issues around bias and fairness, either in the representation of generated images [39] or the implicit encoding of stereotypes [55].”
Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models (2022, background): “These concerns include, for instance, models producing stereotypical and derogatory content [3] and gender and racial biases [10, 24, 38, 41].”
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers (2022, background): “Much research work has been done on evaluating common social biases in image-only [52, 60] and text-only [9, 65] models.”