Measuring Social Biases in Grounded Vision and Language Embeddings

By Candace Ross, Boris Katz, Andrei Barbu. 2020

73 citations recorded by the source corpus.

We generalize the notion of measuring social biases in word embeddings to visually grounded word embeddings. Biases are present in grounded embeddings, and indeed seem to be equally or more significant than for ungrounded embeddings. This is despite the fact that vision and language can suffer from different biases, which one might hope could attenuate the biases in both. Multiple ways exist to generalize metrics measuring bias in word embeddings to this new setting. We introduce the space of generalizations (Grounded-WEAT and Grounded-SEAT) and demonstrate that three generalizations answer different yet important questions about how biases, language, and vision interact. These metrics are used on a new dataset, the first for grounded bias, created by augmenting standard linguistic bias benchmarks with 10,228 images from COCO, Conceptual Captions, and Google Images. Dataset construction is challenging because vision datasets are themselves very biased. The presence of these biases in systems will begin to have real-world consequences as they are deployed, making carefully measuring bias and then mitigating it critical to building a fair society.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In AI Ethics, Fairness & Society

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…Gyamfi et al. (2020); Nadeem et al. (2020); Nissim et al. (2020); Papakyriakopoulos et al. (2020); Popović et al. (2020); Ravfogel et al. (2020); Ross et al. (2020); Rozado (2020); Saunders and Byrne (2020); Shin et al. (2020); Vig et al. (2020); Wang et al. (2020); Yang and Feng (2020); Zhao…”
Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022, background): “There are increasing efforts to address this problem in the community (Ross et al., 2021).”
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models (2021, unknown): “There are increasing efforts to address the problem in the community (Ross et al., 2021; Zhao et al., 2021).”
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension (2022, background): “language models suffer from biases such as gender bias (Ross et al., 2021; Srinivasan and Bisk, 2021).”
Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study (2023, background): “Previous work in natural language processing (NLP) has primarily focused on cultural investigation of models (Hutchinson et al., 2020; Ross et al., 2021; Ma et al., 2022), with little emphasis on dialogue agents.”
Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale (2022, background): “Many seminal papers have demonstrated extensive biases in previous language and vision models trained on similar data (Burns et al., 2018; Wang et al., 2021; Ross et al., 2021; Wolfe and Caliskan, 2022b,a; Wolfe et al., 2022; Weidinger et al., 2021; Cho et al., 2022; Bansal et al., 2022); and recent”
Theories of “Gender” in NLP Bias Research (2022, background): “05233 [117] Candace Ross, Boris Katz, and Andrei Barbu.”
DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (2022, background): “[55, 50] showed social biases in visually grounded word embeddings.”
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers (2022, background): “[47, 51] showed social biases in contextualized visually grounded word embeddings.”
Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy (2021, background): “Joint models of vision and language suggest that biases come from language, rather than from vision (Ross et al., 2021).”
Modular Visual Question Answering via Code Generation (2023, background): “Prior work (Ross et al., 2020; Agarwal et al., 2021) has found evidence of social biases in vision-language models trained on image-captions.”