Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
By Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori B. Hashimoto, Dan Jurafsky, James Y. Zou, Aylin Caliskan. Conference on Fairness, Accountability and Transparency, 2022
535 citations recorded by the source corpus.
Machine learning models that convert user-written text descriptions into images are now widely available online and used by millions of users to generate millions of images a day. We investigate the potential for these models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects. For example, we find cases of prompting for basic traits or social roles resulting in images reinforcing whiteness as ideal, prompting for occupations resulting in amplification of racial and gender disparities, and prompting for objects resulting in reification of American norms. Stereotypes are present regardless of whether prompts explicitly mention identity and demographic language or avoid such language. Moreover, stereotypes persist despite mitigation strategies; neither user attempts to counter stereotypes by requesting images with specific counter-stereotypes nor institutional attempts to add system “guardrails” have prevented the perpetuation of stereotypes. Our analysis justifies concerns regarding the impacts of today’s models, presenting striking exemplars, and connecting these findings with deep insights into harms drawn from social scientific and humanist disciplines. This work contributes to the effort to shed light on the uniquely complex biases in language-vision models and demonstrates the ways that the mass deployment of text-to-image generation models results in mass dissemination of stereotypes and resulting harms.
Subfields: Sociology and Political Science
View Paper: https://www.semanticscholar.org/paper/a19fa76c42fcfed057694805312ba5d308980a21
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Sociology and Political Science
Citations in the corpus
- Stable Bias: Evaluating Societal Representations in Diffusion Models (2023, unknown): “…understand their inner workings, such as the functioning of their safety filters [51], the structure of their semantic space [14], the extent to which they generate stereotypical representations of given demographics [10, 19, 44] and memorize and regenerate specific training examples [16, 62].”
- Unified Concept Editing in Diffusion Models (2023, background): “Diffusion image generation models have been found to be prone to social and cultural biases [7,24,43], even exaggerating and amplifying societal stereotypes beyond simple imbalances in the training data [4,11], although quantifying amplification can be subtle [36].”
- When and why vision-language models behave like bags-of-words, and what to do about it? (2022, background): “…models fail to correctly represent compositional information (like attribution of properties to entities), they may default to stereotypes; e.g., Bianchi et al. (2022) notes a case in which DallE fails to represent the compositional information in the prompt “a disabled woman leading a meting”,…”
- The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation (2023, unknown): “Consequently, a considerable number of studies is devoted to understanding the limitations and algorithmic biases inherent in deep learning systems and generative AI models (7–17).”
- Coarse race data conceals disparities in clinical risk score performance (2023, background): “…et al., 2022; Laufer et al., 2022; Pierson, 2020; Liu and Garg, 2022; Derenoncourt and Montialoux, 2021; Chouldechova, 2017; Garg et al., 2018; Cheng et al., 2023; Bianchi et al., 2023; Abdu et al., 2023), suggesting the importance of examining granular race categories in these domains as well.”