A Note on Data Biases in Generative Models

By Patrick Esser, Robin Rombach, Björn Ommer. 2020

20 citations recorded by the source corpus.

It is tempting to think that machines are less prone to unfairness and prejudice. However, machine learning approaches compute their outputs based on data. While biases can enter at any stage of the development pipeline, models are particularly receptive to mirror biases of the datasets they are trained on and therefore do not necessarily reflect truths about the world but, primarily, truths about the data. To raise awareness about the relationship between modern algorithms and the data that shape them, we use a conditional invertible neural network to disentangle the dataset-specific information from the information which is shared across different datasets. In this way, we can project the same image onto different datasets, thereby revealing their inherent biases. We use this methodology to (i) investigate the impact of dataset quality on the performance of generative models, (ii) show how societal biases of datasets are replicated by generative models, and (iii) present creative applications through unpaired transfer between diverse datasets such as photographs, oil portraits, and animes. Our code and an interactive demonstration are available at https://github.com/CompVis/net2net .

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

High-Resolution Image Synthesis with Latent Diffusion Models (2021, background): “6, 7, 22 [22] Patrick Esser, Robin Rombach, and Björn Ommer.”
Vector-quantized Image Modeling with Improved VQGAN (2021, background): “Esser et al. (2020) discusses these biases in details, and they furthermore project images from the FFHQ dataset onto CelebAHQ: the main effect of which is to produce images of younger people with features conforming more to norms of celebrities popular in the United States of America.”
VBench: Comprehensive Benchmark Suite for Video Generative Models (2023, background): “Moreover, generative models can potentially inherit biases from the training datasets [21].”
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation (2021, methodology): “[18] Patrick Esser, Robin Rombach, and Björn Ommer.”