Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models

By Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, Tom Goldstein. 2022

505 citations recorded by the source corpus.

Cutting-edge diffusion models produce images with high quality and customizability, enabling them to be used for commercial art and graphic design purposes. But do diffusion models create unique works of art, or are they replicating content directly from their training sets? In this work, we study image retrieval frameworks that enable us to compare generated images with training samples and detect when content has been replicated. Applying our frameworks to diffusion models trained on multiple datasets including Oxford flowers, Celeb-A, ImageNet, and LAION, we discuss how factors such as training set size impact rates of content replication. We also identify cases where diffusion models, including the popular Stable Diffusion model, blatantly copy from their training data. Project page: https://somepago.github.io/diffrep.html

Subfields: Computer Vision and Pattern Recognition, Generative Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Safety, Robustness & Interpretability

Citations in the corpus

Foundation Models and Fair Use (2023, background): “As Somepalli et al. (2022) and others note, however, as foundation models for image generation train on more data, they are less likely to output content similar to the training data on average.”
Extracting Training Data from Diffusion Models (2023, background): “[65] show several cases where (non-adversarially) sampling from a diffusion model can produce memorized training examples.”
Art and the science of generative AI (2023, methodology): “Generative AI can also be used to deliberately emulate a specific existing work, either through the use of prompt material or by fine-tuning the AI [65, 73].”
Stable Bias: Evaluating Societal Representations in Diffusion Models (2023, unknown): “…understand their inner workings, such as the functioning of their safety filters [51], the structure of their semantic space [14], the extent to which they generate stereotypical representations of given demographics [10, 19, 44] and memorize and regenerate specific training examples [16, 62].”
Machine Learning for Synthetic Data Generation: a Review (2023, background): “If we plug this into the closed-form solution of the KL-divergence between two multivariate Gaussian distributions, we will have the following for t = 1 , · · · , T − 1 : The diffusion model has achieved wide success in many downstream synthetic problems [80], [152]–[154].”
MACE: Mass Concept Erasure in Diffusion Models (2024, methodology): “An alternative method for safeguarding images against imitation or memorization [8, 65] by T2I models involves an additional step of applying adversarial perturbations to photographs or artworks before they are posted online.”
Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models (2023, background): “[7, 60] have successfully retrieved samples from Stable Diffusion that are highly faithful to real training examples.”
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, background): “[246] showed that the data replication problem existed in diffusion models, where the generated images are close to the training data in terms of semantic similarity.”
Erasing Concepts from Diffusion Models (2023, background): “While the traditional goal of machine learning is to generalize without memo-rization, large models are capable of exact memorization if speciﬁcally trained to do so [48], and unintentional memo-rization has also been observed in large-scale settings [6, 5], including diffusion models [45].”
Unified Concept Editing in Diffusion Models (2023, background): “Recent lawsuits [1,37] have contended that models like Stable Diffusion infringe on many artistic styles, and researchers have found that the models can memorize some copyrighted training data nearly verbatim [5,41].”
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation (2023, result): “These concerns are indicative of broader issues within the field, as highlighted by various studies (Birhane et al., 2021; Schramowski et al., 2023; Somepalli et al., 2023; Bae et al., 2023; Zhang et al., 2023b).”
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis (2024, background): “In the context of generative image models memorization of training samples can lead to a number of issues (Somepalli et al., 2023a; Carlini et al., 2023; Somepalli et al., 2023b).”
Ablating Concepts in Text-to-Image Diffusion Models (2023, methodology): “However, as these models are often trained on copyright images, it learns to mimic various artist styles [64, 61] and other copyrighted content [10].”
AI Art and its Impact on Artists (2023, background): “In fact, while image generators have been shown to even memorize their data and can output almost exact replicas of images from their training set under certain conditions [27, 117], as artist Karla Ortiz writes, artists’ styles are so unique to them,”