GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models

By Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, Ben Y. Zhao. 2023

295 citations recorded by the source corpus.

Recent text-to-image diffusion models such as MidJourney and Stable Diffusion threaten to displace many in the professional artist community. In particular, models can learn to mimic the artistic style of specific artists after"fine-tuning"on samples of their art. In this paper, we describe the design, implementation and evaluation of Glaze, a tool that enables artists to apply"style cloaks"to their art before sharing online. These cloaks apply barely perceptible perturbations to images, and when used as training data, mislead generative models that try to mimic a specific artist. In coordination with the professional artist community, we deploy user studies to more than 1000 artists, assessing their views of AI art, as well as the efficacy of our tool, its usability and tolerability of perturbations, and robustness across different scenarios and against adaptive countermeasures. Both surveyed artists and empirical CLIP-based scores show that even at low perturbation levels (p=0.05), Glaze is highly successful at disrupting mimicry under normal conditions (>92%) and against adaptive countermeasures (>85%).

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Generative Models, Safety, Robustness & Interpretability, Privacy & Security, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Generative Models

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Ablating Concepts in Text-to-Image Diffusion Models (2023, background): “However, as these models are often trained on copyright images, it learns to mimic various artist styles [64, 61] and other copyrighted content [10].”
Erasing Concepts from Diffusion Models (2023, methodology): “Recent work [42] aims to protect the artist by applying an adversarial perturbation to artwork before posting it online to prevent the model from imitating it.”
MACE: Mass Concept Erasure in Diffusion Models (2024, background): “This technique, often referred to as cloaking, enables individuals to effectively conceal their images from models during the training phase but remain accessible and discernible to human viewers [58, 62, 75].”
Unified Concept Editing in Diffusion Models (2023, background): “…called image cloaking that allows artists to protect their content from being imitated by large generative models by adding specially crafted adversarial perturbations to images before publishing them online [34,38]; both these approaches require thorough preparation of the training corpus.”
Art and the science of generative AI (2023, background): “Even when models do not directly copy from existing works, how should artists’ individual styles be protected [72]?”