Diffusion Models already have a Semantic Latent Space
By Mingi Kwon, Jaeseok Jeong, Youngjung Uh. International Conference on Learning Representations, 2022
387 citations recorded by the source corpus.
Diffusion models achieve outstanding generative performance in various domains. Despite their great success, they lack semantic latent space which is essential for controlling the generative process. To address the problem, we propose asymmetric reverse process (Asyrp) which discovers the semantic latent space in frozen pretrained diffusion models. Our semantic latent space, named h-space, has nice properties for accommodating semantic image manipulation: homogeneity, linearity, robustness, and consistency across timesteps. In addition, we introduce a principled design of the generative process for versatile editing and quality boost ing by quantifiable measures: editing strength of an interval and quality deficiency at a timestep. Our method is applicable to various architectures (DDPM++, iD- DPM, and ADM) and datasets (CelebA-HQ, AFHQ-dog, LSUN-church, LSUN- bedroom, and METFACES). Project page: https://kwonminki.github.io/Asyrp/
Subfields: Computer Vision and Pattern Recognition, Signal Processing
View Paper: https://www.semanticscholar.org/paper/a02313d56a6f71be9aafe43628e0f3a1d0cb858e
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Jitendra Malik
- Trevor Darrell
- In-So Kweon
- Xiaogang Wang
- Andrea Vedaldi
In Signal Processing
- In-So Kweon
- Allan Jabri
- Ben Poole
- Niloy J. Mitra
- Alexei A. Efros
- D. Epstein
- Duygu Ceylan
- Chun-Hao Paul Huang
Citations in the corpus
- Pix2Video: Video Editing using Image Diffusion (2023, methodology): “We perform this update process for the early denoising steps, namely the first 25 steps among the total number of 50 steps, as the overall structure of the generated image is already determined in the earlier diffusion steps [26].”
- Text-to-image Diffusion Models in Generative AI: A Survey (2023, methodology): “An unsupervised image translation method is also proposed in DiffusionIT [105], which disentangles style and content representation.”
- Diffusion Self-Guidance for Controllable Image Generation (2023, background): “Recent work [11, 36] has shown that the intermediate outputs of the diffusion U-Net encode valuable information [16, 25] about the structure and content of the generated images.”
- MACE: Mass Concept Erasure in Diffusion Models (2024, background): “Typically, diffusion models generate a general context in the early stage [12, 31, 51].”