DiffusionCLIP: Text-guided Image Manipulation Using Diffusion Models

By Gwanghyun Kim, Jong Chul Ye. 2021

106 citations recorded by the source corpus.

Subfields: Generative Models, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Multimodal Learning

Citations in the corpus

GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models (2021, neutral): “(Zhu et al., 2019) 32.64 DF-GAN (Tao et al., 2020) 21.42 DM-GAN + CL (Ye et al., 2021) 20.79 XMC-GAN (Zhang et al., 2021) 9.33 LAFITE (Zhou et al., 2021) 8.12 DALL-E (Ramesh et al., 2021) ∼ 28 LAFITE (Zhou et al., 2021) 26.94 GLIDE 12.24 GLIDE (Validation ﬁltered)”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, background): “These models have transformed the research community and captured widespread public attention with creative image generation [22, 56] and editing applications [21, 43, 36].”
Text-to-image Diffusion Models in Generative AI: A Survey (2023, methodology): “Similar to [103], Blended diffusion [132] is based on pretrained”
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields (2021, methodology): “Besides applying CLIP to GAN models, DiffusionCLIP [19] combines a diffusion model [38] with CLIP to conduct a text-driven image manipulation.”
Bridging CLIP and StyleGAN through Latent Alignment for Image Editing (2022, unknown): “Different form those transformer-based emthods, Dif-fusionCLIP [20], GLIDE [23] and DALL-E2 [27] has introduced diffusion model [16,37,38] with CLIP [26] guidance and achieved better performance.”
Exploring CLIP for Assessing the Look and Feel of Images (2022, background): “The problem of harnessing CLIP for perception assessment can be more challenging compared to existing works related to objective attributes, such as image manipulation [9,24,38,59], object detection [12,49,72], and semantic segmenta-”
Diffusion Models for Adversarial Purification (2022, methodology): “Since then, they have been used in many image editing tasks, such as image-to-image translation (Meng et al., 2021; Choi et al., 2021; Saharia et al., 2021) and text-guided image editing (Kim & Ye, 2021; Nichol et al., 2021).”
More Control for Free! Image Synthesis with Semantic Diffusion Guidance (2021, methodology): “DiffusionCLIP [25] uses CLIP for language-based image editing, and Blended Diffusion [2] explores mask-guided image editing with CLIP.”
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting (2022, unknown): “The technique has been used effectively more recently in DiffusionCLIP [18].”
Diffusion Models already have a Semantic Latent Space (2022, background): “However, it leads to distorted images or incorrect manipulation (Kim & Ye, 2021).”
Text2LIVE: Text-Driven Layered Image and Video Editing (2022, unknown): “[19] takes this approach a step forward by optimizing the diffusion process itself.”
DiffEdit: Diffusion-based semantic image editing with mask guidance (2022, background): “In DiffusionCLIP (Kim & Ye, 2021), the weights of the diffusion model themselves are updated via gradient descent from a CLIP loss with a target text.”
Compositional Visual Generation with Composable Diffusion Models (2022, background): “Diffusion models have recently shown great promise in image generation [6], enabling effective image editing [25,20], text conditioning [26,32,13], and image inpainting [33].”
Blended Diffusion for Text-driven Editing of Natural Images (2021, methodology): “[28] propose ways to utilize a diffusion model in order to perform global text-guided image manipulations.”