Imagic: Text-Based Real Image Editing with Diffusion Models

By Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Hui-Tang Chang, Tali Dekel, Inbar Mosseri, Michal Irani. 2022

1,531 citations recorded by the source corpus.

Text-conditioned image editing has recently attracted considerable interest. However, most methods are currently limited to one of the following: specific editing types (e.g., object overlay, style transfer), synthetically generated images, or requiring multiple input images of a common object. In this paper we demonstrate, for the very first time, the ability to apply complex (e.g., non-rigid) text-based semantic edits to a single real image. For example, we can change the posture and composition of one or multiple objects inside an image, while preserving its original characteristics. Our method can make a standing dog sit down, cause a bird to spread its wings, etc. – each within its single high-resolution user-provided natural image. Contrary to previous work, our proposed method requires only a single input image and a target text (the desired edit). It operates on real images, and does not require any additional inputs (such as image masks or additional views of the object). Our method, called Imagic, leverages a pre-trained text-to-image diffusion model for this task. It produces a text embedding that aligns with both the input image and the target text, while fine-tuning the diffusion model to capture the image-specific appearance. We demonstrate the quality and versatility of Imagic on numerous inputs from various domains, showcasing a plethora of high quality complex semantic image edits, all within a single unified framework. To better assess performance, we introduce TEdBench, a highly challenging image editing benchmark. We conduct a user study, whose findings show that human raters prefer Imagic to previous leading editing methods on TEdBench.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

In Multimodal Learning

Citations in the corpus

InstructPix2Pix: Learning to Follow Image Editing Instructions (2022, methodology): “Learning to follow instructions Our method differs from existing text-based image editing works [6,13,17,27,38,52] in that it enables editing from instructions that tell the model what action to perform, as opposed to text labels, captions or descriptions of input/output images.”
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation (2022, background): “Several concurrent methods have taken first steps in controlling different properties of the generated content [14,21,39,48,50].”
Video-P2P: Video Editing with Cross-Attention Control (2023, positive): “Diffusion models have demonstrated outstanding generation capabilities in this area [3, 13, 19, 40].”
Emu Edit: Precise Image Editing via Recognition and Generation Tasks (2023, background): “Imagic [9] finetunes the diffusion model to support complex textual instructions.”
A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence (2023, methodology): “In addition, several other methods [11, 16, 25, 62] focus on accessing and editing features for downstream tasks such as stylization and image editing.”
Text-to-image Diffusion Models in Generative AI: A Survey (2023, methodology): “Specifically, Imagic [143] first obtains an optimized embedding for the target text, which generates similar images to the input.”
Diffusion Model Alignment Using Direct Preference Optimization (2023, methodology): “We test DPO-SDXL on TEd-Bench [22], a text-based image-editing benchmark of 100 real image-text pairs, using SDEdit [28] with noise strength 0.6.”
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, background): “…used in numerous downstream tasks, including personalized image generation (Ruiz et al. 2023; Kumari et al. 2023), image editing (Hertz et al. 2022; Brooks, Holynski, and Efros 2023; Kawar et al. 2023), image inpainting (Yang et al. 2022a) and conditional image synthesis (Zhang and Agrawala 2023).”
Pix2Video: Video Editing using Image Diffusion (2023, neutral): “In another line of work, methods have focused on editing images while preserving structures via attention layer manipulation [15, 52], spatial guidance [7], or per-instance finetuning [24].”
Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models (2023, background): “Several works have also explored providing users with more control over the synthesis process solely through the use of the input text prompt [8, 20, 24, 46].”
Ablating Concepts in Text-to-Image Diffusion Models (2023, background): “Fine-tuning aims to adapt the weights of a pretrained generative model to new domains [73, 46, 72, 41, 79, 34, 47, 80, 30, 35, 24, 44], downstream tasks [71, 54, 78], and test images [6, 53, 48, 31, 25, 49].”
Training-Free Layout Control with Cross-Attention Guidance (2023, background): “Text-based image editing can also be achieved through single-image model ﬁne-tuning [27, 47].”
AnyDoor: Zero-shot Object-level Image Customization (2023, neutral): “For example, some works [5, 24, 63] learn to edit the posture, styles, or content of an image via instructions.”
Adding Conditional Control to Text-to-Image Diffusion Models (2023, methodology): “Text-guided control methods focus on adjusting prompts, manipulating CLIP features, and modifying cross-attention [7, 10, 20, 27, 40, 41, 57, 63, 66].”
Muse: Text-To-Image Generation via Masked Generative Transformers (2023, background): “Dreambooth (Ruiz et al., 2022) and Imagic (Kawar et al., 2022) involve ﬁne-tuning of the generative models.”
Zero-shot Image-to-Image Translation (2023, background): “Imagic [Kawar et al. 2022] demonstrates great editing results but requires finetuning the entire model for each image.”
FateZero: Fusing Attentions for Zero-shot Text-based Video Editing (2023, methodology): “Based on this pipeline, several methods have been proposed in terms of cross-attention guidance [38], plug-and-play feature [48], and optimization [25,35].”
FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space (2025, neutral): “…[4] demonstrated the promise of synthetic instruction-response pairs for fine-tuning a diffusion model for image editing, while learning-free methods for personalized text-to-image synthesis[13, 43, 24] enable image modification with off-the-shelf, high-performance image generation models [42, 40].”
DragDiffusion: Harnessing Diffusion Models for Interactive Point-Based Image Editing (2023, methodology): “This is because most diffusion-based methods (Hertz et al., 2022; Mokady et al., 2023; Kawar et al., 2023; Parmar et al., 2023) conduct editing by controling the text embeddings, which makes their applications primarily focus on editing high-level semantic contents or styles.”
EDICT: Exact Diffusion Inversion via Coupled Transformations (2022, neutral): “Since methods that require model finetuning or prompt tuning [6,11,26] can be combined with EDICT, we view them as complementary to, rather than competitive with, EDICT, and do not perform a direct comparison with them.”