SDEdit: Image Synthesis and Editing with Stochastic Differential Equations

By Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, Stefano Ermon. 2021

348 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

High-Resolution Image Synthesis with Latent Diffusion Models (2021, background): “Moreover, even unconditional DMs can readily be applied to tasks such as inpainting and colorization [82] or stroke-based synthesis [52], in contrast to other types of generative models [19,45,67].”
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models (2021, methodology): “Examples of text-conditional SDEdit (Meng et al., 2021) with GLIDE, where the user combines a sketch with a text caption to do more controlled modifications to an image.”
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps (2022, background): “Diffusion probabilistic models (DPMs) [1–3] are emerging powerful generative models with promising performance on many tasks, such as image generation [4, 5], video generation [6], text-to-image generation [7], speech synthesis [8, 9] and lossless compression [10].”
Palette: Image-to-Image Diffusion Models (2021, methodology): “Most diffusion models for inpainting and other linear inverse problems have adapted unconditional models for use in conditional tasks [Meng et al. 2021; Sohl-Dickstein et al. 2015; Song et al. 2021].”
DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models (2022, neutral): “…success on various tasks, such as high-resolution image synthesis (Dhariwal & Nichol, 2021; Ho et al., 2022; Rombach et al., 2022), image editing (Meng et al., 2022; Saharia et al., 2022a; Zhao et al., 2022), text-to-image generation (Nichol et al., 2021; Saharia et al., 2022b; Ramesh et al.,…”
Efficient Diffusion Training via Min-SNR Weighting Strategy (2023, background): “…diffusion models have demonstrated superior performance across a range of generation tasks in various modalities, including text-to-image generation [40, 43, 41, 17], image manipulation [26, 34, 4, 56], video synthesis [18, 47, 22], text generation [28, 16, 59], 3D avatar synthesis [39, 53], etc.”
FateZero: Fusing Attentions for Zero-shot Text-based Video Editing (2023, methodology): “Directly applying the image editing methods [33, 35] to the video will show ob-verse ﬂickering.”
Structure and Content-Guided Video Synthesis with Diffusion Models (2023, methodology): “As a baseline, we compare against SDEdit [26] in two ways; per-frame generated results and a first-frame result propagated by a few-shot video styl-ization method [55] (IVS).”
Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise (2022, background): “The noise is also thought to act as data augmentation to improve score predictions in low density regions, allowing for mode mixing in the stochastic gradient Langevin dynamics (SGLD) sampling.”
Diffusion Models for Adversarial Purification (2022, methodology): “Although adversarial puriﬁcation can be considered as a special image editing task and particularly DiffPure shares a similar procedure with SDEdit (Meng et al., 2021), none of these works apply diffusion models to improve the model robustness.”
Score-based diffusion models for accelerated MRI (2021, methodology): “The same group published a work for image editing (Meng et al., 2021) using VE-SDEs, which uses a similar algorithm to image inpainting used in (Song and Ermon, 2019; Song et al., 2021b).”
More Control for Free! Image Synthesis with Semantic Diffusion Guidance (2021, background): “They have also been explored for conditional generation such as class-conditional generation, image-guided synthesis, and super-resolution [50, 10, 8, 34].”
Generative Diffusion Prior for Unified Image Restoration and Enhancement (2023, background): “Most diffusion models toward linear inverse problems have employed unconditional models for the conditional tasks [50, 74], where only one model needs to be trained.”
On Distillation of Guided Diffusion Models (2022, methodology): “Finally, we apply our method to text-guided image inpainting and text-guided image editing tasks [20], where we reduce the total number of sampling steps to as few as 2-4 steps, demonstrating the potential of the proposed framework in style-transfer and image-editing applications [20, 41].”
Muse: Text-To-Image Generation via Masked Generative Transformers (2023, background): “A number of techniques have been developed on diffusion models to enable editing, personalization and inversion to token space (Gal et al., 2022a; Meng et al., 2021; Ruiz et al., 2022; Kawar et al., 2022; Brooks et al., 2022; Hertz et al., 2022; Mokady et al., 2022).”
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow (2022, positive): “It has been quickly popularized in other domains, such as video [e.g., 24, 92, 21], music [51], audio [e.g., 33, 40, 60], and text [41, 88], and more tasks such as image editing [97, 50].”
Text-to-image Diffusion Models in Generative AI: A Survey (2023, methodology): “Compared to SDEdit [140], SinDDM [137] can generating images with different dimensions.”
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing (2023, methodology): “We mainly compare the proposed tuning-free method to the current prompt-based editing methods with diffusion models, including tuning-free methods SDEdit [16], P2P [10], PnP [35], and Imagic [12].”
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation (2022, methodology): “9, our method outperforms the baselines by achieving both high structure preservation (in par with SDEdit w/ low noising level), and high prompt compliance (in par with SDEdit w/ high noising level).”