UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models

By Hiroshi Sasaki, Chris G. Willcocks, Toby P. Breckon. 2021

220 citations recorded by the source corpus.

We propose a novel unpaired image-to-image translation method that uses denoising diffusion probabilistic models without requiring adversarial training. Our method, UNpaired Image Translation with Denoising Diffusion Probabilistic Models (UNIT-DDPM), trains a generative model to infer the joint distribution of images over both domains as a Markov chain by minimising a denoising score matching objective conditioned on the other domain. In particular, we update both domain translation models simultaneously, and we generate target domain images by a denoising Markov Chain Monte Carlo approach that is conditioned on the input source domain images, based on Langevin dynamics. Our approach provides stable model training for image-to-image translation and generates high-quality image outputs. This enables state-of-the-art Fr\'echet Inception Distance (FID) performance on several public datasets, including both colour and multispectral imagery, significantly outperforming the contemporary adversarial image-to-image translation methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models (2022, background): “, 2021), image-to-image translation (Sasaki et al., 2021), shape generation (Zhou et al.”
Palette: Image-to-Image Diffusion Models (2021, background): “, 2021a, Kadkhodaie and Simoncelli, 2021], as well as unpaired image-to-image translation [Sasaki et al., 2021] and image editing [Meng et al.”
Gotta Go Fast When Generating Data with Score-Based Models (2021, methodology): “They have further been used effectively for super-resolution [Saharia et al., 2021, Kadkhodaie and Simoncelli, 2020], inpainting [Kadkhodaie and Simoncelli, 2020, Song and Ermon, 2020], source separation [Jayaram and Thickstun, 2020], and image-to-image translation [Sasaki et al., 2021].”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “Modality conversion has been achieved using GANs [264], VAE-GANs [140], and DDPMs [192].”
Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models (2023, methodology): “They have also been successfully used for various image editing and processing tasks [18,25,41,43,48,53,66,67,69,74,84].”
Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models (2022, background): “…et al., 2021), voice conversion (Popov et al., 2021), image super-resolution (Sa-haria et al., 2021; Li et al., 2021), image-to-image translation (Sasaki et al., 2021), shape generation (Zhou et al., 2021), 3d point cloud generation (Luo & Hu, 2021) and time series forecasting (Rasul et al.,…”
Itô-Taylor Sampling Scheme for Denoising Diffusion Probabilistic Models using Ideal Derivatives (2021, background): “…including image generation (Ho et al., 2020; Vahdat et al., 2021), image superresoution (Saharia et al., 2021; Ho et al., 2021), image translation (Sasaki et al., 2021), speech synthesis (Chen et al., 2020, 2021; Kong et al., 2021; Popov et al., 2021), symbolic music generation (Mittal et al.,…”
Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction (2021, background): “Interestingly, diffusion models can go beyond unconditional image synthesis, and have been applied to conditional image generation, including super-resolution [5, 17, 25], inpainting [31, 34], MRI reconstruction [6, 13, 33], image translation [5, 19, 28], and so on.”
Unsupervised Medical Image Translation With Adversarial Diffusion Models (2022, background): “Weight for cycle-consistency loss was 1, and the release time was 1 as in [54].”
Diffusion Models for Implicit Image Segmentation Ensembles (2021, background): “While some new work applies diﬀusion models on tasks such as image-to-image translation (Sasaki et al., 2021), style transfer (Choi et al., 2021), or inpainting tasks (Saharia et al., 2021), so far there is only very little work about semantic segmentation.”
Score-Based Generative Modeling with Critically-Damped Langevin Diffusion (2021, background): “…2021), and music synthesis (Mittal et al., 2021), image editing (Meng et al., 2021; Sinha et al., 2021; Furusawa et al., 2021), super-resolution (Saharia et al., 2021; Li et al., 2021), image-to-image translation (Sasaki et al., 2021), and 3D shape generation (Luo & Hu, 2021; Zhou et al., 2021).”
VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation (2023, methodology): “Based on this formulation, DPM has been applied to various generative tasks, such as image-generation [15, 25], super-resolution [19, 28], image translation [31], etc.”
Diffusion Models for Medical Anomaly Detection (2022, methodology): “In the flow of this success, they were also applied on image-to-image translation [23,7], segmentation [4], reconstruction [22] and registration[12].”
Imagic: Text-Based Real Image Editing with Diffusion Models (2022, background): “Diffusion models [19, 55, 57, 63] are a family of generative models that has recently gained traction, as they advanced the state-of-the-art in image generation [13, 28, 58, 62], and have been deployed in various downstream applications such as image restoration [27, 49], adversarial purification [1”
Post-Training Quantization on Diffusion Models (2022, unknown): “, they are effectively introduced for super-resolution [27, 14], inpainting [36, 14], and image-to-image translation [29], dance-to-music generation [42].”