Diffusion Posterior Sampling for General Noisy Inverse Problems
By Hyungjin Chung, Jeongsol Kim, Michael T. McCann, Marc Louis Klasky, Jong Chul Ye. 2022
1,760 citations recorded by the source corpus.
Diffusion models have been recently studied as powerful generative inverse problem solvers, owing to their high quality reconstructions and the ease of combining existing iterative solvers. However, most works focus on solving simple linear inverse problems in noiseless settings, which significantly under-represents the complexity of real-world problems. In this work, we extend diffusion solvers to efficiently handle general noisy (non)linear inverse problems via approximation of the posterior sampling. Interestingly, the resulting posterior sampling scheme is a blended version of diffusion sampling with the manifold constrained gradient without a strict measurement consistency projection step, yielding a more desirable generative path in noisy settings compared to the previous studies. Our method demonstrates that diffusion models can incorporate various measurement noise statistics such as Gaussian and Poisson, and also efficiently handle noisy nonlinear inverse problems such as Fourier phase retrieval and non-uniform deblurring. Code available at https://github.com/DPS2022/diffusion-posterior-sampling
Subfields: Geophysics, Mathematical Physics, Radiation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Geophysics
In Mathematical Physics
- Jian Zhang
- Fredrik K. Gustafsson
- Kai Zhang
- Jingyun Liang
- Luc Van Gool
- Radu Timofte
- Bihan Wen
- Thomas Bo Schön
In Radiation
Citations in the corpus
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (2024, neutral): “Inspire by [50, 51, 52], we first sample ˆX 0 from p θ ( X 0 | X t , X p , C ) , and then sample X t − ∆ t from q ( X t − ∆ t | ˆX 0 , X t ) , which involves remask ⌊ N · σ ( t − ∆ t ) ⌋ tokens in ˆX 0 with the lowest confidence score, where we define the confidence score of ˆ x i in ˆX 0 to p θ (ˆ…”
- Universal Guidance for Diffusion Models (2023, methodology): “[3] studied general guidance functions, and modified the sampling process with the gradient of guidance function calculated on the expected denoised images.”
- DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion (2023, methodology): “DPS [3] endorses Laplacian approximation to calculate the gradient of log-likelihood for posterior sampling and it is capable of many noisy non-linear inverse problems.”
- Consistency Models (2023, neutral): “Iterative generation is also crucial for the zeroshot data editing capabilities of diffusion models, enabling them to solve challenging inverse problems ranging from image inpainting, colorization, stroke-guided image editing, to Computed Tomography and Magnetic Resonance Imaging (Song & Ermon, 2019”
- Practical and Asymptotically Exact Conditional Sampling in Diffusion Models (2023, neutral): “Outside of the context of SMC and the twisting technique, the use of denoising estimate x̂θ(x, t) to form an approximation p̃θ(x | x, y) to pθ(x | x, y) is called reconstruction guidance [5, 16, 27].”
- Image Restoration with Mean-Reverting Stochastic Differential Equations (2023, methodology): “Consider the diffusion process in DDPM: where α t = 1 − β t and ¯ α t = =1 α s .”
- Denoising Diffusion Models for Plug-and-Play Image Restoration (2023, methodology): “For each t start , we also indicate the actual number of NFEs in the Figure.”
- FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model (2023, background): “Considering the expectation of p ( x 0 | x t ) [6]: where ¯ α t = =1 (1 − β i ) and s ( · , t ) is the pre-trained score estimator.”