Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser

By Zahra Kadkhodaie, Eero P. Simoncelli. 2020

106 citations recorded by the source corpus.

Prior probability models are a central component of many image processing problems, but density estimation is notoriously difficult for high-dimensional signals such as photographic images. Deep neural networks have provided state-of-the-art solutions for problems such as denoising, which implicitly rely on a prior probability model of natural images. Here, we develop a robust and general methodology for making use of this implicit prior. We rely on a little-known statistical result due to Miyasawa (1961), who showed that the least-squares solution for removing additive Gaussian noise can be written directly in terms of the gradient of the log of the noisy signal density. We use this fact to develop a stochastic coarse-to-fine gradient ascent procedure for drawing high-probability samples from the implicit prior embedded within a CNN trained to perform blind (i.e., unknown noise level) least-squares denoising. A generalization of this algorithm to constrained sampling provides a method for using the implicit prior to solve any linear inverse problem, with no additional training. We demonstrate this general form of transfer learning in multiple applications, using the same algorithm to produce high-quality solutions for deblurring, super-resolution, inpainting, and compressive sensing.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Numerical Methods & Scientific Computing

Citations in the corpus

Improved Techniques for Training Score-Based Generative Models (2020, background): “[9] noticed that adding an extra denoising step after the original annealed Langevin dynamics in [1], similar to [10, 11, 12], often significantly improves FID scores [13] without affecting the visual appearance of samples.”
D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, background): “These properties c include (but are not limited to2) labels [61], text descriptions [60, 76], noisy or partial observations [11, 5, 47, 24], and manipulation constraints [69].”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, methodology): “For conditioning on text, we use text cross attention at resolutions [32, 16, 8] as well as attention pooled text embedding.”
Adversarial score matching and improved sampling for image generation (2020, methodology): “From Proposition 3, we see that CAS shares some similarities with the algorithm by Kadkhodaie and Simoncelli (2020). While the weight we give to the denoiser (η) decreases geometrically (by its linearity in σ), their schedule appears to be much steeper.”
Gotta Go Fast When Generating Data with Score-Based Models (2021, methodology): “They have further been used effectively for super-resolution [Saharia et al., 2021, Kadkhodaie and Simoncelli, 2020], inpainting [Kadkhodaie and Simoncelli, 2020, Song and Ermon, 2020], source separation [Jayaram and Thickstun, 2020], and image-to-image translation [Sasaki et al., 2021].”
Video Diffusion Models (2022, neutral): “[25, 24].”
CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation (2021, methodology): “To utilize diffusion models for imputation, previous studies [12, 15, 16] approximated the conditional reverse process pθ(x t−1 | x t ,x 0 ) with the reverse process in Eq.”
Post-Training Quantization on Diffusion Models (2022, unknown): “, they are effectively introduced for super-resolution [27, 14], inpainting [36, 14], and image-to-image translation [29], dance-to-music generation [42].”
Stochastic Solutions for Linear Inverse Problems using the Prior Implicit in a Denoiser (2021, background): “1 Earlier versions of this work were presented in [13].”
Flexible Diffusion Modeling of Long Videos (2022, methodology): “The combination of this diffusion process and a data distribution q ( x 0 , y ) (recalling that x 0 = x ) defines the joint distribution q ( x 0: T , y ) = q ( x 0 , y ) DDPMs work by “inverting” the diffusion process: given values of x t and y a neural network is used to parameterize p θ ( x t − 1 ”
Solving Inverse Problems in Medical Imaging with Score-Based Generative Models (2021, methodology): “Many existing works (Song et al., 2021; Kawar et al., 2021; Kadkhodaie & Simoncelli, 2020; Jalal et al., 2021) have implemented this idea in different ways.”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “Therefore, new generation procedures should be further studied [12, 104].”