The Little Engine That Could: Regularization by Denoising (RED)

By Yaniv Romano, Michael Elad, Peyman Milanfar. SIAM Journal of Imaging Sciences, 2016

986 citations recorded by the source corpus.

Removal of noise from an image is an extensively studied problem in image processing. Indeed, the recent advent of sophisticated and highly effective denoising algorithms has led some to believe that existing methods are touching the ceiling in terms of noise removal performance. Can we leverage this impressive achievement to treat other tasks in image processing? Recent work has answered this question positively, in the form of the Plug-and-Play Prior ($P^3$) method, showing that any inverse problem can be handled by sequentially applying image denoising steps. This relies heavily on the ADMM optimization technique in order to obtain this chained denoising interpretation. Is this the only way in which tasks in image processing can exploit the image denoising engine? In this paper we provide an alternative, more powerful, and more flexible framework for achieving the same goal. As opposed to the $P^3$ method, we offer Regularization by Denoising (RED): using the denoising engine in defining the regulariza...

Subfields: Biomedical Engineering, Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b886b8f712a122cbe6af1376981a63d51cff6de4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Inverting Deep Generative models, One layer at a time (2019, background): “Meanwhile, this problem is a starting point for general linear inverse problems: since several recent works leverage inversion as a key step in solving more general inverse problems, see e.g. [23, 21].”
Denoising Prior Driven Deep Neural Network for Image Restoration (2018, methodology): “In addition to these explicitly regularized IR methods, denoising-based IR methods have also been proposed [26], [27], [28], [29], [30].”
Deep Mean-Shift Priors for Image Restoration (2017, methodology): “[27] designed a prior model that is also implemented by a denoiser, but that does not build on a proximal formulation such as ADMM.”
WIRE: Wavelet Implicit Neural Representations (2023, background): “In the past decade, datadriven regularization, including overcomplete dictionarybased [4] and generative network-based [29, 35, 36] techniques, have been developed.”
Denoising Diffusion Models for Plug-and-Play Image Restoration (2023, methodology): “[19] proposed a Moreau-Yosida regularised unadjusted Langevin algorithm for Bayesian computation such as imaging inverse problems.”
Neural Nearest Neighbors Networks (2018, background): “Moreover, denoising has been recognized as a core module for density estimation [2] and serves as a sub-routine for more general image restoration tasks in a ﬂurry of recent work, e. g. [5, 36, 52].”
Learning Deep CNN Denoiser Prior for Image Restoration (2017, background): “Some of the other related work can be found in [6, 12, 48, 49, 54, 58].”
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers (2019, methodology): “(Danielyan et al., 2012; Egiazarian & Katkovnik, 2015) use the notion of Nash equilibrium to propose a scheme similar to PnP. (Danielyan et al., 2010) proposed an augmented Lagrangian method similar to PnP. (Romano et al., 2017; Reehorst & Schniter, 2019) presented Regularization by Denoising (RED),”
Toward Convolutional Blind Denoising of Real Photographs (2018, background): “With decades of studies, numerous promising approaches [3, 12, 17, 53, 11, 61] have been developed and near-optimal performance [8, 31, 50] has been achieved for the removal of additive white Gaussian noise (AWGN).”
Real Image Denoising With Feature Attention (2019, positive): “Furthermore, many image restoration tasks can be solved in the unrolled inference through variable splitting methods by a set of denoising subtasks, which further widens the applicability of image denoising [3, 33, 51, 64].”
Multi-level Wavelet-CNN for Image Restoration (2018, background): “On the other hand, recent studies have revealed that one can plug CNN-based denoisers into model-based optimization methods for solving more complex image restoration tasks [40, 58], which also promotes the widespread use of CNNs.”
Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, background): “On the other hand, there is no doubt that parameter setting is always a non-trivial issue [37].”
Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise (2022, methodology): “Iterative neural models have been used for various inverse problems [Romano et al., 2016, Metzler et al., 2017].”
Learned Primal-Dual Reconstruction (2017, background): “We follow the observation in [21] and [22], that proximal operators can be replaced by other operators that are not necessarily proximal operators.”
GAN-Based Projector for Faster Recovery With Convergence Guarantees in Linear Inverse Problems (2019, background): “Neural networkbased inverse problem solvers have been explored recently [14, 35, 31, 1, 12, 15, 25, 32, 22, 37, 26].”
NETT: solving inverse problems with deep neural networks (2018, methodology): “We note that an early related work [43] uses denoisers as regularizers which also includes certain CNNs.”