From Denoising to Compressed Sensing

By Christopher A. Metzler, Arian Maleki, Richard Baraniuk. IEEE Transactions on Information Theory, 2014

672 citations recorded by the source corpus.

A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, todays denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired using a small number of randomized measurements. Typical CS reconstruction algorithms can be cast as iteratively estimating a signal from a perturbed observation. This paper answers a natural question: How can one effectively employ a generic denoiser in a CS reconstruction algorithm? In response, we develop an extension of the approximate message passing (AMP) framework, called denoising-based AMP (D-AMP), that can integrate a wide class of denoisers within its iterations. We demonstrate that, when used with a high-performance denoiser for natural images, D-AMP offers the state-of-the-art CS recovery performance while operating tens of times faster than competing methods. We explain the exceptional performance of D-AMP by analyzing some of its theoretical features. A key element in D-AMP is the use of an appropriate Onsager correction term in its iterations, which coerces the signal perturbation at each iteration to be very close to the white Gaussian noise that denoisers are typically designed to remove.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/1497bf2a6241803d626f9d618a608c2e0dfb2813

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing (2020, methodology): “In order to obtain stable results in the comparisons, NLRCS method is initialized by the estimated image of BM3DAMP in 8 iterations as in [23].”
A deep learning approach to structured signal recovery (2015, background): “These other algorithms include • One of the state-of-the-art methods that is the denoising-based approximate message passing (D-AMP) [34].”
Learning to invert: Signal recovery via Deep Convolutional Networks (2017, methodology): “Figure 2 shows the plot of average probability of successful recovery for different undersampling ratios ( M/N ) and three different recovery algorithms: D-AMP [10], total variation (TV) minimization [16], and P-AMP [17].”
DR2-Net: Deep Residual Reconstruction Network for Image Compressive Sensing (2017, positive): “Firstly, we compare our DR 2 -Net with ﬁve existing methods, i.e., TVAL3 [20], NLR-CS [7], D-AMP [25], SDA [26], and ReconNet [19].”
Deep Learning-Based Channel Estimation for Beamspace mmWave Massive MIMO Systems (2018, methodology): “Furthermore, the LDAMP network outperforms the start-of-the-art D-AMP algorithms even if BM3D-AMP is regarded as the most accurate algorithm for compressive signal recovery [10].”
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers (2019, background): “…al., 2017; Ono, 2017), ADMM with increasing penalty parameter (Brif-man et al., 2016; Chan et al., 2017), generalized approximate message passing (Metzler et al., 2016), Newton iteration (Buzzard et al., 2018), Fast Iterative Shrinkage-Thresholding Algorithm (Kamilov et al., 2017; Sun et al.,…”
Learned D-AMP: Principled Neural Network based Compressive Image Recovery (2017, methodology): “Note that the state evolution we provide here is the “deterministic” state-evolution proposed in [3] to predict the performance of D-AMP.”
Deep Learning for Massive MIMO CSI Feedback (2017, methodology): “We compare CsiNet with three state-of-the-art CS-based methods, namely, LASSO 1-solver [5], TVAL3 [7], and BM3D-AMP [8].”
One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models (2017, methodology): “proposed method is also similar to the denoising-based approximate message passing algorithm [34] and plug-andplay priors [45], which replace the proximal operator with an image denoiser.”
Rank Minimization for Snapshot Compressive Imaging (2018, methodology): “One recent proposal in CS is to use algorithms that are designed for other data processing tasks such as denoising [21], which employed advanced denoising algorithms in the approximate message passing framework to achieve state-of-the-art results in image CS.”
Convolutional Neural Network-Based Multiple-Rate Compressive Sensing for Massive MIMO CSI Feedback: Design, Simulation, and Analysis (2019, methodology): “12 ms while the LASSO [49], BM3D-AMP [50], and TVAL3 [51] need about 0.”
Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications (2016, background): “Around the same period of time there is an independent series of studies using denoisers for approximate message passing (AMP) algorithms [24]–[27].”
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing (2019, methodology): “For instance, it was reported in [16] that LISTA may be 20 times faster than ISTA after training; DUBLID [33] can be 1000 times faster than TV-based deblurring; ADMM-CSNet [12] can be about four times faster than the BM3D-AMP algorithm [44]; and CORONA [30] is over 50 times faster than the Fast-ISTA”
AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing (2020, methodology): “We compare AMP-Net-K with other eight state-of-the-art methods, namely TVAL3 [48], D-AMP [17], ReconNet [30], LDIT [44], LDAMP [44], DPDNN [39], NN [45] and ISTA-Net+ [37].”
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements (2016, methodology): “At low measurement rates of 0.1, 0.04 and 0.01, our algorithm yields superior quality reconstructions than the traditional iterative CS reconstruction algorithms, TVAL3, NLR-CS, and D-AMP.”
Image-Adaptive GAN based Reconstruction (2019, neutral): “The main drawback in restoring complex images using generative models is the limited representation capabilities of the generators.”
Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, methodology): “[27], to decouple the data term and prior term. In [28], Teodoro et al. plugged class-speciﬁc Gaussian mixture model (GMM) denoiser [4] into ADMM to solve image deblurring and compressive imaging. In [29], Metzler et al. developed a denoising-based approximate message passing (AMP) method to integra”
Snapshot Compressive Imaging: Theory, Algorithms, and Applications (2021, background): “It has been pointed out in several papers [80], [81] that optimization-based algorithms include two essential steps: 1) Gradient descent: updating the current estimate by extracting more information from the measurement 2) Projection to the signal domain: confining the result to the desired signal s”
Deep Learning-Based CSI Feedback Approach for Time-Varying Massive MIMO Channels (2018, methodology): “BM3D-AMP achieves the most accurate recovery performance on natural images and runs 10 times faster than other iterative methods.”