AMP-Inspired Deep Networks for Sparse Linear Inverse Problems

By Mark Borgerding, Philip Schniter, Sundeep Rangan. IEEE Transactions on Signal Processing, 2016

428 citations recorded by the source corpus.

Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem, where one seeks to recover a sparse signal from a few noisy linear measurements. In this paper, we propose two novel neural-network architectures that decouple prediction errors across layers in the same way that the approximate message passing (AMP) algorithms decouple them across iterations: through Onsager correction. First, we propose a “learned AMP” network that significantly improves upon Gregor and LeCun's “learned ISTA.” Second, inspired by the recently proposed “vector AMP” (VAMP) algorithm, we propose a “learned VAMP” network that offers increased robustness to deviations in the measurement matrix from i.i.d. Gaussian. In both cases, we jointly learn the linear transforms and scalar nonlinearities of the network. Interestingly, with i.i.d. signals, the linear transforms and scalar nonlinearities prescribed by the VAMP algorithm coincide with the values learned through back-propagation, leading to an intuitive interpretation of learned VAMP. Finally, we apply our methods to two problems from 5G wireless communications: compressive random access and massive-MIMO channel estimation.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/756bf25afe8e6ca92ba1795bcb5fb4e93d8ed6b2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Bernard Ghanem
2. A. Dimakis
3. Shi Jin
4. Chao-Kai Wen
5. Jian Zhang
6. Wotao Yin
7. Geoffrey Y. Li
8. Richard Baraniuk

In Computer Vision and Pattern Recognition

1. Zhangyang Wang
2. Bernard Ghanem
3. Shi Jin
4. Chao-Kai Wen
5. A. Dimakis
6. Geoffrey Y. Li
7. Wuyang Chen
8. Wotao Yin

Citations in the corpus

• Deep Learning Techniques for Inverse Problems in Imaging (2020, background): “ting [105], [106], block coordinate descent [107]–[109], alternating minimization [82], and approximate message passing [88], [110].”
• AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing (2020, methodology): “balance between reconstruction speed and interpretation. Deep unfolding methods were first developed to solve the sparse linear inverse problem and the unfolded algorithms include ISTA [33], [37], AMP [38] and iterative hard thresholding (IHT) algorithm [39]. However, it is usually necessary for thes”
• ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing (2017, methodology): “Mark et al . extend approximate message passing (AMP) algorithms to so-called Learned AMP networks for solving sparse linear inverse problems [5].”
• Model-Driven Deep Learning for MIMO Detection (2020, background): “On the other hand, model-driven DL constructs the network topology based on known domain knowledge and has been successfully applied to image reconstruction [32], sparse signal recovery [33]–[36], and wireless communications recently [1], [12].”
• Deep Learning in Mobile and Wireless Networking: A Survey (2018, methodology): “[355] Sparse linear inverse problem in MIMO CNN Fujihashi et al.”
• Deep Learning for Physical-Layer 5G Wireless Techniques: Opportunities, Challenges and Solutions (2019, background): “[6] M.”
• Learning to Optimize: A Primer and A Benchmark (2021, methodology): “We compare the following model-based L2O approaches, all of which are unrolled to 16 iterations: (1) a feed-forward version LISTA as proposed in [75], with untied weights across layers; (2) LISTA-CP, a variant of LISTA with weight coupling [235; 43]; (3) LISTACPSS, a variant of LISTA with weight cou”