Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing

By Vishal Monga, Yuelong Li, Yonina C. Eldar. IEEE Signal Processing Magazine, 2019

1,382 citations recorded by the source corpus.

Deep neural networks provide unprecedented performance gains in many real-world problems in signal and image processing. Despite these gains, the future development and practical deployment of deep networks are hindered by their black-box nature, i.e., a lack of interpretability and the need for very large training sets. An emerging technique called algorithm unrolling, or unfolding, offers promise in eliminating these issues by providing a concrete and systematic connection between iterative algorithms that are widely used in signal processing and deep neural networks. Unrolling methods were first proposed to develop fast neural network approximations for sparse coding. More recently, this direction has attracted enormous attention, and it is rapidly growing in both theoretic investigations and practical applications. The increasing popularity of unrolled deep networks is due, in part, to their potential in developing efficient, high-performance (yet interpretable) network architectures from reasonably sized training sets.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/4f71dc75896529d8653d8ef0ccc72bcce2c80e18

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Yi Ma
2. John Wright
3. Lei Zhang
4. Khaled Ben Letaief
5. Wotao Yin
6. Chong You
7. Huan Fu
8. Zhangyang Wang

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Ming-Hsuan Yang
3. Fahad Shahbaz Khan
4. Dacheng Tao
5. Lei Zhang
6. Salman Hameed Khan
7. Ling Shao
8. Chen Change Loy

In Signal Processing

1. Yi Ma
2. John Wright
3. Zhen Liu
4. Khaled Ben Letaief
5. Munawar Hayat
6. Adrian Weller
7. Salman Hameed Khan
8. Jianhua Lu

Citations in the corpus

• Learning with Hyperspherical Uniformity (2021, methodology): “To optimize the standard MHP, we propose to unroll the inner minimization with a few gradient descent steps and then embed it back to the outer maximization, which is conceptually similar to [18, 23, 58, 2].”
• Deep Networks from the Principle of Rate Reduction (2020, background): “Later Monga et al. (2019); Sun et al. (2020) have proposed similar interpretation of deep networks as unrolling algorithms for sparse coding.”
• Transformers in Medical Imaging: A Survey (2022, methodology): “These hybrid methods contain special domain-specific layers, and include unrolled optimization [77], generative models [78], and learned denoiser-based approaches [79].”
• Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, methodology): “signal processing provides a promising modeling framework to simultaneously detect the active devices and estimate their channels [75], [76], which is supported by various efficient algorithms, including the approximate message passing algorithm [184], [185] and DNN algorithm unrolling approach [186”
• Learning to Optimize: A Primer and A Benchmark (2021, methodology): “Algorithm unrolling unrolls and truncates optimization algorithm into the structure of a neural network [159].”