Learning Optimal Nonlinearities for Iterative Thresholding Algorithms
By Ulugbek S. Kamilov, Hassan Mansour. 2015
113 citations recorded by the source corpus.
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simple feedforward neural network and developing a corresponding error backpropagation algorithm for fine-tuning the thresholding functions. Simulations on sparse statistical signals illustrate potential gains in estimation quality due to the proposed data adaptive ISTA.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Bernard Ghanem
- Philip Schniter
- Jian Zhang
- Christopher A. Metzler
- Ali Mousavi
- Richard Baraniuk
- Sundeep Rangan
- Mark Borgerding
In Computer Vision and Pattern Recognition
- Bernard Ghanem
- Jian Zhang
- Richard Baraniuk
- Ali Mousavi
- Michael T. McCann
- Kyong Hwan Jin
- Christopher A. Metzler
- Michael A. Unser
Citations in the corpus
- Interpretable Recurrent Neural Networks Using Sequential Sparse Recovery (2016, background): “Kamilov and Mansour [4] learned improved ISTA nonlinearities from data.”
- Deep Convolutional Neural Network for Inverse Problems in Imaging (2016, background): “Others have investigated learning optimal shrinkage operators for deep-layered neural networks [35], [37].”
- AMP-Inspired Deep Networks for Sparse Linear Inverse Problems (2016, methodology): “The shrinkage in [6] was constructed using 8000 shifts of β(z) spread uniformly over the dynamic range of the signal, each scaled by an adjustable weight.”
- Learned D-AMP: Principled Neural Network based Compressive Image Recovery (2017, methodology): “The inspiration for this work comes from the LISTA [12; 13] and LAMP/LVAMP [14] neural networks.”
- ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing (2017, methodology): “In the context of CS for sparse signals, Kamilov and Mansour propose to learn the optimal thresholding functions for ISTA based on a B-spline decomposition [17].”