ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing

By Jian Zhang, Bernard Ghanem. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

1,342 citations recorded by the source corpus.

With the aim of developing a fast yet accurate algorithm for compressive sensing (CS) reconstruction of natural images, we combine in this paper the merits of two existing categories of CS methods: the structure insights of traditional optimization-based methods and the speed of recent network-based ones. Specifically, we propose a novel structured deep network, dubbed ISTA-Net, which is inspired by the Iterative Shrinkage-Thresholding Algorithm (ISTA) for optimizing a general $$ norm CS reconstruction model. To cast ISTA into deep network form, we develop an effective strategy to solve the proximal mapping associated with the sparsity-inducing regularizer using nonlinear transforms. All the parameters in ISTA-Net (e.g. nonlinear transforms, shrinkage thresholds, step sizes, etc.) are learned end-to-end, rather than being hand-crafted. Moreover, considering that the residuals of natural images are more compressible, an enhanced version of ISTA-Net in the residual domain, dubbed ISTA-Net+, is derived to further improve CS reconstruction. Extensive CS experiments demonstrate that the proposed ISTA-Nets outperform existing state-of-the-art optimization-based and network-based CS methods by large margins, while maintaining fast computational speed. Our source codes are available: http://jianzhang.tech/projects/ISTA-Net.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2cdc9bbde0f847094582b212f980fa4dbc48950d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Stochastic Solutions for Linear Inverse Problems using the Prior Implicit in a Denoiser (2021, methodology): “TVAL3 [53] is an optimization algorithm using total variation regularization, ISTA-Net [54] is a block-based supervised CNN method trained to reconstruct images from measurements obtained from a single pre-speciﬁed measurement matrix.”
AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing (2020, methodology): “owledge and the structure of the operators. Motivated by the iterative algorithm that processes data step by step, deep unfolding maps iterative restoration algorithms onto deep neural networks [33], [34], [35], [36]. It tries to make compromise between the iterative recovery methods and network met”
Deep Learning on Image Denoising: An overview (2019, methodology): “ons in the ﬁeld of image processing, especially image denoising. 6 2.3. CNNs for image denoising Due to their plug-and-play network architectures, CNNs have achieved great success in image processing [257, 145, 121]. As a pioneer in CNN technology, LeNet [113] used convolutional kernels of different”
Small Sample Learning in Big Data Era (2018, methodology): “, 2017; Adler and Öktem, 2018), compressive image reconstruction (Metzler et al., 2017; Zhang and Ghanem, 2018).”
Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks (2020, neutral): “Iterative shrinkage-thresholding algorithm networks The iterative shrinkage-thresholding algorithm network (ISTA-Net) [7] aims to solve the general CS reconstruction problem with L1 regularization i i ( ) g 1 $ $ = in (7).”
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers (2019, methodology): “4 Some recent deep-learning based methods (Yang et al., 2016; Kulkarni et al., 2016; Metzler et al., 2017; Zhang & Ghanem, 2018) are not compared here because we assume we do not have enough medical images for training.”
Learning to Optimize: A Primer and A Benchmark (2021, methodology): “This has yielded great interest in L2O schemes, as evidenced by the fact the works [75; 221; 189; 223; 255; 43; 126; 201; 4; 98; 246; 78; 3; 19], among others, provide various ways to unroll the original ISTA algorithm.”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “Moreover, most DUN methods are beset by information loss due to the feature-to-image transformation at the end of each stage.”
Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser (2020, methodology): “All the numbers are taken from [54] except for ISTA-Net which were obtained by running their open-source code.”
Image Compressed Sensing Using Convolutional Neural Network (2020, methodology): “In [28], Zhang et al.”
Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, positive): “It should be noted that, apart from plug-and-play IR, deep unfolding IR [45], [46], [47], [48] can also incorporate the advantages of both model-based methods and learning-based methods.”
FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging (2020, methodology): “Laplacian regularization [51], [52], FISTA-TV [60], FBPConvNet [21] and ISTA-Net [31].”
Deep Unfolding Network for Image Super-Resolution (2020, background): “…e.g ., half-quadratic splitting [2], alternating direction method of multipliers [6] and primal-dual [1, 9]) have been proposed to solve different image restoration tasks, such as image de-noising [11, 32], image deblurring [29, 49], image compressive sensing [61, 63], and image demosaicking [28].”
Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data (2019, methodology): “, visually outperformed CG-SENSE at R = 2 according to the reader study. Most DL-MRI approaches use supervised learning for network trainingin order to provide improved accelerated MRI reconstruction (26,27,30,31,52). However, acquiring fully-sampled data is challenging in many practical scenarios o”
Simple and Deep Graph Convolutional Networks (2020, background): “…range of applications, including social analysis (Qiu et al., 2018; Li & Goldwasser, 2019), traffic prediction (Guo et al., 2019; Li et al., 2019), biology (Fout et al., 2017; Shang et al., 2019), recommender systems (Ying et al., 2018), and computer vision (Zhao et al., 2019; Ma et al., 2019).”