Deep Convolutional Neural Network for Inverse Problems in Imaging

By Kyong Hwan Jin, Michael T. McCann, Emmanuel Froustey, Michael A. Unser. IEEE Transactions on Image Processing, 2016

2,439 citations recorded by the source corpus.

In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyperparameter selection. The starting point of this paper is the observation that unrolled iterative methods have the form of a CNN (filtering followed by pointwise non-linearity) when the normal operator (<inline-formula> <tex-math notation="LaTeX">$H^{*}H$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$H^{*}$ </tex-math></inline-formula> is the adjoint of the forward imaging operator, <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a <inline-formula> <tex-math notation="LaTeX">$512\times 512$ </tex-math></inline-formula> image on the GPU.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/8a68ca7c5ba4d92de3435961f88188d0b769b920

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Learned Primal-Dual Reconstruction (2017, methodology): “We re-implemented the algorithm from [7] but found that using the training procedure as stated in the paper gave sub-optimal results.”
Deep Identity-aware Transfer of Facial Attributes (2016, background): “ted success in versatile high level vision problems [24], [25], [26], [27], [28], but also exhibited their remarkable power in understanding, generating, and recovering images [16], [29], [30], [31], [32], [33]. In this section, we focus on the task of facial attribute transfer, and brieﬂy survey th”
Classification of COVID-19 patients from chest CT images using multi-objective differential evolution–based convolutional neural networks (2020, methodology): “Rectified linear unit (ReLU) activation function is used to learn complex functional mappings between the inputs and response variables [31, 32] .”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, background): “Multiple works have shown an effective application for this in computed tomography (CT) reconstructions [38, 121].”
Neural Compatibility Modeling With Probabilistic Knowledge Distillation (2020, background): “Deep neural networks have achieved distinguished performance in various application domains ranging from natural language processing [14][15] to computer vision [16][17][18].”
FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging (2020, background): “FBPConvNet is a network-based method.”
NETT: solving inverse problems with deep neural networks (2018, background): “Very recently, several deep learning approaches for inverse problems have been developed.[1, 3, 8, 23, 24, 39, 21, 36, 40, 41].”
MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques (2023, background): “Machine learning approaches, especially deep learning, can also play a vital role in the analysis, segmentation, and classification of cancer images, especially brain tumors [26].”
On the use of deep learning for computational imaging (2020, background): “Among the ﬁrst uses of this approach were for tomography 24,25,29 but its origins are in an insight by Gregor and LeCun.”
Phase recovery and holographic image reconstruction using deep learning in neural networks (2017, positive): “Deep learning has recently been applied to solving inverse problems in imaging science such as in super-resolution 43,44 , acceleration of the image acquisition speed of computed tomography (CT) 45 , magnetic resonance imaging (MRI) 46 , photoacoustic tomography 47 and holography 48,49 .”
Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT (2017, methodology): “Thus, the U-net architecture is good for the sparse view CT reconstruction to deal with the globally distributed streaking artifacts [16], [17].”
Deep Learning with Domain Adaptation for Accelerated Projection Reconstruction MR (2017, background): “Jin et al [19] and Han et al [20] independently proposed multi-scale residual learning networks using U-Net [9] to remove the global streaking artifacts caused by sparse projection views.”