Solving ill-posed inverse problems using iterative deep neural networks

By Jonas Adler, Ozan Öktem. 2017

687 citations recorded by the source corpus.

We propose a partially learned approach for the solution of ill-posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularisation theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularising functional. The method results in a gradient-like iterative scheme, where the ‘gradient’ component is learned using a convolutional network that includes the gradients of the data discrepancy and regulariser as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against filtered backprojection and total variation reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the total variation reconstruction while being significantly faster, giving reconstructions of 512×512 pixel images in about 0.4 s using a single graphics processing unit (GPU).

Subfields: Biomedical Engineering, Mathematical Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Mathematical Physics

Citations in the corpus

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, background): “fine-tuning the trained surrogate on test input [83, 84], learned gradient update [85, 86], meta-learning on a distribution of regression tasks [87], etc.”
Neural Operator: Learning Maps Between Function Spaces (2021, positive): “It remedies the mesh-dependent nature of the approach in the papers (Guo et al., 2016; Zhu and Zabaras, 2018; Adler and Oktem, 2017; Bhatnagar et al., 2019) by producing a single set of network parameters that can be used with different discretizations.”
Learned Primal-Dual Reconstruction (2017, methodology): “1) Ellipse Phantoms: This problem is identical to [5] and we restate it briefly.”
Deep D-Bar: Real-Time Electrical Impedance Tomography Imaging With Deep Neural Networks (2017, background): “This successful transition to experimental data underlines the robustness of the D-bar algorithm and is especially important as the need for good training data is often the bottleneck for the success of such network-based approaches for other imaging modalities, [14], [18], [19].”
DeepView: View Synthesis With Learned Gradient Descent (2019, methodology): “We also measured the effect of varying the number of LGD iterations from 1 to 4.”
Learning to Optimize: A Primer and A Benchmark (2021, methodology): “Other work that applies model-based L2O to medical and biology imaging includes [6; 202; 5; 63; 134; 62; 162; 78; 185; 36].”
Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction (2017, positive): “The second class of approaches attempts to make each stage of iterative optimisation learnable by unrolling the end-to-end pipeline into a deep network [9], [16]–[19].”
Multipole Graph Neural Operator for Parametric Partial Differential Equations (2020, background): “The ﬁrst parametrizes the solution operator as a deep convolutional neural network (CNN) between ﬁnite-dimensional Euclidean spaces F θ : R n → R n [21, 44, 3, 9, 40].”
One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models (2017, background): “Many recent works solve linear inverse problems by unrolling the optimization process into the network architecture [3,6,22,26].”
Choose a Transformer: Fourier or Galerkin (2021, methodology): “Deep convolutional neural networks (DCNN) are widely applied to learn the solution maps with a fixed discretization size [1, 9, 40, 36, 35, 100, 86].”
Multiwavelet-based Operator Learning for Differential Equations (2021, background): “The FNO model with higher values of k m has better performance due to more Fourier bases for representing the high-frequency signal, while MWT does better even with low modes in its A, B, C CNNs, highlighting the importance of using wavelet-based ﬁlters in the signal processing.”
Internet of Underwater Things and Big Marine Data Analytics—A Comprehensive Survey (2020, methodology): “Finally, the new concept of image reconstruction with DL [190], [250], [251] has promise for future research.”
On the use of deep learning for computational imaging (2020, background): “Among the ﬁrst uses of this approach were for tomography 24,25,29 but its origins are in an insight by Gregor and LeCun.”
FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging (2020, background): “proposed an iterative Convolutional Neural Network (CNN) to learn the gradient descent [13], [20].”