Learned Primal-Dual Reconstruction

By Jonas Adler, Ozan Öktem. IEEE Transactions on Medical Imaging, 2017

894 citations recorded by the source corpus.

We propose the Learned Primal-Dual algorithm for tomographic reconstruction. The algorithm accounts for a (possibly non-linear) forward operator in a deep neural network by unrolling a proximal primal-dual optimization method, but where the proximal operators have been replaced with convolutional neural networks. The algorithm is trained end-to-end, working directly from raw measured data and it does not depend on any initial reconstruction such as filtered back-projection (FBP). We compare performance of the proposed method on low dose computed tomography reconstruction against FBP, total variation (TV), and deep learning based post-processing of FBP. For the Shepp-Logan phantom we obtain >6 dB peak signal to noise ratio improvement against all compared methods. For human phantoms the corresponding improvement is 6.6 dB over TV and 2.2 dB over learned post-processing along with a substantial improvement in the structural similarity index. Finally, our algorithm involves only ten forward-back-projection computations, making the method feasible for time critical clinical applications.

Subfields: Pulmonary and Respiratory Medicine, Radiation, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/405be55d237fcbb6b4926f47f90a13ffb92b7c34

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Pulmonary and Respiratory Medicine

In Radiation

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Neural Proximal Gradient Descent for Compressive Imaging (2018, background): “The second category is inspired by unrolling the iterations of classical optimization algorithms, and learns the filters and nonlinearities by training deep CNNs [25, 16, 26, 27].”
Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network (2017, background): “Compared to the recent proposals of learning-based optimization approaches [16], [24], one of the important advantages of our work is that our deep network is no more a black box but can be optimized for speciﬁc restoration tasks by choosing optimal framelet representation.”
AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing (2020, methodology): “For example, ADMM is unfolded for MRI [38] and SCI [4], and the primal-dual algorithm is developed to reconstruct CT [43].”
FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging (2020, background): “proposed an iterative Convolutional Neural Network (CNN) to learn the gradient descent [13], [20].”
ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing (2020, methodology): “The methods in [61], [62], [63] unroll a primal-dual optimization method or a (projected) gradient descent method for low-dose”
Deep Learning Techniques for Inverse Problems in Imaging (2020, methodology): “algorithms that have been investigated in an unrolling context include the alternating directions method of multipliers (ADMM) [19], primal-dual methods [83], half-quadratic split-”
Deep Generative Adversarial Neural Networks for Compressive Sensing (GANCS) MRI (2018, background): “In another approach, [1], [4], [6], [10], [12], and [29] use deep neural networks to learn image priors, or, sparsifying transforms from historical data.”
Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks (2020, neutral): “PD-Net, the learnable PD method proposed by Adler and Öktem for tomographic reconstruction [10], is the unrolled version of the PDHG algorithm and has been applied to MRI reconstruction [11].”
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk (2017, background): “These more sophisticated priors are recently emerging in empirical applications of many fields of imaging, such as medical imaging [70, 30, 24, 55, 17, 1, 49, 68, 53, 43, 32, 42, 16, 69], microscopy [59], inpainting [45, 71], superresolution [60, 33, 28], compressed sensing [38, 37, 51, 50], image m”
Learning to Optimize: A Primer and A Benchmark (2021, methodology): “Other work that applies model-based L2O to medical and biology imaging includes [6; 202; 5; 63; 134; 62; 162; 78; 185; 36].”
Small Sample Learning in Big Data Era (2018, methodology): “, 2017), sparse view/data Computed Tomography(CT) reconstruction (Gupta et al., 2018; Chen et al., 2017; Adler and Öktem, 2018), compressive image reconstruction (Metzler et al.”
Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT (2017, methodology): “This work was followed by many novel extensions for low-dose CT [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27].”
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing (2019, methodology): “[36] unroll the primal–dual hybrid gradient algorithm, a well-known technique for primal–dual nonsmooth optimization.”
Deep Unfolding Network for Image Super-Resolution (2020, background): “…based on certain optimization algorithms ( e.g ., half-quadratic splitting [2], alternating direction method of multipliers [6] and primal-dual [1, 9]) have been proposed to solve different image restoration tasks, such as image de-noising [11, 32], image deblurring [29, 49], image compressive…”
Model-Based Learning for Accelerated, Limited-View 3-D Photoacoustic Tomography (2017, methodology): “Alternatively following the second approach by including the physical forward model into the network has been studied in [23], [24], [25], [26], [27].”