Deep Generative Adversarial Networks for Compressed Sensing Automates MRI

By Morteza Mardani, Enhao Gong, Joseph Y. Cheng, Shreyas S. Vasanawala, Greg Zaharchuk, Marcus T. Alley, N. Thakur, Song Han, William (Bill) J. Dally, John M. Pauly and 1 more. 2017

164 citations recorded by the source corpus.

Magnetic resonance image (MRI) reconstruction is a severely ill-posed linear inverse task demanding time and resource intensive computations that can substantially trade off {\it accuracy} for {\it speed} in real-time imaging. In addition, state-of-the-art compressed sensing (CS) analytics are not cognizant of the image {\it diagnostic quality}. To cope with these challenges we put forth a novel CS framework that permeates benefits from generative adversarial networks (GAN) to train a (low-dimensional) manifold of diagnostic-quality MR images from historical patients. Leveraging a mixture of least-squares (LS) GANs and pixel-wise $\ell_1$ cost, a deep residual network with skip connections is trained as the generator that learns to remove the {\it aliasing} artifacts by projecting onto the manifold. LSGAN learns the texture details, while $\ell_1$ controls the high-frequency noise. A multilayer convolutional neural network is then jointly trained based on diagnostic quality images to discriminate the projection quality. The test phase performs feed-forward propagation over the generator network that demands a very low computational overhead. Extensive evaluations are performed on a large contrast-enhanced MR dataset of pediatric patients. In particular, images rated based on expert radiologists corroborate that GANCS retrieves high contrast images with detailed texture relative to conventional CS, and pixel-wise schemes. In addition, it offers reconstruction under a few milliseconds, two orders of magnitude faster than state-of-the-art CS-MRI schemes.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

An overview of deep learning in medical imaging focusing on MRI (2018, neutral): “[70] Lin M, Chen Q, Yan S. Network in network, arXiv:1312.4400 (2013).”
Generative Adversarial Network in Medical Imaging: A Review (2018, neutral): “Mardani et al. (2017) pix2pix ∗ L1, 2, 15 – M1 [ √ ] Under-sampled K-space”
Learned Primal-Dual Reconstruction (2017, background): “Similar approaches for MRI reconstruction have also been considered [14], [15].”
Deep Learning Techniques for Inverse Problems in Imaging (2020, background): “the range of a generator, and was also independently proposed in [139]–[141].”
LEARN: Learned Experts’ Assessment-Based Reconstruction Network for Sparse-Data CT (2017, unknown): “There are some differences between these two studies: [57] kept the nonzero-padded parts unchanged, and [58] introduced a residual block.”
Cancer Diagnosis Using Deep Learning: A Bibliographic Review (2019, methodology): “GANs have been used for various applications including reconstruction [114–116], segmentation [117–119], domain adaptation [120,121] and detection [122,123].”
GANs for Medical Image Analysis (2018, methodology): “WGAN MSE, WGAN MRI (Brain) unknown Train:891 Test:111 PSNR=35.88 SSIM=0.94 NRMSE=0.0852 Yes No Table 14: Reconstruction GAN-based methods - other method Arch loss modality dataset performance PR code Mardani et al. (2017) GANCS ResNet, LSGAN Adv MRI (Chest) contrast-enhanced MRI abdomen pediatric pa”
Robust Compressed Sensing MRI with Deep Generative Priors (2021, background): “[59] Morteza Mardani, Enhao Gong, Joseph Y Cheng, Shreyas Vasanawala, Greg Zaharchuk, Marcus Alley, Neil Thakur, Song Han, William Dally, John M Pauly, et al.”
How Generative Adversarial Networks and Their Variants Work (2017, background): “Their capability to represent complex and high-dimensional data can be utilized in treating images [2, 13, 60, 68, 132, 138, 150], videos [127, 130, 131], music generation [43, 69, 146], natural languages [51, 76] and other academic domains such as medical images [17, 80, 141] and security [114, 129”
On the use of deep learning for computational imaging (2020, background): “31,36 One of the earliest attempts to solve the super resolution problem ∆ ρ b was by training a neural network to receive images by a low-NA objective as inputs and reproduce images of high-NA quality as outputs.”
Quantitative susceptibility mapping using deep neural network: QSMnet (2018, methodology): “Deep neural networks have also been applied for medical image reconstruction such as X-ray, CT, PET, and MRI (Hammernik et al., 2018; Kwon et al., 2016; Mardani et al., 2017; Prasoon et al., 2013; Teramoto et al., 2016; Xu et al., 2017; Yang et al., 2018; Zhu et al., 2017; Zichuan et al., 2017).”
Solving Inverse Problems in Medical Imaging with Score-Based Generative Models (2021, methodology): “With the rise of machine learning, many methods (Zhu et al., 2018; Mardani et al., 2017; Shen et al., 2019; Würﬂ et al., 2018; Ghani & Karl, 2018; Wei et al., 2020) have been proposed for medical image reconstruction using a small number of measurements.”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “By estimating real data probability distribution pdata(x) where x exists in the d-dimensional real space R, generative models can be applied to various tasks such as semi-supervised learning [97], [22],[64], image translation [52],[56], [11], domain adaptation [2],[13],[100] and other academic domai”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, methodology): “Cardiac Segmentation using short-Axis MRI [104].”
Neural Proximal Gradient Descent for Compressive Imaging (2018, background): “Note, one can additionally augment (P1) with adversarial GAN loss as in our companion work [18] that favors more the image perceptual quality that is critical in medical imaging.”
Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images (2018, unknown): “GANs were introduced by Goodfellow et al. in [34] and have since been used for a variety of medical imaging applications including segmentation [35–37], detection [38, 39], reconstruction [40–42], domain adaptation [43–46].”
Deep learning enables reduced gadolinium dose for contrast‐enhanced brain MRI (2018, methodology): “Finally, here we used the MAE loss as cost function to train the DL model, which can be further improved by using multiple loss functions(39) or using Generative Adversarial Network (GAN) as a data-driven cost function for reconstruction.(26) In this work we used a fixed level of gadolinium contrast”
Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training (2017, methodology): “GANs have also been used for segmentation [31], [32], detection [33], reconstruction [34], [35] and classification [36].”