Image reconstruction by domain-transform manifold learning

By Bo Zhu, Jeremiah Zhe Liu, Bruce R. Rosen, Matthew S. Rosen. 2017

1,685 citations recorded by the source corpus.

Image reconstruction is essential for imaging applications across the physical and life sciences, including optical and radar systems, magnetic resonance imaging, X-ray computed tomography, positron emission tomography, ultrasound imaging and radio astronomy. During image acquisition, the sensor encodes an intermediate representation of an object in the sensor domain, which is subsequently reconstructed into an image by an inversion of the encoding function. Image reconstruction is challenging because analytic knowledge of the exact inverse transform may not exist a priori, especially in the presence of sensor non-idealities and noise. Thus, the standard reconstruction approach involves approximating the inverse function with multiple ad hoc stages in a signal processing chain, the composition of which depends on the details of each acquisition strategy, and often requires expert parameter tuning to optimize reconstruction performance. Here we present a unified framework for image reconstruction—automated transform by manifold approximation (AUTOMAP)—which recasts image reconstruction as a data-driven supervised learning task that allows a mapping between the sensor and the image domain to emerge from an appropriate corpus of training data. We implement AUTOMAP with a deep neural network and exhibit its flexibility in learning reconstruction transforms for various magnetic resonance imaging acquisition strategies, using the same network architecture and hyperparameters. We further demonstrate that manifold learning during training results in sparse representations of domain transforms along low-dimensional data manifolds, and observe superior immunity to noise and a reduction in reconstruction artefacts compared with conventional handcrafted reconstruction methods. In addition to improving the reconstruction performance of existing acquisition methodologies, we anticipate that AUTOMAP and other learned reconstruction approaches will accelerate the development of new acquisition strategies across imaging modalities.

Subfields: Geophysics, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Geophysics

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

An overview of deep learning in medical imaging focusing on MRI (2018, methodology): “26 A unified framework for image reconstruction [139], called automated transform by manifold approximation (AUTOMAP) consisting of a feedforward deep neural network with fully connected layers followed by a sparse convolutional autoencoder, formulate image reconstruction generically as a data-drive”
Deep Learning in Neuroradiology (2018, methodology): “techniques, such as Cartesian and non-Cartesian acquisition schemes.(10) Combining deep learning to k-space undersampling”
Solving Inverse Problems in Medical Imaging with Score-Based Generative Models (2021, methodology): “With the rise of machine learning, many methods (Zhu et al., 2018; Mardani et al., 2017; Shen et al., 2019; Würﬂ et al., 2018; Ghani & Karl, 2018; Wei et al., 2020) have been proposed for medical image reconstruction using a small number of measurements.”
Deep learning in medical imaging and radiation therapy. (2018, methodology): “Zhu et al. 336 leveraged the structured information (e.g., data from individual patients) when selecting batch of candidate unlabeled samples.”
On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. (2020, methodology): “AI-based image reconstruction approaches are being proposed that incorporate fast and image quality–enhancing mechanisms, operating directly from k-space (41) or in combination with new techniques for MR fingerprinting (42).”
TransMorph: Transformer for unsupervised medical image registration (2021, neutral): “2019), image reconstruction (Zhu et al. 2018), and disease diagnostics (Lian et al.”
Artificial intelligence in radiology (2018, positive): “…efforts report the flexibility of deep learning in learning reconstruction transformations for various MRI acquisition strategies, which is achieved by treating the reconstruction process as a supervised learning task where a mapping between the scanner sensors and resultant images is derived 75 .”
Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems (2017, methodology): “21 (see [78] for more calculation of required parameter numbers).”
Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks (2020, neutral): “AUTOMAP [15] can reconstruct the image directly from the non-Cartesian samples, as the forward operator nonuniform FFT is learned by the network.”
LIA: Latently Invertible Autoencoder with Adversarial Learning (2019, background): “Among emerging algorithms of novel forms due to deep learning, deep generative models play a more and more important role in cracking challenges in various scientific disciplines: high-quality image generation [15, 47, 17, 18, 4], text-to-speech transformation [41, 43], information retrieval [44], 3”