Deep MR to CT Synthesis Using Unpaired Data

By Jelmer M. Wolterink, Anna M. Dinkla, Mark H. F. Savenije, Peter R. Seevinck, Cornelis A. T. van den Berg, Ivana Isgum. 2017

681 citations recorded by the source corpus.

MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

An overview of deep learning in medical imaging focusing on MRI (2018, methodology): “Other applications are unsupervised synthesis of T1weighted brain MRI using a GAN [179]; image synthesis with context-aware GANs [216]; synthesis of patientspecific transmission image for PET attenuation correction in PET/MR imaging of the brain using a CNN [217]; pseudo-CT synthesis for pelvis PET/”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “cal fields such as for generating and designing DNA [296], [297], drug discovery [298], generating multi-label discrete patient records [299], medical image processing [300], [301], [302], [303], [304], [305], [306], [307], and doctor recommen-”
Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data size (2018, methodology): “While CycleGAN has already applied to MR-to-CT synthesis [6], all these previous approaches in medical image application targeted CT and MRI of the head in which the scan protocol (i.”
Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation (2018, methodology): “Cross-domain adaptation of highly different modalities, has been applied in medical image analysis for image synthesis using paired images [6] and unpaired images [7], as well as for segmentation [8, 9].”
MedGAN: Medical Image Translation using GANs (2018, methodology): “A similar but unsupervised approach was used in [47] via CycleGANs.”
Unsupervised Medical Image Translation With Adversarial Diffusion Models (2022, background): “For improved flexibility, other studies proposed cycle-consistency or mutual information losses to enable unsupervised learning from unpaired data [21], [33], [39], [40], [41], [42], [43], [44].”
Domain Adaptation for Medical Image Analysis: A Survey (2021, neutral): “Image-to-image translation via cycle consistency loss (Cycle-GAN) [113], [114].”
GANs for Medical Image Analysis (2018, background): “While [29] requires corresponding pairs of CT and MR images for training, [4] successfully utilizes CycleGANs to transform 2D MR images to CT images without the need for explicit, co-registered data pairs.”
Deformable Image Registration Using Cue-aware Deep Regression Network (2018, methodology): “Deep learning techniques, such as convolutional neural network (CNN), have been widely applied in medical image analysis [1] due to its strong learning ability, such as disease diagnosis [2]–[4], image segmentation [5], [6], and image synthesis [7].”
An Asymetric Cycle-Consistency Loss for Dealing with Many-to-One Mappings in Image Translation: A Study on Thigh MR Scans (2020, background): “Over the last three years, numerous similar approaches were effectively applied in the field of medical image analysis [3,4,5].”
Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks (2018, methodology): “one-fold cross-validation [30], [35] or even directly adopt hyperparameter selection from published work [24], [28],”
Synthetic Data for Deep Learning (2019, methodology): “Similar image-to-image translation techniques have been applied to generating images from 2D MR brain images to CT and back [282, 424, 639], PET to CT [27], cardiac CT to MR [96], virtual”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, methodology): “GANs and their variants have been used in neuroimaging for cross-modality synthesis [146], motion artifact reduction [147], resolution upscaling [148]–[150], estimating full-dosage PET images from low-dosage PET [151]–[153], image harmonization [14], [154], heterogeneity analysis [155], and more.”
A Review of Deep-Learning-Based Medical Image Segmentation Methods (2021, methodology): “[65] used CycleGAN to convert 2D MR images into CT images.”