Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss

By Qi-Yan Dou, Cheng Ouyang, Cheng Chen, Hao Chen, P. Heng. 2018

329 citations recorded by the source corpus.

Convolutional networks (ConvNets) have achieved great successes in various challenging vision tasks. However, the performance of ConvNets would degrade when encountering the domain shift. The domain adaptation is more significant while challenging in the field of biomedical image analysis, where cross-modality data have largely different distributions. Given that annotating the medical data is especially expensive, the supervised transfer learning approaches are not quite optimal. In this paper, we propose an unsupervised domain adaptation framework with adversarial learning for cross-modality biomedical image segmentations. Specifically, our model is based on a dilated fully convolutional network for pixel-wise prediction. Moreover, we build a plug-and-play domain adaptation module (DAM) to map the target input to features which are aligned with source domain feature space. A domain critic module (DCM) is set up for discriminating the feature space of both domains. We optimize the DAM and DCM via an adversarial loss without using any target domain label. Our proposed method is validated by adapting a ConvNet trained with MRI images to unpaired CT data for cardiac structures segmentations, and achieved very promising results.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. P. Heng
2. Qi-Yan Dou
3. Jing Qin
4. Hao Chen
5. Lequan Yu
6. B. van Ginneken
7. Bo Du
8. Pingkun Yan

In Radiology, Nuclear Medicine and Imaging

1. P. Heng
2. Hao Chen
3. Lequan Yu
4. Qi-Yan Dou
5. Jing Qin
6. Daniel Rueckert
7. Cheng Chen
8. Ronald M. Summers

Citations in the corpus

• Generative Adversarial Network in Medical Imaging: A Review (2018, background): “Ad-versarial training, or more specifically unpaired cross modality translation, enables reuse of the labels in all modalities and opens new ways for unsupervised transfer learning (Dou et al., 2018; Ying et al., 2019).”
• Cancer Diagnosis Using Deep Learning: A Bibliographic Review (2019, methodology): “GANs have been used for various applications including reconstruction [114–116], segmentation [117–119], domain adaptation [120,121] and detection [122,123].”
• Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation (2020, methodology): “situation due to the significant domain shift caused by the different physical principles of modalities [7], [36], [37], [42].”
• Boundary-weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation (2019, background): “Similar to the referenced works in [26], [40], and [50], to make D obtain more useful information and enhance the accuracy of”
• RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment (2019, background): “DANN [6] and Seg-CT-UDA [5] minimizes adversarial discriminator accuracy to reduce the distribution divergence between source and target features.”
• MS-Net: Multi-Site Network for Improving Prostate Segmentation With Heterogeneous MRI Data (2020, methodology): “Most previous works focus on unsupervised domain adaptation [49], [54], aiming to adapt the model trained on source site into a target site.”
• Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation (2019, methodology): “Most methods within this stream discriminate feature distributions of source/target domains in an adversarial learning scenario (Ganin et al. 2016; Tzeng et al. 2017; Dou et al. 2018).”
• Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classification (2019, unknown): “We see a spike at low intensities (especially 0 ) for every center, but the distribution for higher intensities is somewhat different for every center, which might implicate in a decrease of the models predictive performance.”
• Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization (2020, background): “The general idea of domain adaption is to align the source domain and target domain distributions in the input level [3,19], semantic feature level [9,34], or output space level [5,39,40,43,45].”
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation (2020, background): “In the field of medical image analysis, [13, 29, 45] have investigated this topic with existing approaches, i.”
• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, background): “In [59], a domain adaptation module that maps the target input to features that are aligned with source domain feature space is proposed for cross-modality biomedical image segmentation, using a domain critic module for discriminating the feature space of both domains.”
• Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images (2018, background): “proposed to generate synthetic retanopathy data from retinal masks [35].”
• Unsupervised domain adaptation for medical imaging segmentation with self-ensembling (2018, background): “…medical imaging directly by using adversarial training (Kamnitsas et al., 2017; Chen et al., 2018; Zhang et al., 2018; Lafarge et al., 2017; Javanmardi and Tasdizen, 2018; Dou et al., 2018), with some studies using generative models to augment training (Mahmood et al., 2018; Madani et al., 2018).”