Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

By Cheng Chen, Qi-Yan Dou, Hao Chen, Jing Qin, P. Heng. 2020

398 citations recorded by the source corpus.

Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional cross-modality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuanda Zhu
2. Li Tong
3. Anirudh Choudhary
4. May D. Wang

In Computer Vision and Pattern Recognition

1. Yuanda Zhu
2. Li Tong
3. Anirudh Choudhary
4. May D. Wang
5. Zaiyi Liu

In Radiology, Nuclear Medicine and Imaging

1. Yuanda Zhu
2. Li Tong
3. Anirudh Choudhary
4. May D. Wang

Citations in the corpus

• Domain Adaptation for Medical Image Analysis: A Survey (2021, neutral): “For a machine learning model, domain shift [4], [26]– [28] refers to the change of data distribution between its training dataset (source/reference domain) and test dataset (target domain).”
• Advancing Medical Imaging Informatics by Deep Learning-Based Domain Adaptation (2020, methodology): “Chen et al ., [59] proposed state-of-the-art unsupervised segmentation method using bidirectional DA-DT between MRI and CT, combining Cy - cleGAN with shared feature encoder layers between domains.”
• Annotation-efficient deep learning for automatic medical image segmentation (2020, background): “Methods such as data augmentation 41,46 , transfer learning 42,43 , semi-supervised learning 44,47 , and self-supervised learning 48,49 have been extensively investigated to handle cases with limited training annotations or no target domain annotations.”
• Artificial intelligence and machine learning for medical imaging: A technology review. (2021, neutral): “…useful applications for medical imaging exist, such as domain adaptation (e.g., adapting a segmentation model trained on an image modality to work on a different image modality) [35 – 37], data generation (e.g., generate artificial realistic images) [38 – 40] or even image segmentation [41].”