Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training
By Faisal Mahmood, Richard J. Chen, Nicholas J. Durr. 2017
266 citations recorded by the source corpus.
To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
Citations in the corpus
- Boundary-weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation (2019, background): “[39] introduced a novel unsupervised reverse domain adaptation framework for addressing the issue of cross-patient network adaptability and limited availability of annotated medical images.”
- Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks (2018, background): “Thus, it is quite common in deep learning research to perform one-fold cross-validation [30], [35] or even directly adopt hyperparameter selection from published work [24], [28], [29], [38], [48], [62].”
- Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk (2017, background): “These more sophisticated priors are recently emerging in empirical applications of many fields of imaging, such as medical imaging [70, 30, 24, 55, 17, 1, 49, 68, 53, 43, 32, 42, 16, 69], microscopy [59], inpainting [45, 71], superresolution [60, 33, 28], compressed sensing [38, 37, 51, 50], image m”
- A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “ata points) [100], in situations with different label sets for each domain [223], for object detection [39, 97, 106], for person re-identification [11,57,73,142,251,287,288], and for depth estimation [8,155,174]. 7.2 Natural Language Processing Domain adaptation has been used in natural language pro”
- Domain Adaptation for Medical Image Analysis: A Survey (2021, unknown): “Mahmood et al. [126] propose a Reverse Domain Adaptation method based on GAN for endoscopy image analysis.”
- Transformation-Consistent Self-Ensembling Model for Semisupervised Medical Image Segmentation (2019, background): “The lack of the labeled data motivates the study of methods that can be trained with limited supervision, such as semisupervised learning [4]–[6], weakly supervised learning [7]–[9], and unsupervised domain adaptation [10]–[12].”
- Synthetic Data for Deep Learning (2019, background): “[387] show an interesting take on the problem by doing the reverse: they make real medical images look more like synthetic images in order to then apply a network trained on synthetic data (see also Section 6.”
- Unsupervised domain adaptation for medical imaging segmentation with self-ensembling (2018, unknown): “A few studies have applied domain adaptation to medical imaging directly by using adversarial training (Kamnitsas et al., 2017; Chen et al., 2018; Zhang et al., 2018; Lafarge et al., 2017; Javanmardi and Tasdizen, 2018; Dou et al., 2018), with some studies using generative models to augment…”
- Target-Independent Domain Adaptation for WBC Classification Using Generative Latent Search (2020, background): “The principles of adversarial feature learning has been used in [23], [24] to transform real images to a synthetic-like representation using unlabeled synthetic endoscopy images and achieve stain independence.”
- Secure and Robust Machine Learning for Healthcare: A Survey (2020, methodology): “An unsupervised domain adaptation method using reverse flow and adversarial training for generating synthetic medical images is presented in [197].”
- Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images (2018, methodology): “Joint CNN-CRF models have also been explored for more global and context aware CNN training [44, 67, 68].”