A Survey of Unsupervised Deep Domain Adaptation

By Garrett Wilson, Diane Joyce Cook. 2018

1,027 citations recorded by the source corpus.

Deep learning has produced state-of-the-art results for a variety of tasks. While such approaches for supervised learning have performed well, they assume that training and testing data are drawn from the same distribution, which may not always be the case. As a complement to this challenge, single-source unsupervised domain adaptation can handle situations where a network is trained on labeled data from a source domain and unlabeled data from a related but different target domain with the goal of performing well at test-time on the target domain. Many single-source and typically homogeneous unsupervised deep domain adaptation approaches have thus been developed, combining the powerful, hierarchical representations from deep learning with domain adaptation to reduce reliance on potentially costly target data labels. This survey will compare these approaches by examining alternative methods, the unique and common elements, results, and theoretical insights. We follow this with a look at application areas and open research directions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning (2021, background): “GAN-based DA approaches require having access to the large amount of unlabeled data from both the source and target datasets (Tzeng et al., 2017; Zhang et al., 2019; Wilson & Cook, 2020).”
Domain Adaptation for Medical Image Analysis: A Survey (2021, neutral): “In terms of label availability, existing DA methods can be divided into supervised DA, semi-supervised DA, and unsupervised DA [11], [13].”
Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift (2020, background): “Some of these methods, such as importance weighting approaches [60], only handle a single predeﬁned shift and do not constitute test time adaptation [10, 68].”
MEMO: Test Time Robustness via Adaptation and Augmentation (2021, background): “The general problem of distribution shift has been studied under a number of frameworks [29], including domain adaptation [33, 4, 37], domain generalization [3, 26, 7], and distributionally robust optimization [2, 14, 31], to name just a few.”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “Other surveys Comprehensive reviews on DA exist, each with a different focus: visual applications (Csurka, 2017; Patel et al., 2015; Wilson and Cook, 2020), machine translation (MT) (Chu and Wang, 2018), pre-neural DA methods in NLP (Jiang, 2008; Margolis, 2011).”
Unsupervised Domain Adaptation of Object Detectors: A Survey (2021, background): “This problem where models, trained on one particular dataset (also known as source dataset), do not generalize well to a dataset that has a different distribution (also known as target dataset) is commonly referred to as domain shift or distribution shift in the literature [21], [22], [23], [24].”
Federated learning for medical image analysis: A survey (2023, background): “Domain adaptation (Guan and Liu, 2022; Wilson and Cook, 2020; Kouw and Loog, 2019) is a sound machine learning technique that aims to reduce domain shift among different datasets and enhance the generalizability of a learning model.”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, background): “…mapping (Csurka, 2017; Kouw, 2018; Kouw and Loog, 2019), normalization statistic-based (Csurka, 2017; Kouw and Loog, 2019; Zhao et al., 2018, 2020b), ensemble-based (Csurka, 2017; Wang and Deng, 2018; Kouw and Loog, 2019; Zhao et al., 2018), or self-training-based methods (Wilson and Cook, 2020).”
Transfer learning for medical image classification: a literature review (2022, background): “Wilson and Cook [9] surveyed a large number of articles of unsupervised deep domain adaptation.”
An Overview of Deep Semi-Supervised Learning (2020, background): “One popular type of transfer learning is domain adaptation [120, 116, 164].”
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation (2021, methodology): “To handle this issue and avoid the expensive laborious annotations, lots of research efforts (Bousmalis et al., 2017; Kuroki et al., 2019; Wilson & Cook, 2020; VS et al., 2021) are devoted on Unsupervised Domain Adaptation (UDA).”
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift (2020, methodology): “Wilson and Cook [2018] survey different methods in unsupervised domain adaptation.”
Secure and Robust Machine Learning for Healthcare: A Survey (2020, methodology): “for each method are mostly used in the literature [196].”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “We briefly introduce a few popular techniques and refer the reader to the existing literature on DA ( e.g. , [6, 17]) for further information.”