An introduction to domain adaptation and transfer learning

By Wouter M. Kouw. arXiv.org, 2018

346 citations recorded by the source corpus.

In machine learning, if the training data is an unbiased sample of an underlying distribution, then the learned classification function will make accurate predictions for new samples. However, if the training data is not an unbiased sample, then there will be differences between how the training data is distributed and how the test data is distributed. Standard classifiers cannot cope with changes in data distributions between training and test phases, and will not perform well. Domain adaptation and transfer learning are sub-fields within machine learning that are concerned with accounting for these types of changes. Here, we present an introduction to these fields, guided by the question: when and how can a classifier generalize from a source to a target domain? We will start with a brief introduction into risk minimization, and how transfer learning and domain adaptation expand upon this framework. Following that, we discuss three special cases of data set shift, namely prior, covariate and concept shift. For more complex domain shifts, there are a wide variety of approaches. These are categorized into: importance-weighting, subspace mapping, domain-invariant spaces, feature augmentation, minimax estimators and robust algorithms. A number of points will arise, which we will discuss in the last section. We conclude with the remark that many open questions will have to be addressed before transfer learners and domain-adaptive classifiers become practical.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/c98bcc8689c34d72fd0b696f2a49e7f86d151782

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, background): “In addition, some reviews do not touch deeply on domain mapping (Csurka, 2017; Kouw, 2018; Kouw and Loog, 2019), normalization statistic-based (Csurka, 2017; Kouw and Loog, 2019; Zhao et al.”
Lifelong Teacher-Student Network Learning (2021, background): “The challenge in this case is to overcome the differences between the domains in order to ensure a good generalization, [12].”
A Survey of Unsupervised Deep Domain Adaptation (2018, background): “In some cases, prior surveys do not discuss domain mapping [48, 49, 121], normalization statistic-based [48, 49, 121, 285], or ensemble-based [48, 49, 121, 246, 285] methods.”
Two standard-free approaches to correct for external influences on near-infrared spectra to make models widely applicable (2020, methodology): “To deal with such problems, a family of methods called Domain Adaption (DA) or Transfer Learning has recently emerged (Kouw and Loog, 2018).”
Speech Translation and the End-to-End Promise: Taking Stock of Where We Are (2020, background): “Note that our definition does not entail covariance shift and other forms of domain mismatch (Kouw and Loog, 2018) which, though relevant, are not unique to cascaded ST and are widely covered by general ASR and MT literature (Cuong and Sima’an, 2018).”
Applications of Generative Adversarial Networks (GANs): An Updated Review (2019, background): “21 Example—domain adaptation [57] y, similar to conditional VAE [127].”