Return of Frustratingly Easy Domain Adaptation

By Baochen Sun, Jiashi Feng, Kate Saenko. 2015

2,086 citations recorded by the source corpus.

Unlike human learning, machine learning often fails to handle changes between training (source) and test (target) input distributions. Such domain shifts, common in practical scenarios, severely damage the performance of conventional machine learning methods. Supervised domain adaptation methods have been proposed for the case when the target data have labels, including some that perform very well despite being ``frustratingly easy'' to implement. However, in practice, the target domain is often unlabeled, requiring unsupervised adaptation. We propose a simple, effective, and efficient method for unsupervised domain adaptation called CORrelation ALignment (CORAL). CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any target labels. Even though it is extraordinarily simple--it can be implemented in four lines of Matlab code--CORAL performs remarkably well in extensive evaluations on standard benchmark datasets.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Refining Language Models with Compositional Explanations (2021, background): “In another relevant thread, unsupervised domain adaptation (UDA) looks to adapt a trained model to a new domain using unlabeled data from the target domain, by updating feature representation to minimize the distribution divergence between domains [12, 11], or updating models to match the distributi”
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization (2017, background): “It is possible that replacing AdaIN with correlation alignment [49] or histogram matching [54] could further improve quality by transferring higher-order statistics.”
Adversarial Discriminative Domain Adaptation (2017, positive): “This is generally achieved by optimizing the representation to minimize some measure of domain shift such as maximum mean discrepancy [5, 6] or correlation distances [7, 8].”
Learning to cluster in order to Transfer across domains and tasks (2017, background): “…of work dealing with domain shift between image datasets by minimizing domain discrepancy (Tzeng et al., 2014; Long et al., 2015b; Ganin et al., 2016; Sun et al., 2016; Long et al., 2016; Sener et al., 2016; Carlucci et al., 2017; Long et al., 2017; Zellinger et al., 2017; Bousmalis et al., 2017).”
Moment Matching for Multi-Source Domain Adaptation (2018, methodology): “Other commonly used methods include correlation alignment [39, 32], Kullback-Leibler (KL) divergence [51], and H divergence [1].”
Domain Separation Networks (2016, methodology): “We show in Table 4 that our new domain adaptation method, which is theoretically as powerful as an MMD loss with a second–order polynomial kernel, outperforms CORAL in all our datasets.”
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping (2018, background): “l-level adaptation. Feature-level domain adaptation focuses on learning domain-invariant features, either by learning a transformation of ﬁxed, pre-computed features between source and target domains [18], [19], [20], [21] or by learning a domain-invariant feature extractor, often represented by a c”
A Comprehensive Survey on Transfer Learning (2019, methodology): “[105] proposed an approach termed co-relation alignment (CORAL).”
Deep Hashing Network for Unsupervised Domain Adaptation (2017, methodology): “The features are fixed and then a model is trained to align the source and target domains [16, 20, 33, 38, 42, 43, 44].”
Deep CORAL: Correlation Alignment for Deep Domain Adaptation (2016, methodology): “A recently proposed CORAL method [1] aligns the second-order statistics of the source and target distributions with a linear transformation.”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “ssiﬁcation [9, 10, 11], sentiment analysis [12, 13], WiFi-based localization [14], speech recognition across different speakers [15, 16], object recognition in images acquired in different conditions [17, 18, 19, 20, 21], video concept detection [22], video event recognition [23], activity recogniti”
Gibson Env: Real-World Perception for Embodied Agents (2018, background): “Domain Adaptation and Transferring to Real-World: With popularity of simulators, different approaches for domain adaption for transferring the results to real world has been investigated [11, 25, 85, 71, 89, 93], e.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “For example, methods that learn features that have similar distributions across domains are equally applicable to both settings (Ben-David et al., 2006; Long et al., 2015; Sun et al., 2016; Ganin et al., 2016; Tzeng et al., 2017; Shen et al., 2018; Wu et al., 2019).”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, methodology): “Prior methods can be split into: (1) feature-level adaptation, where domain-invariant features are learned between source and target domains [17, 15, 57, 7, 14, 36, 5, 55], or (2) pixellevel adaptation, which focuses on re-stylizing images from the source domain to make them look like images from th”
SpotTune: Transfer Learning Through Adaptive Fine-Tuning (2018, methodology): “hods in computer vision [9,37]. Early approaches have concentrated on shallow classiﬁers, using techniques such as instance re-weighting [50,12], model adaptation [11,52], and feature space alignment [44,35,18]. In the multi-task setting, knowing which tasks or parameters are shareable is a longstan”
Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning (2017, methodology): “Using this ”fully-transductive” protocol, we compare our method with other state-of-the-art approaches including DLID (Chopra et al., 2013), DDC (Tzeng et al., 2014), DAN (Long et al., 2015), Deep CORAL (Sun & Saenko, 2016), and DANN (Ganin et al., 2016), based on fine-tuning of the baseline model A”
Domain Adaptive Faster R-CNN for Object Detection in the Wild (2018, methodology): “Conventional methods include domain transfer multiple kernel learning [10, 11], asymmetric metric learning [33], subspace interpolation [23], geodesic flow kernel [22], subspace alignment [14], covariance matrix alignment [52, 57], etc.”