Deep Domain Confusion: Maximizing for Domain Invariance

By Eric Tzeng, Judy Hoffman, N. Zhang, Kate Saenko, Trevor Darrell. 2014

2,889 citations recorded by the source corpus.

Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias on a standard benchmark. Fine-tuning deep models in a new domain can require a significant amount of data, which for many applications is simply not available. We propose a new CNN architecture which introduces an adaptation layer and an additional domain confusion loss, to learn a representation that is both semantically meaningful and domain invariant. We additionally show that a domain confusion metric can be used for model selection to determine the dimension of an adaptation layer and the best position for the layer in the CNN architecture. Our proposed adaptation method offers empirical performance which exceeds previously published results on a standard benchmark visual domain adaptation task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Domain randomization for transferring deep neural networks from simulation to the real world (2017, background): “, [50]), and learning a mapping from the target domain to the source domain (e.”
One-Shot Imitation Learning (2017, background): “Success stories include domain adaptation in computer vision [64, 34, 28, 4, 15, 24, 33, 59, 14] and control [60, 45, 46, 20, 54].”
Coupled Generative Adversarial Networks (2016, methodology): “Method [17] [18] [19] [20] CoGAN From MNIST 0.”
Domain-Adversarial Training of Neural Networks (2015, methodology): “We make our approach more comparable with Tzeng et al. (2014) by using exactly the same network architecture replacing domain mean-based regularization with the domain classiﬁer.”
Adversarial Discriminative Domain Adaptation (2017, methodology): “The DDC method [5] used MMD in addition to the regular clas-siﬁcation loss on the source to learn a representation that is both discriminative and domain invariant.”
Learning Transferable Features with Deep Adaptation Networks (2015, methodology): “We evaluate our method across the 3 transfer tasks,A → W, D → W andW → D, which are commonly adopted in deep learning methods (Donahue et al., 2014; Tzeng et al., 2014).”
The Variational Fair Autoencoder (2015, methodology): “th “domain-regularized” supervised approaches that simultaneously try to predict the label for a data point and remove domain speciﬁc information. This is done with either MMD (Long & Wang, 2015; Tzeng et al., 2014) or adversarial (Ganin et al., 2015) penalties at the hidden layers of the networ”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “The domain confusion loss , as proposed by Tzeng et al. (2014), is another solution that learns a semantically meaningful and domain invariant representation.”
Learning to cluster in order to Transfer across domains and tasks (2017, background): “Cross-domain Transfer Learning: Also known as domain adaptation (Pan & Yang, 2010), there has recently been a large body of work dealing with domain shift between image datasets by minimizing domain discrepancy (Tzeng et al., 2014; Long et al., 2015b; Ganin et al., 2016; Sun et al., 2016; Long et al”
Few-Shot Adversarial Domain Adaptation (2017, neutral): “LB [60] [34] [15] [58] [27] [38] FADA A →W 61.”
Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation (2022, methodology): “C→I Step 1 [0, 2, 7, 9, 11, 27, 28, 29, 30, 33] Step 2 [37, 39, 40, 44, 45, 47, 50, 60, 62, 68] Step 3 [71, 75, 76, 82, 85, 86, 87, 88, 89, 90] Step 4 [92, 94, 96, 97, 106, 107, 108, 109, 110, 112] Step 5 [114, 115, 116, 123, 126, 128, 133, 134, 141, 145] Step 6 [146, 150, 151, 157, 160, 163, 165, 1”
Cross Modal Distillation for Supervision Transfer (2015, background): “. More recently, unsupervised domain adaptation techniques have been introduced which learn to adapt deep representations so as to minimize the discrepancy between the source and target distributions [50,10,37]. All these lines of work study and solve the problem of domain adaptation within the same”
The limits and potentials of deep learning for robotics (2018, background): “In [70], a ‘confusion’ loss is proposed which forces the model to ignore variations in the data that separate the two domains [122], [123], and [121] attempts to address the simulation to reality gap by using aligned data.”
Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment (2019, methodology): “Concerning algorithms, several prior works demonstrate empirical success of domain-adversarial approaches, (Tzeng et al., 2014; Ganin et al., 2016; Bousmalis et al., 2016; Tzeng et al.; Hoﬀman et al., 2017; Shu et al., 2018).”
Moment Matching for Multi-Source Domain Adaptation (2018, methodology): “For example, several UDA methods [27, 41, 25] incorporate Maximum Mean Discrepancy loss into a neural network to diminish the domain discrepancy; other models introduce different learning schema to align the source and target domains, including aligning second order correlation [39, 32], sk et ch re”
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (2016, background): “While much prior work exists, our literature review focuses primarily on Convolutional Neural Network (CNN) methods due to their empirical superiority on the problem [14, 31, 41, 44].”