Domain-Adversarial Neural Networks

By Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand. 2014

351 citations recorded by the source corpus.

We introduce a new neural network learning algorithm suited to the context of domain adaptation, in which data at training and test time come from similar but different distributions. Our algorithm is inspired by theo ry on domain adaptation suggesting that, for effective domain transfer to be achiev ed, predictions must be made based on a data representation that cannot discriminate between the training (source) and test (target) domains. We propose a training objective that implements this idea in the context of a neural network, whose hidden layer is trained to be predictive of the classification target, but uninformati ve as to the domain of the input. Our experiments on a sentiment analysis classificati on benchmark, where the target data available at the training time is unlabeled, show that our neural network for domain adaption algorithm has better performance than either a standard neural networks and a SVM, trained on input features extracted with the state-ofthe-art marginalized stacked denoising autoencoders of Chen et al. (2012).

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Concrete Problems in AI Safety (2016, background): “Robot locomotion or industrial control tasks might be a natural candidate for such experiments.”
Domain-Adversarial Training of Neural Networks (2015, background): “This version extends Ganin and Lempitsky (2015) very considerably by incorporating the report Ajakan et al. (2014) (presented as part of the Second Workshop on Transfer and Multi-Task Learning ), which brings in new terminology, in-depth theoretical analysis and justiﬁcation of the approach,…”
Professor Forcing: A New Algorithm for Training Recurrent Networks (2016, methodology): “We evaluate Professor Forcing on character-level language modeling on Penn-Treebank corpus, which has an alphabet size of 50 and consists of 5059k characters for training, 396k characters for validation and 446k characters for test.”
Sim-to-Real Robot Learning from Pixels with Progressive Nets (2016, background): “Several recent works from the supervised learning literature, e.g. [4, 1, 3], demonstrate how ideas from the adversarial training of neural networks can be used to reduce the sensitivity of a trained network to inter-domain variations, without requiring aligned training data.”
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (2016, methodology): “[3] introduced the Domain–Adversarial Neural Network (DANN): an architecture trained to extract domain-invariant features.”
Domain Separation Networks (2016, background): “[2] use adversarial training to find domain–invariant representations in-network.”
Generative Multi-Adversarial Networks (2016, background): “…framework is one of the more recent successes in a line of research on adversarial training in machine learning (Schmidhuber (1992); Bagnell (2005); Ajakan et al. (2014)) where games between learners are carefully crafted so that Nash equilibria coincide with some set of desired optimality…”
Self-Supervised Learning: Generative or Contrastive (2020, background): “[1], [2], [37], [105] studies how GAN can help on domain adaptation.”
A Survey on Deep Transfer Learning (2018, methodology): “[1], [5], [21], [22], [13], [16] 3 TrAdaBoost proposed by [4] use AdaBoost-based technology to ﬁlter out instances that are dissimilar to the target domain in source domains.”
Adversarial Multi-task Learning for Text Classification (2017, background): “Formally, (Ajakan et al., 2014) linked the adversarial loss to the H-divergence between two distributions and successfully achieve unsupervised domain adaptation with adversarial network.”
Invariant Risk Minimization Games (2020, background): “Domain adaptation tries to ﬁnd a representation Φ whose distribution is invariant across source and target domains [18, 19, 20, 21].”
Adversarial Domain Adaptation with Domain Mixup (2019, background): “butions. In addition, speciﬁc deep neural networks are constructed to restrict the domain-invariance of top layers by aligning the second-order statistics (Sun and Saenko 2016). Adversarial Training (Ajakan et al. 2013; Ganin et al. 2016) is another way to transfer domain information. RevGrad (Ganin”
Wasserstein Distance Guided Representation Learning for Domain Adaptation (2017, methodology): “…representation contains no discriminative information about the origin (i.e. domain) of the input, domain adversarial neural network (DANN) (Ajakan et al. 2014; Ganin et al. 2016) was proposed to learn domain invariant features by a minimax game between the domain classifier and the…”
How Generative Adversarial Networks and Their Variants Work (2017, methodology): “…be noted that the method of [12] can check whether the domain adaptation process is working well during the training phase because the transformed data is visible, while [2, 108] cannot visually check the domain adaptation process because the common representation space cannot be easily visualized.”
Adapting Object Detectors via Selective Cross-Domain Alignment (2019, methodology): “A variety of methods have been developed along this line, which have shown encouraging results on image clas-siﬁcation [1, 12, 13, 16, 28, 43] and semantic segmentation [20, 39, 42, 46, 49].”
Reliable Weighted Optimal Transport for Unsupervised Domain Adaptation (2020, methodology): “Based on this, to maximally confuse the domain classiﬁer, Domain Adversarial Neural Network (DANN) [1] is proposed with a feature extractor.”