Domain Agnostic Learning with Disentangled Representations

By Xingchao Peng, Zijun Huang, Ximeng Sun, Kate Saenko. 2019

304 citations recorded by the source corpus.

Unsupervised model transfer has the potential to greatly improve the generalizability of deep models to novel domains. Yet the current literature assumes that the separation of target data into distinct domains is known as a priori. In this paper, we propose the task of Domain-Agnostic Learning (DAL): How to transfer knowledge from a labeled source domain to unlabeled data from arbitrary target domains? To tackle this problem, we devise a novel Deep Adversarial Disentangled Autoencoder (DADA) capable of disentangling domain-specific features from class identity. We demonstrate experimentally that when the target domain labels are unknown, DADA leads to state-of-the-art performance on several image classification datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiashi Feng
2. Dacheng Tao
3. Kate Saenko
4. Trevor Darrell
5. Mingsheng Long
6. Jianmin Wang
7. Ziwei Liu
8. Jian Liang

Citations in the corpus

• Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, background): “The analysis of the latent space organization is becoming crucial towards understanding and improvement of classification models [68, 49].”
• A Comprehensive Survey on Transfer Learning (2019, background): “A few studies focus on the setting that m ≥ 2 [21].”
• Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay (2020, background): “…performance, where different types of latent variables are proposed to better capture the variations in the dataset (e.g., domainrelevant and class-relevant information), and the reconstruction is either on the image level (Ilse et al., 2019; Cai et al., 2019) or feature level (Peng et al., 2019).”
• Target-Independent Domain Adaptation for WBC Classification Using Generative Latent Search (2020, methodology): “Domain Agnostic Learning (DAL) [22] uses Deep Adversarial Disentangled Auto-Encoders (DADA) to disentangle domain-invariant features in the latent space by minimizing the mutual information between domain-invariant and domain-specific features.”
• Disentangled Speaker and Language Representations Using Mutual Information Minimization and Domain Adaptation for Cross-Lingual TTS (2021, methodology): “The MI is notoriously hard to compute exactly since it needs to compute these distributions at the same time, thus recent works usually optimize the MI by approximating it [17, 18, 16].”
• Heterogeneous Graph Attention Network for Unsupervised Multiple-Target Domain Adaptation (2020, background): “disentangled representations [25] are utilized to transfer knowledge from labeled source domain to unlabeled data, which can remove class-irrelevant features to boost domain adaptation performance.”
• Learning to Balance Specificity and Invariance for In and Out of Domain Generalization (2020, background): “In the context of domain adaptation, disentangling features into domain-specific and domain-independent factors have been proposed in [7,39].”
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation (2020, methodology): “The BWT in Crt.+Src. is higher than other existing memory-free method, i.e., DANN, MCD and DADA, which indicates contrastive loss has the some ability to overcome catastrophic forgetting.”
• Federated Adversarial Domain Adaptation (2019, unknown): “Following Peng et al. (2019), we adopt the Mutual Information Neural Estimator (MINE) (Belghazi et al., 2018) to estimate the mutual information by leveraging a neural network T θ : ) .”
• Minimum Class Confusion for Versatile Domain Adaptation (2019, methodology): “Recently, several highly practical scenarios were proposed, such as Partial DA (PDA) [2,51] with the source label set subsuming the target one, Multi-Source DA (MSDA) [54,48] with multiple source domains, and Multi-Target DA (MTDA) [32] with multiple target domains.”
• Open Compound Domain Adaptation (2019, methodology): “Table 6 summarizes the key methodological differences between MTDA [9], BTDA [5], DADA [39] and our OCDA.”
• Domain2Vec: Domain Embedding for Unsupervised Domain Adaptation (2020, methodology): “Feature Reconstruction Previous literature [15] has shown that the deep information could be missing in the feature disentangle process, especially when the feature disentangler D is composed of several fully connected and Relu layers and it cannot guarantee the information integrity in the feature…”
• Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (2021, background): “major challenge in artificial intelligence, and can be beneficial to many computer vision tasks, such as domain adaptation [33, 45], controllable image generation [32, 38, 41, 48], and image manipulation [37].”
• Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, background): “[125] disentangled the fine-grained domain information and category information that are learned in VAEs.”
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020, methodology): “Later, the original one-to-one adaptation is extended to the multi-source (Peng et al., 2019a) and the multi-target (Peng et al., 2019b) setting, respectively.”
• Single-Source Domain Expansion Network for Cross-Scene Hyperspectral Image Classification (2022, background): “Peng et al. designed a Deep Adversarial Decoupled Autoencoder (DADA) to decouple class-specific features from class identity [24].”