Domain Generalization via Conditional Invariant Representations

By Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, Dacheng Tao. 2018

305 citations recorded by the source corpus.

Domain generalization aims to apply knowledge gained from multiple labeled source domains to unseen target domains. The main difficulty comes from the dataset bias: training data and test data have different distributions, and the training set contains heterogeneous samples from different distributions. Let X denote the features, and Y be the class labels. Existing domain generalization methods address the dataset bias problem by learning a domain-invariant representation h(X) that has the same marginal distribution P(h(X)) across multiple source domains. The functional relationship encoded in P(Y|X) is usually assumed to be stable across domains such that P(Y|h(X)) is also invariant. However, it is unclear whether this assumption holds in practical problems. In this paper, we consider the general situation where both P(X) and P(Y|X) can change across all domains. We propose to learn a feature representation which has domain-invariant class conditional distributions P(h(X)|Y). With the conditional invariant representation, the invariance of the joint distribution P(h(X),Y) can be guaranteed if the class prior P(Y) does not change across training and test domains. Extensive experiments on both synthetic and real data demonstrate the effectiveness of the proposed method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Toward Causal Representation Learning (2021, background): “It also led to a range of follow-up work [32], [78], [97], [114], [115], [152], [153], [156], [167], [195], [204], [243], [263], [267], [277], [278], [281], complementing the studies of Bareinboim and Pearl [14], [185], and it inspired a thread of work in the statistics community exploiting…”
Causality for Machine Learning (2019, neutral): “…Blöbaum et al., 2016; Guo et al., 2018; Wang et al., 2019; Subbaswamy et al., 2018; Rabanser et al., 2018; Li et al., 2018a; Lipton et al., 2018; Li et al., 2018b; Magliacane et al., 2018; Rojas-Carulla et al., 2018), and nicely complementary to studies of Elias Bareinboim and Judea (Bareinboim…”
In Search of Lost Domain Generalization (2020, background): “– MMD [Li et al., 2018b] matches the MMD [Gretton et al., 2012] of feature distributions.”
Domain Agnostic Learning with Disentangled Representations (2019, background): “Recently, domain generalization approaches (Li et al., 2018a; Carlucci et al., 2018b; Li et al., 2018b) have been introduced to adapt from multiple labeled source domains to an unseen target domain.”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, background): “For the explicit distribution matching, more and more works tend to match the joint distributions rather than just match the marginal [6], [72] or conditional [63] distributions.”
A Survey of Domain Adaptation for Neural Machine Translation (2018, background): “Domain generalization aims to apply knowledge gained from labeled source domains to unseen target domains (Li et al., 2018).”
Domain Generalization using Causal Matching (2020, methodology): “(Li et al., 2018c; Ghifary et al., 2016; Li et al., 2018d) Even in the ideal case where we observe Y true , d-separation on the SCM reveals that X C (cid:54)⊥⊥ D | Y true due to a path through O .”
Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization (2021, background): “…[56, 58, 53], Meta-Learning for Domain Generalization (MLDG) [28], DomainAdversarial Neural Networks (DANN) [15], Class-conditional DANN (C-DANN) [32], Deep CORrelation ALignment (CORAL) [45], Maximum Mean Discrepancy (MMD) [29], Invariant Risk Minimization (IRM) [3], Adaptive Risk Minimization…”
Learning to Optimize Domain Specific Normalization for Domain Generalization (2019, background): “domain-agnostic representations [21, 20, 15, 7] while others are more interested in designing deep neural network architectures to achieve similar goals [13, 6, 17].”
Domain Generalization: A Survey (2021, background): “[166] learned a feature transformation by minimizing for all classes the variance of class-conditional distributions across source domains.”