Domain Generalization via Invariant Feature Representation

By Krikamol Muandet, David Balduzzi, Bernhard Schölkopf. 2013

1,377 citations recorded by the source corpus.

This paper investigates domain generalization: How to take knowledge acquired from an arbitrary number of related domains and apply it to previously unseen domains? We propose Domain-Invariant Component Analysis (DICA), a kernel-based optimization algorithm that learns an invariant transformation by minimizing the dissimilarity across domains, whilst preserving the functional relationship between input and output variables. A learning-theoretic analysis shows that reducing dissimilarity improves the expected generalization ability of classifiers on new domains, motivating the proposed algorithm. Experimental results on synthetic and real-world datasets demonstrate that DICA successfully learns invariant features and improves classifier performance in practice.

Subfields: Artificial Intelligence, Cancer Research, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Representation & Self-Supervised Learning

Citations in the corpus

Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “[60] Muandet, K.”
Learning Robust Global Representations by Penalizing Local Predictive Power (2019, methodology): “In domain generalization (DG) methods (Muandet et al., 2013) (or sometimes known as “zero shot domain adaptation” (Kumagai and Iwata, 2018; Niu et al., 2015; Erfani et al., 2016; Li et al., 2017c)) one possesses domain identiﬁers for a number of known in-sample domains, and the goal is to…”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “…2013, Chen et al. 2014, Schölkopf et al. 2015), covariate shift (Gretton et al. 2009, Pan et al. 2011) and domain adaptation (Blanchard et al. 2011, Muandet et al. 2013), selection bias correction (Huang et al. 2007), herding (Chen et al. 2010, Huszar and Duvenaud 2012), Markov chain Monte Carlo…”
Multiple-Source Adaptation with Domain Classifiers (2020, background): “Muandet et al. [2013] proposed Domain-Invariant Component Analysis to transform features onto a low dimensional subspace that minimizes the dissimilarity across domains.”
A Theory of Multiple-Source Adaptation with Limited Target Labeled Data (2020, background): “Hence, the convergence guarantees of the min-max mirror descent algorithm (Nemirovski and Yudin, 1983) do not hold directly.”
Agnostic Federated Learning (2019, background): “The problem of domain generalization (Pan and Yang, 2010; Muandet et al., 2013; Xu et al., 2014), where knowledge from an arbitrary number of related domains is combined to perform well on a previously unseen domain is very closely related to that of federated learning, though the assumptions about…”
Learning explanations that are hard to vary (2020, methodology): “ILC can be used in a setting of domain generalization [Muandet et al., 2013], but it is not limited to it.”
Learning Robust Representations by Projecting Superficial Statistics Out (2018, background): “…(Muandet et al., 2013) is relatively new, but has also been studied extensively: covering a wide spectrum of techniques from kernel methods (Muandet et al., 2013; Niu et al., 2015; Erfani et al., 2016; Li et al., 2017c) to more recent deep learning end-to-end methods, where the methods…”
Domain Generalization via Conditional Invariant Representations (2018, methodology): “(Muandet, Balduzzi, and Sch¨olkopf 2013) proposed domain-invariant component analysis (DICA) through learning an invariant feature representation h ( X ) , in which the difference between marginal distributions P ( h ( X )) is minimized.”
Domain Adaptation with Conditional Transferable Components (2016, methodology): “Other methods learn a low dimensional projection by explicitly minimizing the discrepancy between the distributions of projected features on source and target domains (Pan et al., 2011; Long et al., 2014; 2015; Baktashmotlagh et al., 2013; Si et al., 2010; 2011; Muandet et al., 2013).”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, neutral): “…successfully in many domains such as graphical models (Song et al. 2010, 2011), two-sample testing (Gretton et al. 2012), domain adaptation (Huang et al. 2007; Gretton et al. 2009; Muandet et al. 2013), and supervised learning on probability distributions (Muandet et al. 2012; Szabó et al. 2014).”
Few-Shot Adversarial Domain Adaptation (2017, background): “Others find a shared latent space that both domains can be mapped to before classification [35, 2, 39, 13, 14, 41, 37, 38].”
Deep Sets (2017, background): “…however, can only deal with ﬁxed, ﬁnite-dimensional representations, and researchers only recently started to investigate the case when the inputs or outputs are permutation invariant sets rather than ﬁnite dimensional vectors (Oliva et al., 2013; Szabo et al., 2016; Muandet et al., 2013; 2012).”
Learning to Generalize: Meta-Learning for Domain Generalization (2017, neutral): “This has been achieved with multi-view au-toencoders (Ghifary et al. 2015) and mean map embedding-based techniques (Muandet, Balduzzi, and Sch¨olkopf 2013).”
In Search of Lost Domain Generalization (2020, background): “In particular, the literature in domain generalization assumes access to multiple datasets during training, each of them containing examples about the same task, but collected under a different domain or environment [Blanchard et al., 2011, Muandet et al., 2013].”