Domain Generalization by Marginal Transfer Learning
By Gilles Blanchard, Aniket Anand Deshmukh, Ü. Dogan, Gyemin Lee, Clayton D. Scott. 2017
354 citations recorded by the source corpus.
In the problem of domain generalization (DG), there are labeled training data sets from several related prediction problems, and the goal is to make accurate predictions on future unlabeled data sets that are not known to the learner. This problem arises in several applications where data distributions fluctuate because of environmental, technical, or other sources of variation. We introduce a formal framework for DG, and argue that it can be viewed as a kind of supervised learning problem by augmenting the original feature space with the marginal distribution of feature vectors. While our framework has several connections to conventional analysis of supervised learning algorithms, several unique aspects of DG require new methods of analysis. This work lays the learning theoretic foundations of domain generalization, building on our earlier conference paper where the problem of DG was introduced Blanchard et al., 2011. We present two formal models of data generation, corresponding notions of risk, and distribution-free generalization error analysis. By focusing our attention on kernel methods, we also provide more quantitative results and a universally consistent algorithm. An efficient implementation is provided for this algorithm, which is experimentally compared to a pooling strategy on one synthetic and three real-world data sets.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- In Search of Lost Domain Generalization (2020, background): “See Blanchard et al. [2017], Deshmukh et al. [2019] for theoretical results on this family of algorithms.”
- SelfReg: Self-supervised Contrastive Regularization for Domain Generalization (2021, background): “This testbed currently provides seven multi-domain datasets (i.e. ColoredMNIST [1], RotatedMNIST [16], VLCS [13], PACS [26], Office-Home [42], and TerraIncognita [2], DomainNet [34]) and provides benchmarks results of 14 baseline approaches (i.e. ERM [41], IRM [1], GroupDRO [36], Mixup [45], MLDG [27”
- Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021, methodology): “…for group shifts) (Sagawa et al. 2020) optimizes ERM and adjusts the weights of domains with larger errors; MTL (Marginal Transfer Learning) (Blanchard et al. 2021) concatenates the original featureswith an augmented vector using the marginal distribution of feature vectors, which…”
- Towards Out-Of-Distribution Generalization: A Survey (2021, neutral): “…[46], [47], [48], [49] [37], [39], [50], [51], [51], [52], [53], [53], [54], [54], [55], [56], [57], [58], [59], [60] [6], [61], [62], [63], [63], [64], [65], [66], [67], [68], [69], [70](Section 4.1.1) [52], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82] [83], [84], [85],…”
- On the Value of Target Data in Transfer Learning (2020, background): “More recently, [22, 23] consider the nonparametric case (unbounded VC) under alternative performance criteria than 0-1 loss.”
- Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, methodology): “Blanchard et al. [6] construct such a kernel using kernels k X , k X on X and kernel κ on the RKHS the kernel embedding of distribution P X via kernel k (cid:48) .”
- Domain Generalization: A Survey (2021, background): “Such a problem refers to the distribution shift between a set of training (source) data and a set of test (target) data [1], [2], [3], [4], [5].”
- Domain Generalization Under Posterior Drift (2025, unknown): “• A quantification of the decision-theoretic gap between DG and domain adaptation, resolving an open question posed in Blanchard et al. [2021, Lemma 9].”