Covariate Shift by Kernel Mean Matching

By Arthur Gretton, Alex Smola, Jiayuan Huang, M. Schmittfull, Karsten M. Borgwardt, Bernhard Schölkopf, Quiñonero Candela, Masashi Sugiyama, Anton Schwaighofer. 2009

699 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “Several modern works address principled learning techniques under covariate shift (when p ( y | x ) does not change) (Gretton et al., 2009) and under label shift (when p ( x | y ) doesn’t change) (Storkey, 2009; Zhang et al., 2013; Lipton et al., 2018), and various other assumptions (e.g. bounded…”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “…et al. 2014, Chwialkowski and Gretton 2014), causal discovery (Sgouritsa et al. 2013, Chen et al. 2014, Schölkopf et al. 2015), covariate shift (Gretton et al. 2009, Pan et al. 2011) and domain adaptation (Blanchard et al. 2011, Muandet et al. 2013), selection bias correction (Huang et al.…”
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, background): “Owing to the impossibility of correcting for shift absent assumptions [3], these papers often assume either covariate shift q(x, y) = q(x)p(y|x) [15, 45, 49] or label shift q(x, y) = q(y)p(x|y) [7, 29, 38, 48, 56].”
Learning Robust Representations by Projecting Superficial Statistics Out (2018, background): “Under restrictive assumptions, such as covariate shift (Shimodaira, 2000; Gretton et al., 2009), and label shift (also known as target shift or prior probability shift ) (Storkey, 2009; Sch¨olkopf et al., 2012; Zhang et al., 2013; Lipton et al., 2018), principled methods exist for estimating the…”
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms (2018, methodology): “These methods work by re-weighting the training data or extracting transferable features (e.g., Shimodaira (2000); Gretton et al. (2009); Gong et al. (2016); Zhang et al. (2013)).”
Detecting and Correcting for Label Shift with Black Box Predictors (2018, background): “Covariate shift, also called sample selection bias , is well-studied (Zadrozny, 2004; Huang et al., 2007; Sugiyama et al., 2008; Gretton et al., 2009).”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, neutral): “2012), domain adaptation (Huang et al. 2007; Gretton et al. 2009; Muandet et al. 2013), and supervised learning on probability distributions (Muandet et al.”
Domain Generalization via Invariant Feature Representation (2013, background): “A similar idea has been adopted in domain adaptation, where the learned subspace reduces mismatch between source and target domains (Gretton et al. 2009, Pan et al. 2011).”
Batch Stationary Distribution Estimation (2020, methodology): “Assumption 2 follows previous work (Liu and Lee, 2017; Nachum et al., 2019), and is common in density ratio estimation (Sugiyama et al., 2008; Gretton et al., 2009) and off-policy evaluation (Wang et al., 2017; Xie et al., 2019).”
DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections (2019, methodology): “…which may magnify the estimation error), various direct ratio estimators have been proposed [44], including the moment matching approach [17], probabilistic classiﬁcation approach [4, 7, 40], and ratio matching approach [22, 33, 45] The proposed DualDICE algorithm, as a direct approach…”
Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment (2019, background): “Traditional approaches to both problems require the source distributions’ support to cover the target support, estimating adapted classifiers via importance-weighted risk minimization (Shimodaira, 2000; Huang et al., 2007; Gretton et al., 2009; Yu & Szepesvári, 2012; Lipton et al., 2018).”
Troubling Trends in Machine Learning Scholarship (2018, background): “a change in the distribution of model inputs. In fact, covariate shift refers to a speciﬁc type of shift where although the input distribution p(x) might change, the labeling function p(y|x) does not [27]. Moreover, due to the inﬂuence of [33], Google Scholar lists batch normalization as the ﬁrst re”
Causal Transfer in Machine Learning (2015, methodology): “In a general setting, for instance, assumptions about the support of the training distributions P 1 , . . . , P D and the test distribution P T must be made for methods such as re-weighting to be expected to work [e.g. Gretton et al., 2009].”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “endently with a domain classiﬁer [267] or by approximating directly the ratio between the densities with a Kullback-Leibler Importance Estimation Procedure [215, 125]. Another popular measure used in [118, 108, 159] to weight data instances is the Maximum Mean Discrepancy (MMD) [17] computed between”
Selective Transfer Machine for Personalized Facial Action Unit Detection (2013, methodology): “Considering distribution mismatch, Kernel Mean Matching (KMM) [16] directly infers the re-sampling weights by matching training and test distributions.”