Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation

By Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul von Bünau, Motoaki Kawanabe. 2007

962 citations recorded by the source corpus.

Subfields: Causal Inference & Discovery, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Causal Inference & Discovery

In Statistical Learning Theory

Citations in the corpus

Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, background): “Owing to the impossibility of correcting for shift absent assumptions [3], these papers often assume either covariate shift q(x, y) = q(x)p(y|x) [15, 45, 49] or label shift q(x, y) = q(y)p(x|y) [7, 29, 38, 48, 56].”
Detecting and Correcting for Label Shift with Black Box Predictors (2018, background): “Covariate shift, also called sample selection bias , is well-studied (Zadrozny, 2004; Huang et al., 2007; Sugiyama et al., 2008; Gretton et al., 2009).”
Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions (2017, methodology): “Any method (e.g., least-squares regression) could in principle be used to predict Y from a given set of covariates, but it is advisable to use a prediction method that works well under covariate shift, e.g., [Sugiyama et al., 2008].”
GenDICE: Generalized Offline Estimation of Stationary Values (2020, background): “Classical density ratio estimation includes moment matching (Gretton et al., 2008), probabilistic classification (Bickel et al., 2007), and ratio matching (Nguyen et al., 2008; Sugiyama et al., 2008; Kanamori et al., 2009).”
How to Explain Individual Classification Decisions (2009, methodology): “For further approaches to data set shift see Sugiyama, Nakajima, Kashima, von Buenau, and Kawanabe (2007b), Sugiyama, Krauledat, and M¨uller (2007a) and the book by Quionero-Candela, Sugiyama, Schwaighofer, and Lawrence (2009).”
Domain Adaptive Transfer Learning with Specialist Models (2018, methodology): “We adopt similar ideas of importance weighting examples [32, 28, 39] and adapt them to the pre-training step instead, showing that this is an effective approach.”
Causal Transfer in Machine Learning (2015, positive): “For instance, if an unlabeled sample from the test task is available at training in the DG setting, the training sample can be re-weighted via importance sampling [Gretton et al., 2009, Shimodaira, 2000, Sugiyama et al., 2008] so that it becomes representative of the test task.”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “This is done either by estimating these probabilities independently with a domain classiﬁer [267] or by approximating directly the ratio between the densities with a Kullback-Leibler Importance Estimation Procedure [215, 125].”
Optimal Transport for Domain Adaptation (2014, background): “ing [45], manyworks have considered finding a common fea-”
Deep Transfer Learning with Joint Adaptation Networks (2016, background): “covariate shift (Huang et al., 2006; Sugiyama et al., 2008)), the conditional distributions P (Y|X) (a.”
Stable Prediction across Unknown Environments (2018, methodology): “Recently, many methods (Shimodaira (2000); Bickel et al. (2009); Sugiyama et al. (2008); Huang et al. (2007); Dud´ık et al. (2006); Liu and Ziebart (2014)) have been proposed to address this problem.”
Small Sample Learning in Big Data Era (2018, methodology): “The early works proposed in this manner were constructed based on instance re-weighting (Zadrozny, 2004; Sugiyama et al., 2008; Kanamori et al., 2009; Huang et al., 2007; Yan et al., 2017), which estimated the ratio between the likelihoods of being a source or target example or use maximum mean…”
A Survey on Transfer Learning (2010, methodology): “Sugiyama et al. [34] proposed an algorithm known as Kullback-Leibler Importance Estimation Procedure (KLIEP) to estimate P ðx S i Þ P ðxT i Þ directly, based on the minimization of the Kullback-Leibler divergence. can be integrated with cross-validation to perform model selection automatically in tw”
Fairness Violations and Mitigation under Covariate Shift (2019, methodology): “This criterion generalizes the covariate shift criterion [69], which assumes independence between Y andC conditioned on all the fea-”
Domain Adaptation with Regularized Optimal Transport (2014, methodology): “Two main strategies have been considered to tackle the domain adaptation problem: on the one hand, there are approaches considering the transfer of instances, mostly via sample re-weighting schemes based on density ratios between the source and target domains [2,8].”
A Minimax Surrogate Loss Approach to Conditional Difference Estimation (2018, methodology): “Conversely, in our work, we use a traditional inverse propensity score model [35, 14] or other method to estimate the ratio of densities, and propose a single support vector machine formulation to estimate treatment effects.”
Cross Validation Framework to Choose amongst Models and Datasets for Transfer Learning (2010, methodology): “Some methods have been exploited for this problem [9, 10].”