Direct importance estimation for covariate shift adaptation

By Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul von Bünau, Motoaki Kawanabe. 2008

498 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

Domain Adaptation with Conditional Transferable Components (2016, background): “In this case, one can match the feature distribution P ( X ) on source and target domains by importance reweighting methods if the source domain is richer than the target domain (Shimodaira, 2000; Sugiyama et al., 2008; Huang et al., 2007).”
Multi-Source Domain Adaptation: A Causal View (2015, methodology): “…is usually the case, under-ﬁt of the conditional model causes the predicted Y to depend on the input distribution P X ; importance reweighting according to the difference in P X between the source and target domains is widely used to correct covariate shift (Shi-modaira 2000; Sugiyama et al. 2008).”
Learngene: From Open-World to Your Learning Task (2021, background): “Transfer learning: Methods such as (Huang et al. 2006; Sugiyama et al. 2008; Sun et al. 2011; Shen et al. 2018; Lee et al. 2019; Duan et al. 2009; Zhuang et al. 2009; Evgeniou, Micchelli, and Pontil 2005), which focus on transferring the knowledge across domains, aim at improving the performance of ”
Batch Stationary Distribution Estimation (2020, methodology): “Assumption 2 follows previous work (Liu and Lee, 2017; Nachum et al., 2019), and is common in density ratio estimation (Sugiyama et al., 2008; Gretton et al., 2009) and off-policy evaluation (Wang et al., 2017; Xie et al., 2019).”
DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections (2019, methodology): “error), various direct ratio estimators have been proposed [37], including the moment matching approach [13], probabilistic classification approach [3, 5, 33], and ratio matching approach [17, 27, 38] The proposed DualDICE algorithm, as a direct approach for density ratio estimation, bears some simi”
A Comprehensive Survey on Transfer Learning (2019, methodology): “Therefore, they adapt the domains by correcting the sample selection bias [5] or covariate shift [6].”
Conditional Density Estimation via Least-Squares Density Ratio Estimation (2010, methodology): “Under the above assumptions, we have the following theorem (its proof is omitted since it follows essentially the same line as Sugiyama et al., 2008).”
Toward Unsupervised Adaptation of LDA for Brain–Computer Interfaces (2011, methodology): “Importance estimation is a separate task for which several methods have been proposed [ 35 ], [36], all of which require an unlabeled sample from the test distribution.”
Distinguishing Cause from Effect Based on Exogeneity (2015, background): “Otherwise, if the model is over-simpliﬁed, the estimated conditional distribution will depend on the marginal, which inspires the importance-reweighting scheme to handle learning problems under co-variate shift (see e.g., Footnote 1 in [24]).”
EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy (2019, unknown): “…this information together facilitates investigation and proliferation of several interesting and important questions in BCI and neuroscience in general, such as mental state estimation [59,60], multi-modal data-fusion [56,61,62], and covariance shifts of the input space [63,64], among many others.”
Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition (2012, methodology): “So far, various methods for estimating the importance has been developed [22, 7, 18, 12, 27, 13].”
Tighter Variational Representations of f-Divergences via Restriction to Probability Measures (2012, result): “We also experimented with another density ratio estimation method (Sugiyama et al., 2008), with very similar results.”
Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation (2012, methodology): “λ (≥ 0) denotes the regularization parameter, which is chosen by cross-validation (Sugiyama et al., 2008).”
Domain Adaptation by Joint Distribution Invariant Projections (2020, unknown): “Generally, this can be accomplished by minimizing the Maximum Mean Discrepancy (MMD) distance [24], the KL divergence [25], or the logistic loss [26].”
Causal Generative Domain Adaptation Networks (2018, methodology): “In the covariate shift scenario, to correct the shift in PX , earlier approaches reweight the source domain data by density ratios estimated from unlabeled data in both domains (Shimodaira, 2000; Sugiyama et al., 2008; Huang et al., 2007).”
Invariant Risk Minimization Games (2020, background): “Importance weighting or more generally sample weighting attempts to match test and train distributions by reweighting samples (Shimodaira, 2000; Sugiyama et al., 2008; Gretton et al., 2009; Zhao et al., 2018).”
On Localized Discrepancy for Domain Adaptation (2020, background): “There are also works concentrating on domain adaptation with importance weighting methods Sugiyama et al. (2007); Sugiyama et al. (2008); Cortes et al. (2010).”
Relative deviation learning bounds and generalization with unbounded loss functions (2013, background): “…are not uniformly bounded, and a variety of applications such as sample bias correction (Dudı́k et al., 2006; Huang et al., 2006; Cortes et al., 2008; Sugiyama et al., 2008; Bickel et al., 2007), domain adaptation (Ben-David et al., 2007; Blitzer et al., 2008; Daumé III and Marcu, 2006; Jiang and…”
Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation (2012, methodology): “This formulation is called the KL importance estimation procedure (KLIEP; Sugiyama et al. 2008).”