Discriminative Learning Under Covariate Shift

By Steffen Bickel, Michael Brückner, Tobias Scheffer. 2009

443 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “understanding of shortcut learning which has led to a variety of mathematical concepts and an increasing amount of work under different terms such as learning under covariate shift [16], anti-causal learning [17], dataset bias [18], the tank legend [19] and the Clever Hans effect [20].”
Recommendations as Treatments: Debiasing Learning and Evaluation (2016, methodology): “Weighting approaches are also widely used in domain adaptation and covariate shift, where data from one source is used to train for a different problem (e.g., Huang et al., 2006; Bickel et al., 2009; Sugiyama & Kawanabe, 2012).”
Detecting and Correcting for Label Shift with Black Box Predictors (2018, methodology): “By contrast, the heuristic method of using logistic-regression to construct importance weights (Bickel et al., 2009) lacks theoretical justiﬁcation that the estimated weights are correct.”
Domain Generalization via Invariant Feature Representation (2013, background): “In this case, the data exhibits covariate (or dataset) shift (Widmer and Kurat, 1996; Quionero-Candela et al., 2009; Bickel et al., 2009): although the marginal distributions PX on cell attributes vary due to biological or technical variations, the functional relationship P(Y |X) across different do”
Domain Adaptation by Joint Distribution Invariant Projections (2020, methodology): “Generally, this can be accomplished by minimizing the Maximum Mean Discrepancy (MMD) distance [24], the KL divergence [25], or the logistic loss [26].”
Neurosymbolic Programming (2021, background): “More recently, there has been increased interest in studying generalization (and sample efficiency) when the target or test domain is not the same data distribution as the training domain (Bickel et al., 2009; Dudík et al., 2014; Chen et al., 2016; Beery et al., 2018; Taori et al., 2020).”
A unifying view on dataset shift in classification (2012, background): “Some of the most important ones include weighting the loglikelihood function [47], importance-weighted cross-validation [51], asymptotic Bayesian generalization error [59], discriminative learning [9], kernel mean matching [23], or adversarial search [22].”
A Two-Stage Weighting Framework for Multi-Source Domain Adaptation (2011, methodology): “Many existing methods re-weight source domain data in order to minimize the marginal probability differences between the source and target domains and learn a hypothesis on the re-weighted source data [6, 7, 8, 9].”
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation (2019, methodology): “That is, we sample rare examples more often during training, or, equivalently, utilize non-uniform misclassiﬁcation cost [14, 3].”
Multi-source domain adaptation and its application to early detection of fatigue (2011, unknown): “[5] discriminated training against test data with a probabilistic model that accounts for the marginal probability difference between training and test distribution.”
An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics (2013, methodology): “Furthermore, we may design algorithms that are capable of working under dataset shift conditions, either by means of preprocessing techniques [95] or with ad hoc algorithms [1,16,60].”
Approximating Likelihood Ratios with Calibrated Discriminative Classifiers (2015, background): “Despite its theoretical consistency, this approach is known to be ineffective in practice (Sugiyama et al., 2007; Bickel et al., 2009), since it relies on modeling numerator and denominator highdimensional densities, which is a harder problem than modeling their ratio only.”
A Systematic Review on Imbalanced Data Challenges in Machine Learning (2019, background): “…articles in the literature that have focused on the designing of imbalanced classification algorithms that may work under dataset shift conditions, which is only possible either by employing some pre-processing technique (Moreno-Torres et al. 2013) or by using ad hoc algorithm (Bickel et al. 2009).”
Domain Generalization by Marginal Transfer Learning (2017, background): “In the covariate shift problem, the marginal test distribution is diﬀerent from the marginal training distribution(s), but the posterior distribution is assumed to be the same (Bickel et al., 2009).”
The Case for Learned Index Structures (2017, methodology): “When this assumption does not hold, one could use randomly generated keys, non-keys generated by a machine learning model [34], importance weighting to directly address covariate shift [18], or adversarial training for robustness [65]; we leave this as future work.”
The Risks of Invariant Risk Minimization (2020, background): “Though this approach has become popular in the last year, it is somewhat similar to the existing concept of covariate shift (Shimodaira, 2000; Bickel et al., 2009), which considers the same setting.”
Online domain adaptation of a pre-trained cascade of classifiers (2011, unknown): “A similar argument holds true for the relevance of the previous work related to the analysis of covariate shift [2].”
Domain Adaptation via Transfer Component Analysis (2009, background): “Daum ´ e III [8] designed a heuristic kernel to augment features for solving some speciﬁc domain adaptation problems in natural language processing.”