When Training and Test Sets Are Different: Characterizing Learning Transfer

By Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, Neil D. Lawrence. 2009

310 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, methodology): “In contrast, under dataset shift one assumes that there are two environments, namely training and testing [56], with the constraint that the model cannot be adapted using data from the target environment.”
Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “…techniques under covariate shift (when p ( y | x ) does not change) (Gretton et al., 2009) and under label shift (when p ( x | y ) doesn’t change) (Storkey, 2009; Zhang et al., 2013; Lipton et al., 2018), and various other assumptions (e.g. bounded divergences between source and target…”
Learning Robust Representations by Projecting Superficial Statistics Out (2018, methodology): “…such as covariate shift (Shimodaira, 2000; Gretton et al., 2009), and label shift (also known as target shift or prior probability shift ) (Storkey, 2009; Sch¨olkopf et al., 2012; Zhang et al., 2013; Lipton et al., 2018), principled methods exist for estimating the shifts and retraining…”
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms (2018, background): “Proactive and Reactive Approaches: Reactive predictive modeling methods for countering dataset shift typically require representative unlabeled samples from the test distribution (Storkey, 2009).”
Detecting and Correcting for Label Shift with Black Box Predictors (2018, methodology): “Compared to approaches based on Kernel Mean Matching (KMM) (Zhang et al., 2013), EM (Chan & Ng, 2005), and Bayesian inference (Storkey, 2009”
Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions (2017, background): “…domain, but much more accurate ones in the target domain. across the different domains, allows one to transfer knowledge from one domain to the other in order to address the problem of domain adaptation [Spirtes et al., 2000, Storkey, 2009, Schölkopf et al., 2012, Bareinboim and Pearl, 2016].”
Multi-Source Domain Adaptation: A Causal View (2015, background): “This is called prior probability shift (Storkey 2009) or target shift (Zhang et al. 2013).”
Learning under Concept Drift: A Review (2019, background): “Concept drift has also been defined by various authors using alternative names, such as dataset shift [15] or concept”
Domain Adaptation under Target and Conditional Shift (2013, background): “We term this situation Target Shift (TarS) which is frequently encountered in practice; for instance, it is known as choice-based or endogenous stratified sampling (Manski & Lerman, 1977) in econometrics, and is sometimes called prior probability shift (Storkey, 2009).”
Causal Generative Domain Adaptation Networks (2018, methodology): “(Storkey, 2009; Zhang et al., 2013; Iyer et al., 2014) assumed only changes in PY and proposed principled methods to estimate PY in the target domain by matching marginal distributions.”
Causal Transfer Learning (2017, background): “…of the causal graph that describes the data generating mechanism, and which parts of the model are invariant under the different settings, allows one to transfer knowledge from one situation to the other [Spirtes et al., 2000, Storkey, 2009, Schölkopf et al., 2012, Bareinboim and Pearl, 2016].”
Transportability of Causal and Statistical Relations: A Formal Approach (2011, background): “In practice, such knowledge may only be partially available and, as is the case in every mathematical exercise, the benefit of the analysis lies primarily in understanding what knowledge is needed for the task to succeed and how sensitive conclusions are to knowledge that we do not possess.”
Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features (2018, background): “assumption is often violated in practice (Storkey, 2009).”
Preventing Failures Due to Dataset Shift: Learning Predictive Models That Transport (2018, methodology): “By reasoning about the causal relationship between the inputs and target, various forms of dataset shift can be defined (Storkey, 2009; Schölkopf et al., 2012) which has led to methods for tackling specific instances such as covariate shift (e.”
Gradient Regularized Contrastive Learning for Continual Domain Adaptation (2020, background): “The performance of the model learned on one domain may degrade significantly on other domains because of different data distribution (Luo et al. 2019; Ben-David et al. 2010; Moreno-Torres et al. 2012; Storkey 2009; Yu et al. 2020).”
SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary (2018, background): “Prior Probability Shift : when the class distribution is different between the training and test sets (Storkey, 2009).”