When Training and Test Sets are Different: Characterising Learning Transfer

By Amos J. Storkey. 2013

135 citations recorded by the source corpus.

Subfields: Applied Psychology, Education, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In Education

In Evaluation & Benchmarks

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, methodology): “Storkey [54] discusses and precisely defines different possibilities how Ptrain and Ptest can differ.”
Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “, 2009) and under label shift (when p(x|y) doesn’t change) (Storkey, 2009; Zhang et al., 2013; Lipton et al., 2018), and various other assumptions (e.”
Data Augmentation Generative Adversarial Networks (2017, background): “The term dataset shift (Storkey, 2009) generalises the concept of covariate shift (Shimodaira, 2000; Storkey & Sugiyama, 2007; Sugiyama & Müller, 2005) to multiple cases of changes between domains.”
Generalization in anti-causal learning (2018, background): “This is in line with a large body of work utilizing generative modeling (without referring to causality and ICM), e.g., (Storkey, 2009).”
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, background): “Owing to the impossibility of correcting for shift absent assumptions [3], these papers often assume either covariate shift q(x, y) = q(x)p(y|x) [15, 45, 49] or label shift q(x, y) = q(y)p(x|y) [7, 29, 38, 48, 56].”
Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions (2017, background): “scribes the data generating mechanism, and which parts of the model are invariant under the different settings, allows one to transfer knowledge from one situation to the other [Spirtes et al., 2000, Storkey, 2009, Schölkopf et al., 2012, Bareinboim and Pearl, 2016]. In practice, the causal graph is”
Error asymmetry in causal and anticausal regression (2016, background): “ generation 18 Error Asymmetry in Causal and Anticausal Regression process of data and the corresponding implications for the prediction error. In terms of considering the causal nature of a problem, [22] points out that the causal direction in data matters for transfer learning tasks where the goal”
Multi-Source Domain Adaptation: A Causal View (2015, background): “This is called prior probability shift (Storkey 2009) or target shift (Zhang et al.”
Domain Adaptive Transfer Learning with Specialist Models (2018, background): “related3 to prior probability shift [28, 31] also known as target shift [39].”
Preventing Failures Due to Dataset Shift: Learning Predictive Models That Transport (2018, methodology): “By reasoning about the causal relationship between the inputs and target, various forms of dataset shift can be defined (Storkey, 2009; Schölkopf et al., 2012) which has led to methods for tackling specific instances such as covariate shift (e.”
Controlling Selection Bias in Causal Inference (2011, background): “Machine learning tasks suffer from a similar problem when training samples are selected preferentially, depending on feature-class combinations that differ from those encountered in the target environment (Zadrozny, 2004; Smith and Elkan, 2007; Storkey, 2009; Hein, 2009).”
Domain Generalization by Marginal Transfer Learning (2017, background): “Another common assumption is target shift, which stipulates that the source and target have the same class-conditional distributions, allowing the prior class probability to change (Hall, 1981; Titterington, 1983; Latinne et al., 2001; Storkey, 2009; Du Plessis and Sugiyama, 2012; Sanderson and Scot”
The Importance of Calibration for Estimating Proportions from Annotations (2018, background): “The former is assumed in the case of prior probability shift , where we assume that p ( y ) differs but p ( x | y ) is constant, and the later is assumed in the case of covariate shift , where we assume that p ( x ) differs but p ( y | x ) is constant (Storkey, 2009).”
Transportability from Multiple Environments with Limited Experiments (2013, background): “The machine learning literature has been concerned about discrepancies among domains in the context, almost exclusively, on predictive or classification tasks as opposed to learning causal or counterfactual measures [16, 17].”
Causality matters in medical imaging (2019, background): “An underﬁtted model (‘too simple’) may however introduce spurious dependencies, for which importance reweighting with p te ( x ) / p tr ( x ) is a common mitigation strategy 20,21 .”
Domain Adaptation under Target and Conditional Shift (2013, background): “We term this situation Target Shift (TarS) which is frequently encountered in practice; for instance, it is known as choice-based or endogenous stratified sampling (Manski & Lerman, 1977) in econometrics, and is sometimes called prior probability shift (Storkey, 2009).”
Recovering from Selection Bias in Causal and Statistical Inference (2014, background): “The advent of causal diagrams (Pearl 1995; Spirtes, Glymour, and Scheines 2000; Pearl 2000; Koller and Friedman 2009) provides such a language and renders the formalization of the selection problem possible.”