Domain Adaptation under Target and Conditional Shift

By Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, Zhikun Wang. 2013

701 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “, 2009) and under label shift (when p(x|y) doesn’t change) (Storkey, 2009; Zhang et al., 2013; Lipton et al., 2018), and various other assumptions (e.”
An Analysis of the Adaptation Speed of Causal Models (2020, background): “A recent line of work describes these distribution shifts as interventions in an underlying causal model (Zhang et al., 2013; Magliacane et al., 2018).”
Learning Transferable Features with Deep Adaptation Networks (2015, methodology): “A rich line of prior works have focused on learning shallow features by jointly minimizing a distance metric of domain discrepancy (Pan et al., 2011; Long et al., 2013; Baktashmotlagh et al., 2013; Gong et al., 2013; Zhang et al., 2013; Ghifary et al., 2014; Wang & Schneider, 2014).”
Federated Visual Classification with Real-World Data Distribution (2020, methodology): “Importance sampling is a classic technique for variance reduction in Monte Carlo methods [12,9] and has been used widely in domain adaption literature for countering covariate and target shift [24,36,22].”
Towards Causal Representation Learning (2021, background): “The SMS hypothesis was stated in [181, 24, 221, 115], and in earlier form in [218, 279, 220].”
A theory of independent mechanisms for extrapolation in generative models (2020, background): “Our ultimate goal is more akin to the use of a causal framework to enforce domain adaptation (Zhang et al. 2013; Zhang, Gong, and Sch¨olkopf 2015) and domain shift robustness of leaning algorithms, which has been done by exploiting additional information in the context of classiﬁcation (Heinze-Deml…”
Causality for Machine Learning (2019, background): “The line of work provided rather fruitful (Zhang et al., 2013; Weichwald et al., 2014; Zhang et al., 2015; Gong et al., 2016; Huang et al., 2017; Blöbaum et al., 2016; Guo et al., 2018; Wang et al., 2019; Subbaswamy et al., 2018; Rabanser et al., 2018; Li et al., 2018a; Lipton et al., 2018; Li et al”
Causal Discovery from Heterogeneous/Nonstationary Data (2019, background): “Since distribution shifts in heterogeneous/nonstationary data are usually constrained - it might be due to the changes in the data-generating processes of only a few variables, and thus it is only necessary to adapt a small proportion of the joint distribution in domain adaptation problems (Zhang et”
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, background): “Owing to the impossibility of correcting for shift absent assumptions [3], these papers often assume either covariate shift q(x, y) = q(x)p(y|x) [15, 45, 49] or label shift q(x, y) = q(y)p(x|y) [7, 29, 38, 48, 56].”
Learning Robust Representations by Projecting Superficial Statistics Out (2018, methodology): “…2000; Gretton et al., 2009), and label shift (also known as target shift or prior probability shift ) (Storkey, 2009; Sch¨olkopf et al., 2012; Zhang et al., 2013; Lipton et al., 2018), principled methods exist for estimating the shifts and retraining under the importance-weighted ERM…”
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms (2018, methodology): “These methods work by re-weighting the training data or extracting transferable features (e.g., Shimodaira (2000); Gretton et al. (2009); Gong et al. (2016); Zhang et al. (2013)).”
Domain Generalization via Conditional Invariant Representations (2018, background): “(Zhang et al. 2013) made an assumption that features from source domains can be transferred to the target domain by a locationscale transformation, which is restricted in real-world applications because of the presence of noises in features.”
Detecting and Correcting for Label Shift with Black Box Predictors (2018, methodology): “In our experiments, we ﬁnd that the largest n for which we can feasibly run the KMM code is roughly 8 , 000 and that is where we unfortunately have to stop for the MNIST experiment.”
Conditional variance penalties and domain shift robustness (2017, background): “…advantage that the predictions will be valid even under arbitrarily large interventions on all predictor variables (Haavelmo, 1944; Aldrich, 1989; Pearl, 2009; Sch¨olkopf et al., 2012; Peters et al., 2016; Zhang et al., 2013, 2015; X. Yu, 2017; M. Rojas-Carulla, 2017; Magliacane et al., 2017).”
Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions (2017, methodology): “The methods proposed in [Schölkopf et al., 2012, Zhang et al., 2013, 2015, Gong et al., 2016] all address challenging settings in which conditional independences that follow from the usual Markov and faithfulness assumptions alone do not sufﬁce to solve the problem, but additional assumptions on…”
Group invariance principles for causal generative models (2017, background): “This includes important cases related to transfer leaning such as covariate-shift or changes in the input distribution [Zhang et al., 2013, 2015; Peters et al., 2016; Bareinboim and Pearl, 2016; Rojas-Carulla et al., 2015], making causal generative models highly relevant in machine learning.”
Multi-Source Domain Adaptation: A Causal View (2015, background): “For instance, (Zhang et al. 2013) considers DA with a single source domain, and assumes that the change in PX|Y follows the location-scale (LS) transformation; P t X|Y is then generally identifiable.”
Unsupervised Domain Adaptation with Residual Transfer Networks (2016, background): “The goal of this paper is to design a deep neural network that enables learning of transfer classifiers y = fs (x) and y = ft (x) to close the source-target discrepancy, such that the expected target risk Rt (ft) = E(x,y)∼q [ft (x) 6= y] can be bounded by leveraging the source domain supervised…”