Adaptation Algorithm and Theory Based on Generalized Discrepancy

By Corinna Cortes, Mehryar Mohri, Andrés Muñoz Medina. 2014

68 citations recorded by the source corpus.

We present a new algorithm for domain adaptation improving upon the discrepancy minimization algorithm (DM), which was previously shown to outperform a number of popular algorithms designed for this task. Unlike most previous approaches adopted for domain adaptation, our algorithm does not consist of a fixed reweighting of the losses over the training sample. Instead, it uses a reweighting that depends on the hypothesis considered and is based on the minimization of a new measure of generalized discrepancy. We give a detailed description of our algorithm and show that it can be formulated as a convex optimization problem. We also present a detailed theoretical analysis of its learning guarantees, which helps us select its parameters. Finally, we report the results of experiments demonstrating that it improves upon the DM algorithm in several tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Agnostic Federated Learning (2019, background): “…source to a known target domain (Ben-David, Blitzer, Crammer, and Pereira, 2006; Mansour, Mohri, and Rostamizadeh, 2009b; Cortes and Mohri, 2014; Cortes, Mohri, and Mu˜noz Medina, 2015), either through unsupervised adaptation techniques (Gong et al., 2012; Long et al., 2015; Ganin and…”
A New PAC-Bayesian Perspective on Domain Adaptation (2015, unknown): “To summarize, the domain adaptation bounds reviewed in this section (see Zhang et al., 2012; Cortes et al., 2010; 2015, for other bounds) express a similar trade-oﬀ between three terms: (i) the source risk, (ii) the distance between source and target marginal distributions over X , (iii) a…”
Bounding User Contributions: A Bias-Variance Trade-off in Differential Privacy (2019, background): “As such, we could in principle leverage the literature in domain adaptation to bound this term (Blitzer et al., 2007; Cortes et al., 2015).”
Bridging Theory and Algorithm for Domain Adaptation (2019, background): “These earliest theories have later been extended in many ways, from loss functions to Bayesian settings and regression problems (Mohri & Medina, 2012; Germain et al., 2013; Cortes et al., 2015).”
On Localized Discrepancy for Domain Adaptation (2020, methodology): “06 24 2v 1 [ cs by a series of works Mansour et al. (2009a); Germain et al. (2013); Cortes et al. (2015); Zhang et al. (2019).”
Regularized Policy Iteration with Nonparametric Function Spaces (2016, background): “A number of works in the domain adaptation literature consider this scenario under the name of covariate shift problem, see e.g., Ben-David et al. 2006; Mansour et al. 2009; Ben-David et al. 2010; Cortes et al. 2015. where the supremum comes from the negative sign of h − T π Q 2 ν .”
Domain Generalization by Marginal Transfer Learning (2017, background): “Finally, there are several works that assume the existence of a predictor that achieves good performance on both source and target domains (Ben-David et al., 2007, 2010; Blitzer et al., 2008; Mansour et al., 2009a; Cortes et al., 2015; Germain et al., 2016).”
An introduction to domain adaptation and transfer learning (2018, background): “Nonetheless, it is possible to derive generalization error bounds if more is known on the relationship between S and T [4,29,43,144,16,215,78].”