Stability and Hypothesis Transfer Learning

By Ilja Kuzborskij, Francesco Orabona. 2013

195 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Dacheng Tao
2. Mingsheng Long
3. Jiashi Feng
4. Qiang Yang
5. Trevor Darrell
6. Yoshua Bengio
7. Yi Yang
8. Han-Jia Ye

Citations in the corpus

• The Benefit of Multitask Representation Learning (2015, background): “…their applications to specific problems (see Ando and Zhang, 2005; Argyriou et al., 2008; Baxter, 2000; Ben-David and Schuller, 2003; Caruana, 1998; Cavallanti et al., 2010; Kuzborskij and Orabona, 2013; Maurer et al., 2013; Pentina and Lampert, 2014; Widmer et al., 2013, and references therein).”
• Scalable Transfer Learning with Expert Models (2020, background): “This can improve both convergence rates [4, 3, 6, 14, 32, 39, 40] and empirical performance [11, 13, 43, 54, 59].”
• Multi-Task Learning and Algorithmic Stability (2015, methodology): “Moreover, stability has been applied in (Maurer 2005; Kuzborskij and Orabona 2013) to analyze transfer learning algorithms which are related to multi-task learning but have different settings since transfer learning aims to improve the performance of only the target task by leveraging the…”
• Overcoming Negative Transfer: A Survey (2020, unknown): “Error restriction &Output limit [18], [20], [88], [89]”
• Privacy-preserving Transfer Learning for Knowledge Sharing (2018, methodology): “In this paper, we start from the hypothesis transfer learning method of (Kuzborskij and Orabona 2013), which we use to combine with private-preserving logistic regression (Chaudhuri, Monteleoni, and Sarwate 2011).”
• Distilled Person Re-Identification: Towards a More Scalable System (2019, background): “Hypothesis transfer learning (HTL) [26] studies learning from source models without source data.”
• Efficient Meta Learning via Minibatch Proximal Update (2019, background): “Minibatch proximal learning is also identical in principle to biased regularized hypothesis transfer learning, which has been explored experimentally with success in many applications [31, 32, 33] and theoretically with rigourous guarantees [34, 35].”
• Differentially Private Hypothesis Transfer Learning (2018, background): “In the transfer learning literature, there are four major approaches – instance re-weighting [22,17,11], feature representation transfer [32], hypothesis transfer [19,38,20] and relational knowledge transfer [26].”
• Data-Dependent Stability of Stochastic Gradient Descent (2017, background): “…(Koren and Levy, 2015; Gonen and Shalev-Shwartz, 2017), or even to design enhanced algorithms that minimize these bounds in a data-driven way, e.g. by exploiting side information as in transfer (Kuzborskij and Orabona, 2013; Ben-David and Urner, 2013) and metric learning (Perrot and Habrard, 2015).”
• Lifelong Learning with Weighted Majority Votes (2016, background): “The need to keep little information from previous tasks has also been argued for in the context of domain adaptation [16].”
• On Localized Discrepancy for Domain Adaptation (2020, background): “Kuzborskij and Orabona (2013); l Perrot and Habrard (2015); Du et al. (2017); Hanneke and Kpotufe (2019) study hypothesis transfer which learns from smaller target labeled data and auxiliary hypothesis.”
• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, unknown): “Shallow HTL methods [54], [55], [56], [57], [58], [59] generally assume that the optimal target hypothesis is closely related to these source hypotheses.”
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020, background): “Hypothesis Transfer Learning (HTL).”
• Towards Knowledgeable Supervised Lifelong Learning Systems (2020, unknown): “A number of theoretical (Kuzborskij & Orabona, 2013; Kuzborskij, 2018), experimental (Yang et al., 2007; Aytar & Zisserman, 2011; Tommasi et al., 2014; Kuzborskij et al., 2015; Oneto et al., 2015; Mozafari & Jamzad, 2016; Wang & Hebert, 2016), and application-specific methods (Valerio, Passarella,…”
• A Survey on Negative Transfer (2020, background): “Regression Transfer Output truncation [24] Regularization [25] Bayesian optimization [26]”
• Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer (2020, background): “Generally, HTL is an attractive and efficient framework that assumes access to a given number of source hypotheses and a small set of training samples from the target domain.”
• Understanding How Feature Structure Transfers in Transfer Learning (2017, background): “Most of the existing theoretical analyses [Baxter, 2000; Maurer, 2009; Kuzborskij and Orabona, 2013; Maurer et al., 2013] for learning to learn focus on this assumption.”
• On the Value of Target Data in Transfer Learning (2020, methodology): “Interestingly, we show that the optimal (nP , nQ)-rate is achieved by a generic approach, akin to so-called hypothesis-transfer [1, 2], which optimizes Q-error under the constraint of low P -error, and does so without knowledge of distributional parameters such as ⇢.”