A theory of transfer learning with applications to active learning

By Liu Yang, Steve Hanneke, Jaime G. Carbonell. 2012

184 citations recorded by the source corpus.

We explore a transfer learning setting, in which a finite sequence of target concepts are sampled independently with an unknown distribution from a known family. We study the total number of labeled examples required to learn all targets to an arbitrary specified expected accuracy, focusing on the asymptotics in the number of tasks and the desired accuracy. Our primary interest is formally understanding the fundamental benefits of transfer learning, compared to learning each target independently from the others. Our approach to the transfer problem is general, in the sense that it can be used with a variety of learning protocols. As a particularly interesting application, we study in detail the benefits of transfer for self-verifying active learning; in this setting, we find that the number of labeled examples required for learning with transfer is often significantly smaller than that required for learning each target independently.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

On the Value of Target Data in Transfer Learning (2020, background): “The importance of unlabeled data in transfer-learning, given the cost of target labels, has always been recognized, with various approaches developed over the years [15, 16], including more recent research efforts into so-called semisupervised or active transfer, where, given unlabeled target data, ”
Characterizing and Avoiding Negative Transfer (2018, background): “Transfer learning [20, 35] uses knowledge learned in the source domain to assist training in the target domain.”
On Localized Discrepancy for Domain Adaptation (2020, background): “Yang et al. (2013); Chattopadhyay et al. (2013); Berlind and Urner (2015) study active transfer learning which allows requirements for a few labels of target data.”
Domain Generalization by Marginal Transfer Learning (2017, background): “A related approach is to learn directly the task sampling distribution µ (Carbonell et al., 2013).”
Towards more Reliable Transfer Learning (2018, methodology): “ng Hilbert space embeddings of distributions. A key benet of these methods is to address the cold start problem of active learning by introducing a version space prior learned from transfer learning [34]. However, unlike our approach, these methods all assume that source data are richly labeled and ”