On the Value of Target Data in Transfer Learning

By Steve Hanneke, Samory Kpotufe. 2020

87 citations recorded by the source corpus.

We aim to understand the value of additional labeled or unlabeled target data in transfer learning, for any given amount of source data; this is motivated by practical questions around minimizing sampling costs, whereby, target data is usually harder or costlier to acquire than source data, but can yield better accuracy. To this aim, we establish the first minimax-rates in terms of both source and target sample sizes, and show that performance limits are captured by new notions of discrepancy between source and target, which we refer to as transfer exponents. Interestingly, we find that attaining minimax performance is akin to ignoring one of the source or target samples, provided distributional parameters were known a priori. Moreover, we show that practical decisions -- w.r.t. minimizing sampling costs -- can be made in a minimax-optimal way without knowledge or estimation of distributional parameters nor of the discrepancy between source and target.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Model Zoo: A Growing Brain That Learns Continually (2021, positive): “We build upon Hanneke and Kpotufe (2019; 2020) to construct the transfer exponent in §2; their work shows that even in very favorable settings, e.g., when all tasks have the same optimal classiﬁer, having access to a large number of tasks may not help.”
On Localized Discrepancy for Domain Adaptation (2020, methodology): “We also analyze how to achieve super transfer Hanneke and Kpotufe (2019) in unsupervised domain adaptation.”
A No-Free-Lunch Theorem for MultiTask Learning (2020, methodology): “To illustrate this, we recall some of the examples from (Hanneke and Kpotufe, 2019).”