Understanding How Feature Structure Transfers in Transfer Learning

By Tongliang Liu, Qiang Yang, Dacheng Tao. 2017

42 citations recorded by the source corpus.

Transfer learning transfers knowledge across domains to improve the learning performance. Since feature structures generally represent the common knowledge across different domains, they can be transferred successfully even though the labeling functions across domains differ arbitrarily. However, theoretical justification for this success has remained elusive. In this paper, motivated by selftaught learning, we regard a set of bases as a feature structure of a domain if the bases can (approximately) reconstruct any observation in this domain. We propose a general analysis scheme to theoretically justify that if the source and target domains share similar feature structures, the source domain feature structure is transferable to the target domain, regardless of the change of the labeling functions across domains. The transferred structure is interpreted to function as a regularization matrix which benefits the learning process of the target domain task. We prove that such transfer enables the corresponding learning algorithms to be uniformly stable. Specifically, we illustrate the existence of feature structure transfer in two well-known transfer learning settings: domain adaptation and learning to learn.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Dacheng Tao
3. Yu Zhang
4. Ying Wei
5. Tongliang Liu
6. Junzhou Huang
7. Mingming Gong
8. Ya Li

In Computer Vision and Pattern Recognition

1. Qiang Yang
2. Dacheng Tao
3. Tongliang Liu
4. Yu Zhang
5. Ying Wei
6. Junzhou Huang
7. Mingming Gong
8. Xinmei Tian

Citations in the corpus

• Domain Generalization via Conditional Invariant Representations (2018, background): “Domain adaptation is a related problem which has been extensively studied in the literature (Baktashmotlagh et al. 2013; Huang et al. 2007; Pan et al. 2011; Long et al. 2017; Shao, Kit, and Fu 2014; Shao et al. 2016; Luo et al. 2017; Liu, Yang, and Tao 2017; ?”
• Transfer Learning via Learning to Transfer (2018, methodology): “Our proof generalizes the proof in (Liu et al., 2017) in which the authors proved the uniform stability of selftaught learning algorithms.”