A Survey on Deep Transfer Learning
By Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, Chunfang Liu. 2018
2,931 citations recorded by the source corpus.
As a new classification platform, deep learning has recently received increasing attention from researchers and has been successfully applied to many domains. In some domains, like bioinformatics and robotics, it is very difficult to construct a large-scale well-annotated dataset due to the expense of data acquisition and costly annotation, which limits its development. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed (i.i.d.) with the test data, which motivates us to use transfer learning to solve the problem of insufficient training data. This survey focuses on reviewing the current researches of transfer learning by using deep neural network and its applications. We defined deep transfer learning, category and review the recent research works based on the techniques used in deep transfer learning.
Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Quoc V. Le
- Mohit Bansal
- Percy Liang
- Sergey Levine
- Trevor Darrell
- Jianfeng Gao
- Dacheng Tao
In Representation & Self-Supervised Learning
- Mingsheng Long
- Jianmin Wang
- Trevor Darrell
- Andrea Vedaldi
- Andrew Zisserman
- Neil Houlsby
- Zhangjie Cao
- Xiaohua Zhai
In Meta & Continual Learning
- Piyush Rai
- Razvan Pascanu
- Mingsheng Long
- Zhangjie Cao
- Trevor Darrell
- Jianmin Wang
- Andrea Vedaldi
- Pravendra Singh
Citations in the corpus
- LiT: Zero-Shot Transfer with Locked-image text Tuning (2021, background): “This work is closely related to a vast amount of literature on transfer learning in vision [44, 58].”
- SOCRATES: Introducing Depth in Visual Wildlife Monitoring Using Stereo Vision (2022, background): “This requirement of large annotated training datasets is slightly relaxed by transfer learning [16].”
- A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “nagement, we only summarize the high-level ideas based on surveys [172], [173]. There are studies of transfer learning techniques in the context of NLP [176], Computer Vision [177], and deep learning [184] as well. An early survey of transfer learning [172] identifies three main research issues in tr”
- FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning (2022, methodology): “A few EFCIL methods [1, 6] are inspired by transfer learning [37, 39].”
- Disentangling Transfer in Continual Reinforcement Learning (2022, background): “General surveys of transfer learning techniques are provided in [51, 40].”
- Scalable Transfer Learning with Expert Models (2020, background): “, 2009) and empirical performance (Dai et al., 2007; Donahue et al., 2014; Oquab et al., 2014; Tan et al., 2018).”
- Language Conditioned Imitation Learning Over Unstructured Data (2020, background): “When one of the training sources is plentiful and the other scarce, multicontext learning can be seen as transfer learning [45] through a shared goal space.”
- Grounding Language in Play (2020, background): “When one of the training sources is plentiful and the other scarce, multicontext learning can be seen as transfer learning (Tan et al., 2018) through a shared goal space.”
- A Decade Survey of Transfer Learning (2010–2020) (2020, background): “Arecent survey[7]focusesonTLwithdeeplearning.”
- VaLaR NMT : Vastly Lacking Resources Neural Machine Translation (2019, background): “Transfer Learning Transfer learning has proved to be effective in various fields [14].”
- Applications of Unsupervised Deep Transfer Learning to Intelligent Fault Diagnosis: A Survey and Comparative Study (2019, background): “According to Tan et al. [22], label-consistent UDTL can be classified into four categories: network-based, instanced-based, mapping-based, and adversarial-based methods from a methodological level.”
- One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers (2019, background): “Transfer learning has been studied extensively with many studies aiming to transfer learned representations from one dataset to another [17, 31, 34, 38].”
- Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both? (2019, methodology): “Here, following the taxonomy by the recent tutorial [148], we categorize transfer learning techniques into four main classes.”
- Towards Learning a Universal Non-Semantic Representation of Speech (2020, methodology): “This strategy has been used successfully in many application domains [13, 14].”