Transfer of Learning

By Deborah A. Maranville. 2015

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “…the domain of G θ ’s pretraining data, with a potentially different distribution from the downstream task; straightforward application of supervised training will result in overﬁtting to the pretraining domain and diminishing generalization ability, a common challenge in transfer learning [64, 87].”
AdapterDrop: On the Efficiency of Adapters in Transformers (2020, methodology): “While transfer learning has become the go-to method for solving NLP tasks (Pan and Yang, 2010; Torrey and Shavlik, 2010; Ruder, 2019; Howard and Ruder, 2018; Peters et al., 2018), transformerbased models are notoriously deep requiring millions or even billions of parameters (Radford et al.”
AdapterFusion: Non-Destructive Task Composition for Transfer Learning (2020, background): “In this section, we formalize our goal of transfer learning (Pan and Yang, 2010; Torrey and Shavlik, 2010; Ruder, 2019), highlight its key challenges, and provide a brief overview of common methods that can be used to address them.”
Real-Time Federated Evolutionary Neural Architecture Search (2020, background): “Second, many NAS techniques for accelerating model training [9], [11], [13]–[15], [17] adopt transfer learning techniques [18], [19] by searching upon cell based small models and transferring the found promising cell structures to large models.”
Learngene: From Open-World to Your Learning Task (2021, background): “There are currently some learning paradigms that can solve some of the above problems, such as continual learning [45], transfer learning [47] and meta-learning [25].”
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021, background): “As a result, Continual Learning (CL) (Ring, 1998; Thrun, 1998) has received more attention recently as it can enable models to perform positive transfer (Perkins et al., 1992) as well as remember previously seen tasks.”
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability (2017, background): “Taylor & Stone (2009) and Torrey & Shavlik (2009) provide excellent surveys of transfer and multi-task RL, which almost exclusively target single-agent, fully-observable settings.”
A Review of Student Difficulties in Upper-Level Quantum Mechanics (2015, background): “concepts learned in other contexts, students must first learn the basic structure of quantum mechanics and then contemplate how the previously learned knowledge applies to this novel framework [26].”
Do the benefits of chess instruction transfer to academic and cognitive skills? A meta-analysis (2016, background): “In fact, transferability of skills is either a tacit assumption or a specific aim of nearly every educational program (Donovan et al., 1999; Perkins & Salomon, 1994).”
An Information-Theoretic Approach to Transferability in Task Transfer Learning (2019, background): “…where knowledge transfer is achieved by making representations learned from one input domain work on a different input domain, e.g. adapt models for RGB images to infrared images [21]; and task transfer learning , where knowledge is transferred between different tasks on the same input domain [22].”
Multi-task transfer learning deep convolutional neural network: Application to computer-aided diagnosis of breast cancer on mammograms (2017, background): “Transfer learning also has the potential to help improve the convergence rate by speeding up the target learning tasks and improve the generalization capability that will lead to better independent test performance in comparison to training DCNNs without transfer learning (Torrey and Shavlik 2009).”
Flexibility-Oriented HRM Systems, Absorptive Capacity, and Market Responsiveness and Firm Innovativeness (2013, background): “…(i.e., internal knowledge related to new external knowledge) is among the most important antecedents of learning performance, facilitating knowledge mastery or acquisition at both the individual level (e.g., ellis, 1965) and the firm level (e.g., Lane & Lubatkin, 1998; Lane, Salk, & Lyles, 2001).”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “Seminal surveys in machine learning on transfer learning include Pan and Yang (2009), Weiss et al. (2016), and Yang et al. (2020).”
Meta-Learning Requires Meta-Augmentation (2020, background): “The goal is to learn how to learn new tasks from a small number of examples, by leveraging knowledge from learning other tasks [5, 7, 14, 30, 33].”
Transfer learning using VGG-16 with Deep Convolutional Neural Network for Classifying Images (2019, background): “The study of transfer learning is motivated by the fact that people can intelligently apply knowledge learned previously to solve new problems faster or with better solutions [1].”