Transferring Knowledge across Learning Processes

By Sebastian Flennerhag, Pablo G. Moreno, Neil D. Lawrence, Andreas C. Damianou. 2018

65 citations recorded by the source corpus.

In complex transfer learning scenarios new tasks might not be tightly linked to previous tasks. Approaches that transfer information contained only in the final parameters of a source model will therefore struggle. Instead, transfer learning at a higher level of abstraction is needed. We propose Leap, a framework that achieves this by transferring knowledge across learning processes. We associate each task with a manifold on which the training process travels from initialization to final parameters and construct a meta-learning objective that minimizes the expected length of this path. Our framework leverages only information obtained during training and can be computed on the fly at negligible cost. We demonstrate that our framework outperforms competing methods, both in meta-learning and transfer learning, on a set of computer vision tasks. Finally, we demonstrate that Leap can transfer knowledge across learning processes in demanding reinforcement learning environments (Atari) that involve millions of gradient steps.

Subfields: Developmental and Educational Psychology, Education

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Developmental and Educational Psychology

In Education

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “There have been a number of recent improvements on this topic of meta-optimization in multi-task RL (Nichol and Schulman, 2018; Kim et al., 2018; Rothfuss et al., 2018; Flennerhag et al., 2018; Nagabandi et al., 2018; Mendonca et al., 2019; Finn et al., 2019; Lin et al., 2020; Berseth et al., 2021; ”
Knowledge Distillation: A Survey (2020, methodology): “b; Liu et al., 2019d; Flennerhag et al., 2019) adopt meta-learning. Jang et al. (2019) designed meta-transfernetworksthat can determine what and where to transfer in the teacher-student architecture. Flennerhag et al. (2019) proposed a light-weight framework called Leap for meta-learningovertaskmani”
Meta-Learning in Neural Networks: A Survey (2020, background): “Long horizon meta-learning can also be achieved by learning an initialization that minimizes the gradient descent trajectory length over task manifolds [298].”
Meta-Learning with Warped Gradient Descent (2019, methodology): “We use the same convolutional neural network architecture and hyper-parameters as in Flennerhag et al. (2019).”
Hierarchically Structured Meta-learning (2019, background): “Most of the gradient-based meta-learning algorithms (Finn et al., 2017; Li et al., 2017; Flennerhag et al., 2018) assume a globally shared initialization across all tasks, as shown in Figure 1a.”
On Modulating the Gradient for Meta-learning (2020, methodology): “Following the multi-shot setting in [10], we assess the rate of convergence and the performance of ModGrad in comparison to the LEAP [10], Reptile [26], and FOMAML [9] for a 20-way problem with 25 tasks in total (see Fig.”
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, methodology): “Prior knowledge about learning procedure: [54], [98] tackle the second type of prior knowledge, namely learning procedure.”
Bootstrapped Meta-Learning (2021, methodology): “Similarly, optimizing the learner’s performance after K updates can fail to account for the process of learning, causing another form of myopia (Flennerhag et al., 2019; Stadie et al., 2018; Chen et al., 2016; Cao et al., 2019).”
Explaining Knowledge Distillation by Quantifying the Knowledge (2020, unknown): “We utilize ∑m k=1 ‖wk−wk−1‖ ‖w0‖ , named “weight distance” , to measure the learning effect at m-th epoch [12, 7].”
Empirical Bayes Transductive Meta-Learning with Synthetic Gradients (2020, background): “The situation is similar in gradient based meta-learning methods (Finn et al., 2017; Ravi & Larochelle, 2016; Li et al., 2017b; Nichol et al., 2018; Flennerhag et al., 2019) and many other meta-learning methods (Vinyals et al.”