Generative Knowledge Distillation for General Purpose Function Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference (2018, methodology): “This procedure closely follows recent enhancements discussed in (Zhang & Sutton, 2017; Riemer et al., 2017b;a) The sample function randomly samples k − 1 examples from the memory buffer M and interleaves them with the current example to form a single size k batch.”
A Scalable Approach to Multi-Context Continual Learning via Lifelong Skill Encoding (2018, methodology): “Our work is perhaps most similar to Lifelong Generative Modeling [7] and Generative Knowledge Distillation [8]; both approaches use a student-teacher model, in which the current network first generates synthetic samples that match previously seen data and then trains on both the new and synthetic da”
On the Role of Weight Sharing During Deep Option Learning (2019, background): “While naive experience replay may suffer from issues related to the differences in the policy over time and issues with scaling storage, these problems have been addressed in the literature by leveraging importance sampling (Precup 2000a; Peshkin and Shelton 2002; Jie and Abbeel 2010; Mahmood, van H”
Learning Abstract Options (2018, background): “This a somewhat arbitrary choice of the extent of weight sharing in light of recent work that focuses on more dynamic sharing patterns in multi-task learning, lifelong learning, and continual learning settings [29, 23, 32, 34, 6, 33, 10, 17, 30, 31].”