Continual Learning through Evolvable Neural Turing Machines

By Benno Lüders, Mikkel Schläger, Sebastian Risi. 2016

Subfields: Model Architectures, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

1. Ang Li
2. Mehrdad Farajtabar
3. Navid Azizan
4. Dong Yin
5. Alex Mott

In Meta & Continual Learning

1. Mehrdad Farajtabar
2. Dong Yin
3. Ang Li
4. Navid Azizan
5. Alex Mott
6. Seyed Iman Mirzadeh
7. Dilan Gorur
8. Razvan Pascanu

Citations in the corpus

• Orthogonal Gradient Descent for Continual Learning (2019, methodology): “L¨uders et al. (2016) used a Neural Turing Machine that enables agents to store long-term memories by progressively employing additional memory components.”
• SOLA: Continual Learning with Second-Order Loss Approximation (2020, background): “cess. The repetition and memory based methods store previous data or, alternatively, train a generative model of them and replay samples from them interleaved with samples drawn from the current task [41, 19, 48, 38, 27, 26, 8]. They achieve promising performance however at the cost of higher risk o”
• Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint. (2020, methodology): “The replay-based methods store previous data or, alternatively, train a generative model of them and replay samples from them interleaved with samples drawn from the current task (Shin et al., 2017; Kamra et al., 2017; Zhang et al., 2019; Rios and Itti, 2018; Lüders et al., 2016; Lopez-Paz and Ranza”