Reply to Huszár: The elastic weight consolidation penalty is empirically valid
By James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska and 4 more. 2018
16 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Materials Chemistry
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Vincenzo Lomonaco
- Davide Maltoni
- Christopher Kanan
- Tyler L. Hayes
- Aleksandar Botev
- Hippolyt Ritter
- David Barber
- Feng Zhu
Citations in the corpus
- Continuous Learning in Single-Incremental-Task Scenarios (2018, methodology): “A single set of optimal weights Θ is also maintained by using the most recent ones (Θ = Θ) since Θ already incorporates constraints from all previous batches (refer to the discussion in [7][13]).”
- Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting (2018, background): “However, it continues to add a penalty for every new task [17].”
- Efficient Continual Learning in Neural Networks with Embedding Regularization (2019, methodology): “When training on more than two tasks, different penalties can be combined by storing the mean vectors and Fisher diagonal matrices for each task [13], or a single composite penalty can be obtained in various ways [18, 10, 14].”