On the role of neurogenesis in overcoming catastrophic forgetting

By German Ignacio Parisi, Xu Ji, Stefan Wermter. 2018

15 citations recorded by the source corpus.

Lifelong learning capabilities are crucial for artificial autonomous agents operating on real-world data, which is typically non-stationary and temporally correlated. In this work, we demonstrate that dynamically grown networks outperform static networks in incremental learning scenarios, even when bounded by the same amount of memory in both cases. Learning is unsupervised in our models, a condition that additionally makes training more challenging whilst increasing the realism of the study, since humans are able to learn without dense manual annotation. Our results on artificial neural networks reinforce that structural plasticity constitutes effective prevention against catastrophic forgetting in non-stationary environments, as well as empirically supporting the importance of neurogenesis in the mammalian brain.

Subfields: Meta & Continual Learning, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

1. Sen Lin
2. Li Yang
3. Deliang Fan
4. Junshan Zhang
5. Prashant Bhat
6. Bahram Zonooz
7. Elahe Arani

Citations in the corpus

• Compacting, Picking and Growing for Unforgetting Continual Learning (2019, background): “Dynamic architecture: Dynamic-architecture approaches [38, 20, 36, 29, 48] adapt the architecture with a sequence of tasks.”
• Online Continual Learning on Sequences (2020, unknown): “lected as the BMU or its neighbor). Empirical studies have shown that Gamma-GWR networks with additive neurogenesis show a better performance than a static GWR network with the same number of neurons [59]. While the mechanisms of structural plasticity in the Gamma-GWR do not resemble biologically pl”