The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects

By Martial Mermillod, Aurélia Bugaiska, Patrick Bonin. 2013

546 citations recorded by the source corpus.

The stability-plasticity dilemma is a well-know constraint for artificial and biological neural systems. The basic idea is that learning in a parallel and distributed system requires plasticity for the integration of new knowledge, but also stability in order to prevent the forgetting of previous knowledge. Too much plasticity will result in previously encoded data being constantly forgotten, whereas too much stability will impede the efficient coding of this data at the level of the synapses. However, for the most part, neural computation has addressed the problems related to excessive plasticity or excessive stability as two different fields in the literature.

Subfields: Cognitive Neuroscience, Developmental and Educational Psychology, Model Architectures, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Developmental and Educational Psychology

In Model Architectures

In Meta & Continual Learning

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “The stability–plasticity dilemma regards the extent to which a systemmust be prone to integrate and adapt to new knowledge and, importantly, how this adaptation process should be compensated by internalmechanisms that stabilize andmodulate neural activity to prevent catastrophic forgetting (Ditzler ”
On the role of neurogenesis in overcoming catastrophic forgetting (2018, background): “ was also able to retain newly trained classes faster, which is particularly interesting given that the stability-plasticity dilemma suggests lower retention rates are required to mitigate forgetting [4], as fast retention contravenes representation stability. 2 Growing Self-Organizing Memory Neurog”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “The extent to which a system must be flexible for learning novel knowledge and stable for preventing the disruption of consolidated knowledge is known as the stabilityplasticity dilemma, which has been extensively studied for both computational and biological systems (e.g., Grossberg, 1980, 2007; Me”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, background): “The menace of catastrophic forgetting occurs due to the stability-plasticity dilemma: the extent to which the system must be stable to retain consolidated knowledge and be plastic to assimilate new information (Mermillod et al., 2013).”
FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning (2022, background): “Their objective is to ensure a good balance between plasticity, i.e. integration of new information, and stability, i.e. preservation of knowledge about past classes [27].”
Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer (2022, background): “A fundamental challenge in incremental learning is in the stability-plasticity trade-off [37], where stability relates to maintaining accuracy on the previous tasks, while plasticity relates to learning the current task effectively.”
Online Continual Learning with Contrastive Vision Transformer (2022, background): “Continual learning requires neural networks to be both stable to prevent forgetting as well as plastic to learn new concepts, which is referred to as the stability-plasticity dilemma [55,27].”
EPicker is an exemplar-based continual learning approach for knowledge accumulation in cryoEM particle picking (2022, background): “A low-cost alternative to joint training is continual (or incremental) learning, which aims to adapt a new model for a new task while maintaining performance for old tasks 16 – 24 .”
Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “The trade-off between learning new classes and retaining old classes is called the stability-plasticity dilemma [14].”
Continual Learning via Local Module Composition (2021, background): “CL is often modeled as a trade-off between knowledge retention (stability) and knowledge expansion (plasticity) [26, 64].”
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector (2021, positive): “It is a direct cause of catastrophic forgetting problem [24], and the plasticity-stability dilemma [3, 28] is a more general problem: (1) plasticity: the deep model should learn the new knowledge of the current task, and (2) stability: it should also preserve the knowledge of previous tasks.”
Subspace Regularizers for Few-Shot Class Incremental Learning (2021, background): “By contrast, subspace regularizers enable a good balance between plasticity and stability (Mermillod et al., 2013).”
Training Networks in Null Space of Feature Covariance for Continual Learning (2021, neutral): “This phenomenon is called plasticity-stability dilemma [33].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “A more complex and general viewpoint of CL is the stabilityplasticity dilemma [3, 4], where stability refers to the ability to preserve”
Adaptive Aggregation Networks for Class-Incremental Learning (2020, background): “This memory constraint causes a serious data imbalance problem between old and new classes, and indirectly causes the main problem of CIL – the stability-plasticity dilemma [29].”
CPR: Classifier-Projection Regularization for Continual Learning (2020, background): “At the heart of catastrophic forgetting is the stability-plasticity dilemma (Carpenter & Grossberg, 1987; Mermillod et al., 2013), where a model exhibits high stability on previously trained tasks, but suffers from low plasticity for the integration of new knowledge (and vice-versa).”
Understanding the Role of Training Regimes in Continual Learning (2020, background): “This forgetting problem is closely related to the “stability-plasticity dilemma” [53], which is a common challenge for both biological and artificial neural networks.”
Dropout as an Implicit Gating Mechanism For Continual Learning (2020, background): “The stability-plasticity dilemma is a well-known problem for both artificial and biological neural networks [19].”