Computational principles of synaptic memory consolidation

By Marcus K. Benna, Stefano Fusi. 2016

246 citations recorded by the source corpus.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

In Neuroscience

Citations in the corpus

Overcoming catastrophic forgetting in neural networks (2016, background): “Together, these experimental ﬁndings—together with neurobiological models [Fusi et al., 2005, Benna and Fusi, 2016]—suggest that continual learning in the mammalian neocortex relies on a process of task-speciﬁc synaptic consolidation, whereby knowledge about how to perform a previously acquired…”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “nspired by theoretical neuroscience models suggesting that consolidated knowledge can be protected from interference through synapses with a cascade of states yielding different levels of plasticity (Benna and Fusi, 2016; Fusi et al., 2005), which is typically modelled with additional regularization”
Experience Replay for Continual Learning (2018, background): “This process complements local consolidation of past experience at the level of individual neurons and synapses, which is also believed to be present in biological neural networks [2, 14].”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “This is inspired by neuroscience findings suggesting that consolidated knowledge can be protected from interference via changing levels of synaptic plasticity (Benna and Fusi, 2016) and is typically modeled in terms of adding regularization terms that penalize changes in the mapping function of a ne”
Deep Generative Dual Memory Network for Continual Learning (2017, background): “Weight consolidation or freezing : Recent evidence from neuroscience also suggests that mammalian brain protects knowledge in the neocortex via task-speciﬁc consolidation of neural synapses over long periods of time (Yang et al., 2014; Benna & Fusi, 2016).”
Synaptic weight decay with selective consolidation enables fast learning without catastrophic forgetting (2019, methodology): “The synaptic cascade model (Fusi et al., 2005; Benna and Fusi, 2016; Kaplanis et al., 2018) postulates hidden states that, diﬀerent from our model, do not contribute to the eﬀective synaptic strength.”
Policy Consolidation for Continual Reinforcement Learning (2019, methodology): “The synaptic model from (Benna & Fusi, 2016) was originally described by analogy to a chain of communicating beakers of liquid (Figure 1a).”
Continual Reinforcement Learning in 3D Non-stationary Environments (2019, methodology): “Secondly by integrating more accurate synaptic plasticity models as shown in [19, 5] and going beyond mere consolidation processes which tend to quickly saturate the model learning capacity.”
Improved multitask learning through synaptic intelligence (2017, background): “While this complexity has been surmised to serve memory consolidation (Fusi et al., 2005; Lahiri & Ganguli, 2013; Benna & Fusi, 2016), few studies have illustrated how the added synaptic complexity beneﬁts learning in neural network models in a supervised setting across multiple tasks.”
The hardware lottery (2020, background): “Our brains are able to ﬁne-tune and retain skills across our lifetime (Benna & Fusi, 2016; Bremner et al., 2013; Stein et al., 2004; Tani & Press, 2016; Gallistel & King, 2009; Tulving, 2002; Barnett & Ceci, 2002).”
SupportNet: solving catastrophic forgetting in class incremental learning with support data (2018, background): “, 2016), which is inspired by the plasticity theory (Benna and Fusi, 2016).”
Switching between internal and external modes: A multiscale learning principle (2017, background): “There are many biological and circuit mechanisms that can store single-exposure information across minutes and hours (Barak & Tsodyks, 2014; Benna & Fusi, 2016; Reinartz, Biro, Gal, Giugliano, & Marom, 2014), but it is an important question to consider which of these mechanisms could support…”
Attention-Based Structural-Plasticity (2019, unknown): “learning for memory consolidation and experience replay. Interestingly, all three strategies have roots in biology. The ﬁrst strategy is inspired by synaptic consolidation in the mammalian neocortex (Benna and Fusi, 2016) where knowledge from a previously acquired task is encoded in a Notation Repre”
CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition (2020, unknown): “Current strategies such as replay of old samples [15, 29] and regularization [3, 14] can be deployed to mitigate this problem.”