Consolidation in Neural Networks and in the Sleeping Brain

By Anthony V. Robins. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “…adversarial network (GAN) (Goodfellow et al., 2014) to generate samples from previously learned data (Draelos et al., 2017; Kemker & Kanan, 2018; Robins, 1995, 1996; Ostapenko et al., 2019; Shin, Lee, Kim, & Kim, 2017; He et al., 2018; French, 1997; Atkinson, McCane, Szymanski, & Robins, 2018).”
Meta-Consolidation for Continual Learning (2020, background): “Interestingly, many years ago, [64] used auto-associative neural networks [40] to study connections between sleep-induced consolidation and reduction in catastrophic forgetting.”
Representation Stability as a Regularizer for Improved Text Analytics Transfer Learning (2017, neutral): “One compelling technique for addressing this problem is the concept of pseudorehearsals (Robins, 1995), (Robins, 1996), where relearning is performed on an artiﬁcially constructed population of pseudoitems instead of the actual old examples.”
Pseudopatterns and dual-network memory models: Advantages and shortcomings (2000, background): “The idea is to have two networks (for simplicity we will call these two networks the “hippocampal” and the “neocortical” networks, although these designations should not be taken too literally) that exchange information by means of pseudopatterns (the neural correlate of pseudopattern information tr”
Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting (2000, methodology): “Robins (1996a, Robins and McCallum, 1998) suggested that consolidation of information in the neocortex may occur by means of pseudopatterns generated during rapid eye movements (REM) sleep phases.”
Catastrophic Forgetting and the Pseudorehearsal Solution in Hopfield-type Networks (1998, background): “Finally, we explore links between the mechanisms investigated in this thesis and the consolidation of newly learnt material during sleep. ii”
Catastrophic forgetting in simple networks: an analysis of the pseudorehearsal solution. (1999, background): “This is known as ‘rehearsal’, and obviously requires the explicit storage of the original training set—something that has its disadvantages, notably that storage of these items is often what the network itself is supposed to do! ‘Pseudorehearsal’ [17] (see also [ 18 , 6, 20]) is an alternative algor”
Towards Knowledgeable Supervised Lifelong Learning Systems (2020, background): “(Ring, 1994) (Robins, 1995) (Robins, 1996) (Ring, 1997) (Silver et al.”
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated. (2016, unknown): “…learning does not actually involve the faithful replay of previous experiences: instead, hippocampal replay of recent experiences might be interleaved with activation of cortical activity patterns consistent with the structured knowledge implicit in the neocortical network (e. g., [23,54 – 56]).”
Sequential learning in neural networks: A review and a discussion of pseudorehearsal based methods (2004, unknown): “In a different approach Robins [52] explored the links between CF and the popular theory that newly learned information is consolidated in the brain during sleep.”
The consolidation of learning during sleep: comparing the pseudorehearsal and unlearning accounts (1999, unknown): “…heteroassociative randomly constructed data sets by Robins ( 1995 ) , Ans and Rousset ( 1997 ) ; autoassociative learning of the Iris data set by Robins ( 1996 ) ; a classification task using the Mushroom data set by French ( 1997 ) ; and a structured " task domain " by Silver and Mercer ( 1998…”
Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma (1997, background): “If this interaction is indeed 18 mediated by pseudopatterns, as suggested by Robins (1996), then it would not be unreasonable to think that the representational compaction observed in the pseudo-recurrent model might also occur in the brain.”
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting (2004, background): “As for the brain correlates of the self-generated pseudopatterns, Robins and colleagues (Robins 1996, Robins and McCallum 1999) have suggested that they could be related to spontaneous cortical reactivations observed during different sleep phases, in particular rapid eye movement (REM) sleep.”