Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting

By Bernard Ans, Stephane Rousset. 2000

75 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Reinforcement Learning

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “However, continual learning by deep models has proven to be very challenging due to catastrophic forgetting , a long known problem of training deep neural networks [4, 5, 14, 15, 28, 31, 34].”
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent (2020, methodology): “Several heuristics were developed in order to characterise it (Ans & Rousset 1997, Ans & Rousset 2000, Goodfellow et al. 2014, French 1999, McCloskey & Cohen 1989, Robins 1995, Nguyen et al. 2019).”
Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills (2015, background): “Additionally, the difference between computational brain models and natural brains with respect to catastrophic forgetting limits the usefulness of such models as tools to study neurological pathologies [16].”
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting (2004, methodology): “We coupled two RSRNs to create a dual-network architecture that is similar to the model originally developed by Ans and Rousset (1997, 2000) .”
Sequential learning in neural networks: A review and a discussion of pseudorehearsal based methods (2004, unknown): “…data sets [1,51]; classiﬁcation tasks using the Mushroom and Congressional Voting Records data sets (see [44]) and a Cat and Dog picture dataset [19,20]; artiﬁcial and natural categories [24]; an artiﬁcial 2D domain [66]; a coronary artery database [65] and a domain of arithmetic facts [2].”
Pseudopatterns and dual-network memory models: Advantages and shortcomings (2000, methodology): “In a preliminary study, Ans & Rousset [2] trained the hippocampal network on a set of 20 arbitrary input-output patterns, { }201 = i i P .”
The contents of long-term memory and the emergence of knowledge (2009, background): “It is also central to connectionist models (e.g., Ans & Rousset, 1997, 2000; McClelland & Rumelhart, 1986; for a review, see, for example, Hintzman, 1990), in which the traces are by definition distributed across all the connections in the network.”
Using Noise to Compute Error Surfaces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting (2002, methodology): “bution will produce a reasonable approximation of f. (See French, Ans, & Rousset,2001,foradiscussionofhowthisapproximationcouldbeimproved by additionally making use of the values of the output associated with each random input, or Ans & Rousset, 2000, for a technique that produces an “attractor ” ”
Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting (2018, methodology): “These methods are further extended by separating the learning agent into two parts, an idea which stems from early research into CF [14], [15], [16] and is still being utilised in recent algorithms such as Deep Generative Dual Memory Network [17] for incremental image classification and others (e.”
Tracelink: A model of consolidation and amnesia (2005, background): “Moreover, our implementation of consolidation— as the strengthening of attractors surfacing out of noise—has been explored in several more theoretical simulation studies ( Ans & Rousset, 2000; Robins, 1995; see Meeter, 2003a, for more discussion of this work).”