Maintaining stability during new learning in neural networks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “While the fast weights are used to learn new information, the slow weights can be used for generative replay (Robins, 1997, 2004).”
Sequential learning in neural networks: A review and a discussion of pseudorehearsal based methods (2004, unknown): “As originally described and implemented by [53] one weight was used as a ﬁxed base weight to generate pseudoitems, and the other as an offset which was added to the base weight and adjusted during the course of learning new items.”
Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting (2000, methodology): “These two points were also addressed independently by Frean and Robins (1997), French (1997), Robins (1997a, b) and Robins and McCallum (1998).”
The consolidation of learning during sleep: comparing the pseudorehearsal and unlearning accounts (1999, unknown): “This temporary storage of a large population is not desirable , and we suggested ( Robins , 1997 ) that it would be more biologically plausible to use a system which effectively uses two weights per connection , an " old " weight used to generate pseudoitems as they are needed and a " new " weight…”