Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.

By Roger Ratcliff. 1990

Subfields: Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Science & Psychology

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “Much effort has gone into mitigating catastrophic forgetting (McCloskey and Cohen, 1989; Ratcliff, 1990; Kemker et al., 2017) by penalizing the norm of parameters when training on a new task (Kirkpatrick et al.”
Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “However, continual learning by deep models has proven to be very challenging due to catastrophic forgetting , a long known problem of training deep neural networks [4, 5, 14, 15, 28, 31, 34].”
Overcoming catastrophic forgetting in neural networks (2016, background): “This phenomenon, termed catastrophic forgetting [French, 1999, McCloskey and Cohen, 1989, McClelland et al., 1995, Ratcliff, 1990], occurs speciﬁcally when the network is trained sequentially on multiple tasks because the weights in the network that are important for task A are changed to meet the…”
Fast Model Editing at Scale (2021, neutral): “, 1989; Ratcliff, 1990; Kirkpatrick et al., 2017).”
Recurrent World Models Facilitate Policy Evolution (2018, background): “While the human brain can hold decades and even centuries of memories to some resolution [2], our neural networks trained with backpropagation have more limited capacity and suffer from issues such as catastrophic forgetting [16, 43, 69].”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “, Parisi, Tani,Weber, andWermter (2017), Parisi, Tani,Weber andWermter (2018) and Richardson and Thomas (2008)).”
Continual Learning with Deep Generative Replay (2017, background): “Several works illustrate empirical consequences in sequential learning settings [7, 29], and provide a few primitive solutions [16, 30] such as replaying all previous data.”
Variational Continual Learning (2017, background): “Too much plasticity leads to the infamous catastrophic forgetting problem (McCloskey and Cohen, 1989; Ratcliff, 1990; Goodfellow et al., 2014a) and too much stability leads to an inability to adapt.”
An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks (2013, background): “Catastrophic forgetting(McCloskey & Cohen, 1989; Ratcliff, 1990) is a problem that affects neural networks, as well as other learning systems, including both biological and machine learning systems.”
World Models (2018, background): “While the human brain can hold decades and even centuries of memories to some resolution (Bartol et al., 2015), our neural networks trained with backpropagation have more limited capacity and suffer from issues such as catastrophic forgetting (Ratcliff, 1990; French, 1994; Kirkpatrick et al., 2016).”
Continuous Learning in Single-Incremental-Task Scenarios (2018, background): “Unfortunately, when (deep or shallow) neural networks are trained only on new data, they experience a rapid overriding of their weights with a phenomenon known in literature as catastrophic forgetting [22][26][2].”
Overcoming catastrophic forgetting with hard attention to the task (2018, background): “That is the case with the so-called catastrophic forgetting or catastrophic interference problem (McCloskey & Cohen, 1989; Ratcliff, 1990).”
The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects (2013, background): “Too much plasticity will result in previously encoded data being constantly forgotten, whereas too much stability will impede the efﬁcient coding of this data at the level of the synapses.”
Selective Experience Replay for Lifelong Learning (2018, background): “As a result of the non-stationary training distribution, the network loses its ability to perform well in previous tasks in what has been termed catastrophic forgetting (McCloskey and Cohen 1989; Ratcliff 1990; Goodfellow et al. 2013).”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, background): “…have been proposed in the literature that mimic one or more neurophysiological processes in the brain to address catastrophic forgetting in DNN. Experience rehearsal (Ratcliff, 1990) is one of the most prominent approaches that mimics the association of past and present experiences in the brain.”
Progressive Prompts: Continual Learning for Language Models (2023, background): “…arise in CL: (1) avoiding catastrophic forgetting , i.e., loss of the knowledge acquired from previous tasks after learning new ones (McCloskey & Cohen, 1989; Ratcliff, 1990), and (2) allowing forward transfer , i.e., leveraging the knowledge from past tasks for efﬁcient learning of new tasks.”
Class Gradient Projection For Continual Learning (2022, background): “Such a phenomenon is known as catastrophic forgetting , which is the research hotspot in Continual Learning (CL) [11, 28, 32].”