Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma

By R. M. French. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “Lifelong learning represents a long-standing challenge for machine learning and neural network systems (French, 1999; Hassabis et al., 2017). This is due to the tendency of learning models to catastrophically forget existing knowledge when learning from novel observations (Thrun & Mitchell, 1995). A”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “Early computational attempts include French (1997) who developed a dual-memory framework using pseudorehearsal (Robins, 1995) to transfer memories, i.e., the training samples are not explicitly kept in memory but drawn from a probabilistic model.”
FearNet: Brain-Inspired Model for Incremental Learning (2017, methodology): “GeppNet is the most recent DNN to be based on this theory, but it was also independently explored in the 1990s in French (1997) and Ans & Rousset (1997).”
Deep Generative Dual Memory Network for Continual Learning (2017, background): “pseudopattern rehearsal (Robins, 1995; French, 1997; Robins, 2004) propose to preserve neural networks’ learnt mappings by arbitrarily sampling random inputs and their corresponding outputs from the neural networks and using them along with new task samples while training.”
Measuring Catastrophic Forgetting in Neural Networks (2017, background): “Early work used fast (hippocampal) and slow (cortical) training networks to separate pattern-processing areas, and they passed pseudopatterns back and forth to consolidate recent and remote memories (French 1997).”
The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects (2013, background): “Inorder to prevent catastrophic forgetting, various researchers have suggested using a dual-memory system which, fundamentally, simulates the presence of a short-term and a long-term memory (Robins, 1995; Ans and Rousset, 1997; French, 1997; Mermillod et al., 2003).”
Continual Learning via Local Module Composition (2021, background): “CL is often modeled as a trade-off between knowledge retention (stability) and knowledge expansion (plasticity) [26, 64].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “While there have been several models inspired by CLS theory (Gepperth & Karaoguz, 2016; French, 1997; Ans & Rousset, 1997; Kemker & Kanan, 2018; Robins, 1995), many existing replay approaches have only focused on modeling medial prefrontal cortex directly and do not have a fast learning network.”
Continual learning: A comparative study on how to defy forgetting in classification tasks (2019, background): “Several directions have been explored, such as reducing representation overlap [22], [34], [35], [36], [37], replaying samples or virtual samples from the past [10], [24] or introducing dual architectures [38], [39], [40].”
A Strategy for an Uncompromising Incremental Learner (2017, background): “Neuro-biological un-derpinnings of this work was also studied by French et. al, [2].”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “For now, we will start with a typical categorization of existing deep continual works into the three categories of regularization, rehearsal and architectural approaches, in consistency ∙ LWF (LiandHoiem,2016) ∙ EBLL (Rannenetal.,2017) ∙ Pseudorehearsal (Robins,1995) ∙ Pseudo-recurrentnets (French,1”
A biologically inspired dual-network memory model for reduction of catastrophic forgetting (2014, methodology): “(1) is a form of Hebbian learning, the proposed hippocampal network acquires new patterns much more rapidly in comparison with the conventional ones learned by the backpropagation algorithm [2,3].”
The Importance of Long-term Memory in Infant Perceptual Categorization (2003, methodology): “Our own research has focused on developing a computational model of early infant categorization and testing that model empirically (French, Mermillod, Quinn, & Mareschal, 2001; Mareschal, & French, 1997; Mareschal, French, & Quinn, 2000; Mareschal, Quinn, & French, 2002). Quinn, Eimas, and Rosenkran”
The consolidation of learning during sleep: comparing the pseudorehearsal and unlearning accounts (1999, methodology): “…various issues—these include: McCloskey and Cohen (1989), Hetherington and Seidenberg (1989), Ratcliff (1990), Lewandowsky (1991), Murre (1992a), French (1992, 1994, 1997), McRae and Hetherington (1993), Lewandowsky and Li (1995), Sharkey and Sharkey (1995), Robins (1995, 1996), Frean and…”
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting (2004, background): “One answer is to generate them on the move in a separate network (Ans and Rousset 1997, 2000, French 1997).”
Continual Learning with Gated Incremental Memories for sequential data processing (2020, background): “CL literature mostly focuses on computer vision and reinforcement learning applications, with approaches ranging from regularization methods [11, 16], to dual models [17, 18], to dynamic architectures [19, 20].”
Online Task-free Continual Learning with Dynamic Sparse Distributed Memory (2022, background): “…networks in which the old information was transferred to a second, independent network and then inter-leaved with the new information being learned [16, 4], creating an approximation of the error surface of the previously learned patterns and integrating that error surface with the error surface…”
Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting (2000, methodology): “These two points were also addressed independently by Frean and Robins (1997), French (1997), Robins (1997a, b) and Robins and McCallum (1998).”