A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems

By Alexander Rainer Tassilo Gepperth, Cem Karaoguz. 2016

160 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Meta & Continual Learning

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “…[Hashimoto et al., 2016], incrementally matching modes [Lee et al., 2017], rehearsing on old tasks [Robins, 1995], using adaptive memory buffers [Gepperth and Karaoguz, 2016], finding task-specific paths through networks [Fernando et al., 2017], and packing new tasks into already trained…”
Continual Lifelong Learning with Neural Networks: A Review (2018, methodology): “, 2017), the GeppNet and GeppNet+STM (Gepperth and Karaoguz, 2015), and the FEL (Coop et al.”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “More recently, Gepperth and Karaoguz (2015) proposed two approaches for incremental learning using a modiﬁed self-organizing map (SOM) and a SOM extended with a short-term memory (STM).”
FearNet: Brain-Inspired Model for Incremental Learning (2017, background): “Gepperth & Karaoguz (2016) introduced a new approach for incremental learning, which we call Gepp-Net.”
Measuring Catastrophic Forgetting in Neural Networks (2017, background): “GeppNet stores previous all training examples and then replays them during incremental learning stages [3].”
Memory Efficient Experience Replay for Streaming Learning (2018, methodology): “two networks [26], [40]: one that learns fast inspired by the hippocampus and one that learns slowly inspired by the neocortex [59].”
A Theoretical Study on Solving Continual Learning (2022, background): “Some systems learn to generate pseudo training data of old tasks to be used in replay, called pseudo-replay [41, 42, 43, 44, 45, 46, 21, 47, 48].”
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions (2022, background): “[13] trains a deep generative model (a.k.a. GAN) to simulate past dataset for future use.”
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector (2021, neutral): “Some algorithms store a subset of previous data, e.g., iCaRL [33] and GeppNet [15].”
On Learning the Geodesic Path for Incremental Learning (2021, methodology): “In order to achieve the equilibrium performance between learning new tasks and maintaining existing base performance, some IL methods store observed tasks in the memory and replay them [10, 25] to prevent catastrophic forgetting.”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “While there have been several models inspired by CLS theory (Gepperth & Karaoguz, 2016; French, 1997; Ans & Rousset, 1997; Kemker & Kanan, 2018; Robins, 1995), many existing replay approaches have only focused on modeling medial prefrontal cortex directly and do not have a fast learning network.”
iTAML: An Incremental Task-Agnostic Meta-learning Approach (2020, background): “, dynamic networks [22, 21], dualmemory modules [7], and network expansion [25].”
Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control (2019, neutral): “There have been several attempts to achieve a similar effect using only a limited portion of the previous data, (Gepperth and Karaoguz 2016; Lopez-Paz 2017; Nguyen et al. 2017) or none at all.”
Continual learning: A comparative study on how to defy forgetting in classification tasks (2019, methodology): “Further, Kemker et al. [31] compare three methods, EWC [28], PathNet [32] and replay based method GeppNet [17].”
Continual Learning for Robotics (2019, background): “The use of dual architecture is explicit in many bio-inspired approaches such as (Furlanello et al., 2016; Gepperth and Karaoguz, 2016; Parisi et al., 2018; Sprechmann et al., 2018; Kemker and Kanan, 2018).”
A comprehensive, application-oriented study of catastrophic forgetting in DNNs (2019, methodology): “Principal recent approaches to avoid CF include ensemble methods (Ren et al., 2017; Fernando et al., 2017), dual-memory systems (Shin et al., 2017; Kemker & Kanan, 2017; Rebufﬁ et al., 2017; Gepperth & Karaoguz, 2016) and regularization approaches.”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “Exemplar Rehearsal: GeppNet (Gepperth and Karaoguz, 2016) explores the use of a dual-memory system that implements various short and long-term memory storages.”
An Adaptive Random Path Selection Approach for Incremental Learning. (2019, background): “The combination of episodic (short-term) and semantic (long-term) memory was studied in [15], [16], [17] to perform memory consolidation and retrieval.”
Marginal Replay vs Conditional Replay for Continual Learning (2018, background): “3, 6], dual-memory systems [15, 32, 8], distillation-based approaches [39, 24, 17] and generative replay methods [38, 15, 22, 14, 42].”
CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition (2020, unknown): “Current strategies such as replay of old samples [15, 29] and regularization [3, 14] can be deployed to mitigate this problem.”