Selfless Sequential Learning

By Rahaf Aljundi, Marcus Rohrbach, Tinne Tuytelaars. 2018

120 citations recorded by the source corpus.

Sequential learning, also called lifelong learning, studies the problem of learning tasks in a sequence with access restricted to only the data of the current task. In this paper we look at a scenario with fixed model capacity, and postulate that the learning process should not be selfish, i.e. it should account for future tasks to be added and thus leave enough capacity for them. To achieve Selfless Sequential Learning we study different regularization strategies and activation functions. We find that imposing sparsity at the level of the representation (i.e.~neuron activations) is more beneficial for sequential learning than encouraging parameter sparsity. In particular, we propose a novel regularizer, that encourages representation sparsity by means of neural inhibition. It results in few active neurons which in turn leaves more free neurons to be utilized by upcoming tasks. As neural inhibition over an entire layer can be too drastic, especially for complex tasks requiring strong representations, our regularizer only inhibits other neurons in a local neighbourhood, inspired by lateral inhibition processes in the brain. We combine our novel regularizer, with state-of-the-art lifelong learning methods that penalize changes to important previously learned parts of the network. We show that our new regularizer leads to increased sparsity which translates in consistent performance improvement %over alternative regularizers we studied on diverse datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Memory Efficient Continual Learning with Transformers (2022, methodology): “Existing methods for CL can be roughly categorized as follows: (1) Replaybased methods [36, 50, 11, 8, 62] retain some training data of old tasks and use them in learning a new task to circumvent the issue of catastrophic forgetting (CF); (2) Regularization-based methods [29, 2, 25] add a regulariza”
Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, background): “This problem is often coined as continual learning, for which three representative approaches have been proposed [57, 67, 20] including replay [52, 29, 66, 73, 70, 44], regularization [38, 91, 3], and expansion techniques [71, 89].”
Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, methodology): “The first strategy [8, 10, 11, 12, 13] consists of avoiding the modification of parameters”
Continual Learning with Fully Probabilistic Models (2021, background): “Current approaches are very diverse: SSL [13] focuses on enhancing sparsity of neural activities, whereas approaches such as LwF [14] rely on knowledge distillation mechanisms.”
Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, background): “Recently, some attention has been devoted to latent regularization in continual image classification [1, 2, 27].”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, background): “, 2018), sequential learning (McCloskey and Cohen, 1989; Shin et al., 2017; Aljundi et al., 2019), explanationbased learning (Thrun, 1996), and never-ending learning (Carlson et al.”
Embracing Change: Continual Learning in Deep Neural Networks. (2020, background): “From a continual learning perspective [56], both sparse representations and gradients will result in less interference and forgetting, as there will either be fewer task-sensitive parameters or fewer effective changes to parameters.”
Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, positive): “regularization-based) [43,94,2] and expansion-based methods [74,93], the replay-based approaches have often shown superior results in terms of performance and memory efficiency.”
CPR: Classifier-Projection Regularization for Continual Learning (2020, methodology): “In this paper, we mainly focus on the regularization-based CL methods (Li & Hoiem, 2017; Kirkpatrick et al., 2017; Aljundi et al., 2018; Chaudhry et al., 2018; Zenke et al., 2017; Nguyen et al., 2018; Ahn et al., 2019; Aljundi et al., 2019; Jung et al., 2020).”
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization (2020, methodology): “To address this dilemma, a comprehensive study for neural network-based continual learning was conducted broadly under the following categories: regularization-based [18, 13, 41, 22, 1, 3], dynamic architecture-based [27, 39, 10], and replay memory-based [26, 20, 34, 11] methods.”
A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning (2020, methodology): “Specifically, Aljundi et al. (2019a) extend MAS (Aljundi et al., 2018) by adding heuristics to determine when to update the importance weights with no task definition.”
Continual Learning with Adaptive Weights (CLAW) (2019, background): “Other algorithms based on replay mechanisms include (Aljundi et al., 2019a;b).”
Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control (2019, methodology): “In Memory Aware Synapses(MAS)(Aljundi et al. 2018), the importance value of each weight is measured by the sample mean of the absolute value of the gradient.”
Meta-Learning Representations for Continual Learning (2019, methodology): “Knowledge retention methods prevent important weights from changing too much, by introducing a regularization term for each parameter weighted by its importance (Kirkpatrick et al., 2017; Aljundi et al., 2018; Zenke et al., 2017; Lee et al., 2017; Liu et al., 2018).”
Gradient based sample selection for online continual learning (2019, methodology): “A similar setting has been used in [1].”
Learning Without Memorizing (2018, methodology): “TI methods LwF [12] IMM [10], EWC [9], MAS [1], [2], [8]”
Continual learning: A comparative study on how to defy forgetting in classification tasks (2019, background): “Continual learning is also referred to as lifelong learning [6], [7], [8], [9], [10], [11], sequential learning [12], [13], [14] or incremental learning [15], [16], [17], [18], [19], [20], [21].”
A comprehensive, application-oriented study of catastrophic forgetting in DNNs (2019, methodology): “Other regularization-oriented approaches are proposed in Aljundi et al. (2018); Srivastava et al.”
Ternary Feature Masks: continual learning without any forgetting (2020, methodology): “Model-based EWC [7] No No No 1 float pp Some Yes weights R-EWC [19] No No No 1 float pp Some Yes weights IMM [8] No No No 1 float pp pt Some Yes weights SI [20] No No No 1 float pp Some Yes weights MAS [6] No No No 1 float pp Some Yes weights SSL [18] No No No 1 float pfp Some Yes both”
La-MAML: Look-ahead Meta Learning for Continual Learning (2020, background): “This involves penalising changes to weights deemed important for old tasks [14] or enforcing weight or representational sparsity [3] to ensure that only a subset of neurons remain active at any point of time.”