Random Path Selection for Incremental Learning

By Jathushan Rajasegaran, Munawar Hayat, Salman Hameed Khan, Fahad Shahbaz Khan, Ling Shao. 2019

114 citations recorded by the source corpus.

Subfields: Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “PathNet [72] and RPSNet [244] pre-allocate multiple parallel networks to construct a few candidate paths, i.e., layer-wise compositions of network modules, and select the best path for each task.”
RMM: Reinforced Memory Management for Class-Incremental Learning (2023, methodology): “Incremental Learning [22, 36, 48, 53, 58] continuously updates the model using data coming in a sequence of phases.”
Long-Tailed Class Incremental Learning (2022, methodology): “Parameter-isolation methods increase model plasticity by adding more neurons, modules [31,33], branches [23] or masks [26,25,34].”
Novel Class Discovery without Forgetting (2022, background): “Dynamically expanding and parameter isolation methods [44,40,39,1,33] form a third class of methods to address forgetting.”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, methodology): “ThecodeforRPSnet:https://github.com/brjathu/RPSnet.”
Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Data (2022, methodology): “Popular structure-driven methods [2,20,21,25,32,33,35] modify the network structure or expand the network for training new tasks.”
Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning (2022, background): “[24] progressively chooses optimal paths for the new tasks while encouraging parameter sharing, which promotes the forward knowledge transfer.”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “…2018), Diffusion-based Neuromodulation (Velez and Clune, 2017), IMM (Lee et al., 2017), Expandable Networks (Yoon et al., 2018; Li et al., 2019), RPSNet (Rajasegaran et al., 2019), reinforcement learning (Kaplanis et al., 2019; Rolnick et al., 2019), and meta-learning (Javed and White, 2019).”
Continual Learning by Using Information of Each Class Holistically (2021, background): “Learning a large number of classes one by one is very challenging for most methods, i.e., EWC, LwF, IMM, PGMA and RPSnet.”
Efficient Continual Learning with Modular Networks and Task-Driven Priors (2020, background): “This is also the case for other evolving architecture approaches proposed in the literature (Li et al., 2019; Rajasegaran et al., 2019; Yoon et al., 2018; Xu & Zhu, 2018) as discussed in §2.”
Meta-Consolidation for Continual Learning (2020, methodology): “Another kind of methods [51, 72, 69, 19, 68, 62, 87] attempt to expand the network to increase the capacity of the model, while learning new tasks.”
iTAML: An Incremental Task-Agnostic Meta-learning Approach (2020, methodology): “56% RPS-net∗[21] 96.”
Continual Learning with Adaptive Weights (CLAW) (2019, background): “Indeed, catastrophic forgetting is one of the main pathologies in continual learning (McCloskey and Cohen, 1989; Ratcliff, 1990; Robins, 1993, 1995; French, 1999; Pape et al., 2011; Goodfellow et al., 2014a; Achille et al., 2018; Kemker et al., 2018; Kemker and Kanan, 2018; Diaz-Rodriguez et al., 20”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, background): “Contrary to some of the previously mentioned approaches, RPS does not need a task-ID at inference time.”
An Adaptive Random Path Selection Approach for Incremental Learning. (2019, methodology): “As an example, we consider CIFAR-100 with 10 and 20 incremental tasks, the modiﬁed RPS-Net with adaptive path selection and path attention surpasses RPS-Net [9] by 4.35% and 2.71% respectively.”
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning (2020, background): “It suggests in prior works [49,34] that LwF performs better than MAS in the context of class-incremental learning.”