DualNet: Continual Learning, Fast and Slow Supplementary Materials

By Quang Pham, Chenghao Liu, Steven C. H. Hoi. 2021

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “In addition to an earlier work on obtaining sparse representations through meta-training [185], recent works have attempted to incorporate the advantages of selfsupervised learning (SSL) [125], [281], [335] and large-scale pre-training [295], [380], [456] to improve the representations in initializa”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, positive): “While rehearsal strategies are by far the most frequently adopted thanks to their effectiveness and flexibility [26, 4], it is not uncommon to adopt solutions combining multiple approaches [32, 1, 15, 60, 18].”
Learning to Prompt for Continual Learning (2021, methodology): “Lastly, recent work of CTN [45] and DualNet [44] start to consider knowledge management via a controller that models task-level information in addition to a back-bone model.”
Continual Learning, Fast and Slow (2021, methodology): “Lastly, by design, the original DualNet [22] utilizes all slow features to learn the current sample.”
Class-Incremental Learning: A Survey (2023, background): “[ 65], [67] maintain a dual-branch network for class-incremental learning, one for fast adaptation and one for slow adaption.”
Deep Class-Incremental Learning: A Survey (2023, background): “Backbone Expansion [20], [21], [22], [67], [68], [69], [70], [71]”
A Unified and General Framework for Continual Learning (2024, background): “Representative works include (Lopez-Paz & Ranzato, 2017; Riemer et al., 2019; Chaudhry et al., 2019c; Buzzega et al., 2020; Pham et al., 2021; Arani et al., 2022; Caccia et al., 2022; Wang et al., 2022b;a; 2023c;a; Yang et al., 2023b).”
Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation (2024, methodology): “• DualNet (Pham et al., 2021).”
CaSpeR: Latent Spectral Regularization for Continual Learning (2023, background): “, 2022) or the outright removal of it, by applying representation learning instead (Cha et al., 2021; Pham et al., 2021).”