Topology-Preserving Class-Incremental Learning

By Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, Yihong Gong. 2020

203 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Data Space iCaRL [356], GSS [17], Mnemonics [272], TPCIL [403], GDumb [340], DER++ [46], RMM [271], HAL [63], MRDC [431], CSI [217], X-DER [41]”
RMM: Reinforced Memory Management for Class-Incremental Learning (2023, background): “The key idea is to enforce predicted label logits [29, 40], features maps [13, 18], or the topology in the feature space [49] of the new model to be close to that of the previous model.”
Few-Shot Continual Active Learning by a Robot (2022, background): “A few works have been proposed for FSCIL, to continually learn from a few-examples per class using limited memory [10, 9, 32, 33].”
S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, methodology): “Distillation-based methods (e.g., [40, 55, 22, 5, 67, 70, 62, 44, 49, 11]) apply the technique of knowledge distillation [21] to mitigate catastrophic forgetting.”
Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Data (2022, methodology): “For ImageNetSubset, we use the original RseNet-18 as our backbone following prior works [6,12,18,28,32,33].”
General Incremental Learning with Domain-aware Categorical Representations (2022, methodology): “The second are the distillation-based methods [4, 9, 13, 28, 30, 35], which adopt knowledge distillation to preserve the representation by penalizing the difference between outputs of previous and current models.”
Class-Incremental Learning with Strong Pre-trained Models (2022, background): “For the first group, the general approach for incremental training is to impose constraints on parameter changes [3, 13, 32], gradients [17], features [8,11,25], or activations [2,6,10,15,23] when finetuning the network with novel samples.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “Some methods [43,48,56,66,67,80,95] play regularization on model parameters to improve its stability.”
Memory Replay with Data Compression for Continual Learning (2022, positive): “…performance in class-incremental learning (Liu et al., 2021a; Hu et al., 2021) and in numerous other continual learning scenarios, such as audio tasks (Ehret et al., 2020), few-shot (Tao et al., 2020b), semi-supervised (Wang et al., 2021a), and unsupervised continual learning (Khare et al., 2021).”
Subspace Regularizers for Few-Shot Class Incremental Learning (2021, methodology): “As noted by past work, a more realistic evaluation of models’ ability to rapidly acquire new concepts should consider their ability to discriminate among both new concepts and old ones, a problem usually referred to as few-shot class-incremental learning (FSCIL) 2 (Tao et al., 2020b).”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “Tao et al. [118] proposed Topology-Preserving Class Incremental Learning (TPCIL) to store an Elastic Hebbian Graph (EHG) instead of previous data instances in the buffer.”
Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning (2021, neutral): “PODNet and TPCIL [6, 26] replace Recently, TOPIC [27] presents a challenging but practical FSCIL problem.”
Few-Shot Class-Incremental Learning via Relation Knowledge Distillation (2021, methodology): “In the C-way K-shot FSCIL setting (Tao et al. 2020b; Zhao et al. 2020), the number of classes in C ( t ) is C and the number of training samples per class is K for each X ( t ) ; t > 1 .”
Layerwise Optimization by Gradient Decomposition for Continual Learning (2021, neutral): “We compare our approach with other competing methods, including LWF [29], iCarl [38], DR [17], End2End [5], LUCR 2 [18], GD [26], MUC [11] and TPL [51].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…Aljundi, Lin, Goujaud, & Bengio, 2019; He, Mao, Shao, & Zhu, 2020; Zhao, Xiao, Gan, Zhang, & Xia, 2020; Kurle, Cseke, Klushyn, van der Smagt, & Günnemann, 2020; Chrysakis & Moens, 2020; Kim, Jeong, & Kim, 2020; Tao, Chang, Hong, Wei, & Gong, 2020; Douil-lard, Cord, Ollion, Robert, & Valle, 2020).”
DER: Dynamically Expandable Representation for Class Incremental Learning (2021, methodology): “TPCIL [34] makes the model preserve the topology of CNN’s feature space.”
Self-Supervised Training Enhances Online Continual Learning (2021, methodology): “All of these models follow a similar training paradigm where they are first pretrained on a subset of 100 [55, 10, 68, 28] or 500 [32, 21, 64] ImageNet classes in an offline setting, where they can shuffle and loop over the data as much as necessary.”
Distilling Causal Effect of Data in Class-Incremental Learning (2021, background): “45% accuracy drop in their ablation study on CIFAR-100 [42].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “Memory-based techniques store a subset of samples from previous tasks for either replay while training on new task [34, 17, 18] or regularization purpose [44, 13, 45].”