Calibrating CNNs for Lifelong Learning

By Pravendra Singh, Vinay Kumar Verma, Pratik Mazumder, Lawrence Carin, Piyush Rai. 2020

78 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “For a regular network, the task-speciﬁc components could be parallel branches [233], [234], adaptive layers [58], [235], masks of intermediate features [186], [236], [237].”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, methodology): “Several works have been proposed to address the issue of scalability in PNNs: CCLL (Singh et al., 2020) employed a fixed capacity model and reused the features captured on the first task by performing spatial and channel-wise calibration for all subsequent tasks.”
A Theoretical Study on Solving Continual Learning (2022, methodology): “CCLL [57] uses task-adaptive calibration in convolution layers.”
Meta-attention for ViT-backed Continual Learning (2022, background): “The knowledge of previous tasks can be preserved by sequentially increasing new masks (weight masks [18, 30–32, 48] or unit masks [41, 43]) and masking out parameters of old tasks simultaneously.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “Some other methods [45,57,75,76,82,91] increase the capacity of the model to have a better tradeoff of stability and plasticity, but with the cost of growing memory of the network.”
Rectification-based Knowledge Retention for Continual Learning (2021, methodology): “Our method utilizes task labels during testing similar to [26, 35, 36].”
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020, methodology): “lass-mean embeddings are estimated using E. Belouadah et al.: Preprint submitted to Elsevier Page 5 of 18 A Comprehensive Study of Replay-based Class Incremental Learning Algorithms new data only. In [68], authors propose an approach that calibrates activation maps of the CNN in order to accommodate”
Efficient Feature Transformations for Discriminative and Generative Continual Learning (2021, neutral): “More details about the experimental setup and hyperparameters can be found in the supplementary material, as well as additional comparisons with SupSup [54] and CCLL [48].”
Deep Class-Incremental Learning: A Survey (2023, methodology): “CCLL [110] aims to calibrate the activation maps of old models during incremental learning.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “One example specific to convolutional layers keeps the filter parameters fixed after initial training on a single task and adapts them to each new task via spatial and channel-wise calibration (Singh et al., 2020).”
Continual Learning Based on OOD Detection and Task Masking (2022, methodology): “CCLL uses task-adaptive calibration on convolution layers.”
Mitigating Forgetting in Online Continual Learning with Neuron Calibration (2021, methodology): “We carefully tuned the parameters for CCLL and the settings are available in Section C of appendix.”
Class-Incremental Learning: A Survey (2023, methodology): “…[96], [97], [98] Relational Distillation [27], [99], [100], [101], [102], [103] § 3.6 Model Rectify Feature Rectify [104], [105], [106], [107], [108], [109] Logit Rectify [83], [89], [110], [111] Weight Rectify [112], [113], [114] § 3.7: Template-Based Classification [37], [82], [104], [115],…”
Lifelong Mixture of Variational Autoencoders (2021, background): “Promising results have been achieved on prediction tasks [6], [7], [20], [21], [22], [23].”