Incremental Meta-Learning via Indirect Discriminant Alignment

By Qing Liu, Orchid Majumder, Alessandro Achille, Avinash Ravichandran, Rahul Bhotika, Stefano Soatto. 2020

6 citations recorded by the source corpus.

Majority of the modern meta-learning methods for few-shot classification tasks operate in two phases: a meta-training phase where the meta-learner learns a generic representation by solving multiple few-shot tasks sampled from a large dataset and a testing phase, where the meta-learner leverages its learnt internal representation for a specific few-shot task involving classes which were not seen during the meta-training phase. To the best of our knowledge, all such meta-learning methods use a single base dataset for meta-training to sample tasks from and do not adapt the algorithm after meta-training. This strategy may not scale to real-world use-cases where the meta-learner does not potentially have access to the full meta-training dataset from the very beginning and we need to update the meta-learner in an incremental fashion when additional training data becomes available. Through our experimental setup, we develop a notion of incremental learning during the meta-training phase of meta-learning and propose a method which can be used with multiple existing metric-based meta-learning algorithms. Experimental results on benchmark dataset show that our approach performs favorably at test time as compared to training a model with the full meta-training set and incurs negligible amount of catastrophic forgetting

Subfields: Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “IDA [219] imposes discriminants of the old and new models to be aligned relative to the old centers, and otherwise leaves the embedding free to accommodate new tasks.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, background): “Some work propose to learn better representations by combining selfsupervised learning for feature extractor [9, 87] and solving class imbalance [39, 46, 54, 99, 102].”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “[75] proposed a method named Indirect Discriminant Alignment (IDA).”