Adaptive Subspaces for Few-Shot Learning

By Christian Simon, Piotr Koniusz, Richard Nock, Mehrtash Harandi. 2020

425 citations recorded by the source corpus.

Object recognition requires a generalization capability to avoid overfitting, especially when the samples are extremely few. Generalization from limited samples, usually studied under the umbrella of meta-learning, equips learning techniques with the ability to adapt quickly in dynamical environments and proves to be an essential aspect of life long learning. In this paper, we provide a framework for few-shot learning by introducing dynamic classifiers that are constructed from few samples. A subspace method is exploited as the central block of a dynamic classifier. We will empirically show that such modelling leads to robustness against perturbations (e.g., outliers) and yields competitive results on the task of supervised and semi-supervised few-shot classification. We also develop a discriminative form which can boost the accuracy even further. Our code is available at https://github.com/chrysts/dsn_fewshot

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

On Generalizing Beyond Domains in Cross-Domain Continual Learning (2022, background): “The projection matrices are also known to be effective to compute similarity between two entities as proposed in [31, 39].”
Subspace Regularizers for Few-Shot Class Incremental Learning (2021, background): “Also related Simon et al. (2020) and Devos & Grossglauser (2019) model class representations over a set of subspaces (disjoint in the latter) for non-incremental few-shot learning.”
On Learning the Geodesic Path for Incremental Learning (2021, methodology): “The projection to subspaces has been used previously in domain adaptation [12, 13] and few-shot learning [36].”
Synthesized Feature based Few-Shot Class-Incremental Learning on a Mixture of Subspaces (2021, methodology): “Modeling data by projection onto subspaces has been widely used in many computer vision and machine learning applications [22, 1, 17, 16, 32, 23].”
Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation (2021, background): “Few-shots learning can be considered as a matching problem where the query is compared with every element in the support set [43, 44, 45, 46].”
Curriculum-Based Meta-learning (2021, methodology): “Two benchmarks are used for performance evaluation, including mini-ImageNet [41] and tiered-ImageNet [35].”
Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning (2021, background): “The results of the baselines in Table 7 (miniImageNet) are obtained from [22], [55], [38], [14] and [56].”
On Modulating the Gradient for Meta-learning (2020, background): “Note that, our meta-learning approach is an adaptive module acting on the model parameters which diﬀers from relation and metric learning approaches e.g ., [17,23,35,36,42,43].”
Leveraging the Feature Distribution in Transfer-based Few-Shot Learning (2020, neutral): “ductive ProtoNet [28] ConvNet64 55:50 0:70% 72:00 0:60% MAML [8] ConvNet32 58:90 1:90% 71:50 1:00% S2M2 R [21] WRN 74:81 0:19% 87:47 0:13% PT+NCM(ours) WRN 74:64 0:21% 87:64 0:15% Transductive DSN-MR [27] ResNet12 78:00 0:90% 87:30 0:60% Transfer+SGC [13] WRN 83:90 0:22% 88:76 0:15% PT+MAP(ours) WRN”