Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning

By Fusheng Hao, Fengxiang He, Jun Cheng, Lei Wang, Jianzhong Cao, Dacheng Tao. 2019

149 citations recorded by the source corpus.

Few-shot learning aims to learn latent patterns from few training examples and has shown promises in practice. However, directly calculating the distances between the query image and support image in existing methods may cause ambiguity because dominant objects can locate anywhere on images. To address this issue, this paper proposes a Semantic Alignment Metric Learning (SAML) method for few-shot learning that aligns the semantically relevant dominant objects through a ``collect-and-select'' strategy. Specifically, we first calculate a relation matrix (RM) to ``collect" the distances of each local region pairs of the $3$D tensor extracted from a query image and the mean tensor of the support images. Then, the attention technique is adapted to ``select" the semantically relevant pairs and put more weights on them. Afterwards, a multi-layer perceptron (MLP) is utilized to map the reweighted RMs to their corresponding similarity scores. Theoretical analysis demonstrates the generalization ability of SAML and gives a theoretical guarantee. Empirical results demonstrate that semantic alignment is achieved. Extensive experiments on benchmark datasets validate the strengths of the proposed approach and demonstrate that SAML significantly outperforms the current state-of-the-art methods. The source code is available at https://github.com/haofusheng/SAML.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, neutral): “85 SAML [15] 64-64-64-64 52.”
Prototype Mixture Models for Few-shot Semantic Segmentation (2020, background): “Existing methods can be broadly categorized as either: metric learning [22,23,17], meta-learning [24,25,26,27], or data argumentation.”
Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning (2021, methodology): “A few methods have also been proposed that aim to identify finergrained local features, such as DN4 [24], SAML [10], STANet [54] and CTM [23].”
Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, background): “While meta-learning [36, 27, 8, 15, 44, 38, 21] contributed important optimization methods and data augmentation [13, 35] aggregated performance, metric learning [32, 30, 11, 5] with prototype models [23, 4, 6, 41, 40, 18] represent the majority of few-shot learning approaches.”
A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2021, unknown): “Generally, the alignment task can be divided into two subtasks: the attentional alignment task [170,171] and the semantic alignment task [172,173].”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, methodology): “Relation network [189] differs from the above-mentioned models in that its similarity is calculated by using a neural network combined with the cosine similarity [190], attention module [191], and graph neural network [192].”