Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions
By Han-Jia Ye, Hexiang Hu, De-chuan Zhan, Fei Sha. 2018
830 citations recorded by the source corpus.
Learning with limited data is a key challenge for visual recognition. Many few-shot learning methods address this challenge by learning an instance embedding function from seen classes and apply the function to instances from unseen classes with limited labels. This style of transfer learning is task-agnostic: the embedding function is not learned optimally discriminative with respect to the unseen classes, where discerning among them leads to the target task. In this paper, we propose a novel approach to adapt the instance embeddings to the target classification task with a set-to-set function, yielding embeddings that are task-specific and are discriminative. We empirically investigated various instantiations of such set-to-set functions and observed the Transformer is most effective --- as it naturally satisfies key properties of our desired model. We denote this model as FEAT (few-shot embedding adaptation w/ Transformer) and validate it on both the standard few-shot classification benchmark and four extended few-shot learning settings with essential use cases, i.e., cross-domain, transductive, generalized few-shot learning, and low-shot learning. It archived consistent improvements over baseline models as well as previous methods, and established the new state-of-the-art results on two benchmarks.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Bernt Schiele
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Han-Jia Ye
- Chunhua Shen
- Xiaogang Wang
In Representation & Self-Supervised Learning
In Meta & Continual Learning
Citations in the corpus
- Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima (2021, methodology): “Current few-shot methods mainly include optimization-based methods [12, 23, 32, 39, 45, 46, 55] and metric-based methods [13, 19, 44, 49, 52, 58, 57, 53].”
- Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, methodology): “Episodic training is a common way to evaluate few-shot learning methods, further we detail the difference from the standard training protocol [3, 11, 18].”
- Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, methodology): “We also follow [47] that incorporates the embedding of the network input into the graph to help the learning of context knowledge.”
- Transformers in Vision: A Survey (2021, methodology): “self-attention between query and support set images excels compared to an image level association learned in [26].”
- Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, methodology): “For example, one line of works uses meta-learned discriminative feature embeddings [31, 44, 52, 55, 62, 70] together with the nonparametric nearest neighbor classifiers, to recognize novel classes given a few exemplars.”
- Meta-Transfer Learning Through Hard Tasks (2019, background): “ResNet-25 is exactly the same as the released code of [41], [67].”
- Interventional Few-Shot Learning (2020, background): “Thanks to them, the classification accuracy has been drastically increased [27, 72, 68, 35].”
- GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task (2023, methodology): “Existing FSL methods are mainly optimization-based [5, 10, 16] and metric-based [6, 28, 29].”
- Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “To resolve the generalization problem, many recent approaches to image classification made various attempts in training deep networks with a few annotated examples [1, 18, 29, 47, 50, 56, 62, 64, 70, 80, 81].”
- Fine-Grained Few-Shot Classification with Feature Map Reconstruction Networks (2020, methodology): “dard data augmentation as in [30, 31, 27, 3], which includes random crop, right-left flip and color jitter.”
- A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, neutral): “75% feat [242] CVPR’20 Embedding ResNet-18 ✓ ✓ ✓ ✓ 61.”
- Few-Shot Learning With a Strong Teacher (2021, methodology): “Similar approaches are [3], [10], [23], [82], [99].”
- Contextualizing Multiple Tasks via Learning to Decompose (2021, methodology): “[38, 54, 68, 58, 59] adapts the feature embeddings or the model initialization given a few-shot support set, [48, 57] compose instance representations with the supervision of attributes, and [4] utilizes the fine-grained visual concepts.”
- Curriculum-Based Meta-learning (2021, methodology): “We also note that ProtoNet and Feat with the pre-trained ResNet-12 feature extractor already achieve better or comparable accuracy to existing state-of-the-art approaches, which demonstrates the effectiveness of the pre-training strategy for model performance, as emphasized in [31, 33, 45].”
- Tailoring Embedding Function to Heterogeneous Few-Shot Tasks by Global and Local Feature Adaptors (2021, methodology): “tion (Ye et al. 2020), and dynamic subspaces (Simon et al.”
- How to Train Your MAML to Excel in Few-Shot Classification (2021, methodology): “Recent years have witnessed an abundance of new approaches on meta-learning [29, 30, 48, 61, 63, 64, 68, 77, 79, 80, 94, 98], developed in various areas including few-shot learning [70, 85, 89, 105, 107, 110, 111, 115], optimization [3, 9, 53, 108], reinforcement and imitation learning [23, 24, 31, ”
- A Universal Representation Transformer Layer for Few-Shot Image Classification (2020, background): “[23], however for a different purpose, where each representation of individual examples in a task support set is influenced by all other examples.”