Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning

By Yinbo Chen, Zhuang Liu, Huijuan Xu, Trevor Darrell, Xiaolong Wang. IEEE International Conference on Computer Vision, 2020

467 citations recorded by the source corpus.

Meta-learning has been the most common framework for few-shot learning in recent years. It learns the model from collections of few-shot classification tasks, which is believed to have a key advantage of making the training objective consistent with the testing objective. However, some recent works report that by training for whole-classification, i.e. classification on the whole label-set, it can get comparable or even better embedding than many meta-learning algorithms. The edge between these two lines of works has yet been underexplored, and the effectiveness of meta-learning in few-shot learning remains unclear. In this paper, we explore a simple process: meta-learning over a whole-classification pre-trained model on its evaluation metric. We observe this simple method achieves competitive performance to state-of-the-art methods on standard bench-marks. Our further analysis shed some light on understanding the trade-offs between the meta-learning objective and the whole-classification objective in few-shot learning. Our code is available at https://github.com/yinboc/few-shot-meta-baseline.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/35172f0513d6a9fb9b1a40b7ad466da96a112d68

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Qiao
2. Peng Gao
3. Hongsheng Li
4. Renrui Zhang
5. Dacheng Tao
6. Liu Liu
7. Zhen Wang
8. Yiqun Duan

In Cancer Research

1. Dacheng Tao
2. Yu Qiao

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Hongsheng Li
3. Yu Qiao
4. Peng Gao
5. Xiaogang Wang
6. Luc Van Gool
7. Renrui Zhang
8. Ming-Hsuan Yang

Citations in the corpus

• SIN: Semantic Inference Network for Few-Shot Streaming Label Learning (2022, methodology): “Metric-based methods [15], [28]–[30] train a model to embed examples into a metric space where examples with the same label are gathered closely and examples with different labels are spread far away.”
• A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, methodology): “Table 10 compares representative meta-learning methods, where the MAML-based approaches [133, 146] further optimize the inner and outer loop to achieve new results on few-shot classification tasks, and the approaches based on new metrics [186, 188] have been greatly improved compared to the approach”
• Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners (2023, background): “Also, meta-learning is proposed to improve the few-shot adaptation ability of the models by finding a set of initialized parameters that can rapidly adapt to novel domains [12,21,38, 43].”