How to Train Your MAML to Excel in Few-Shot Classification

By Han-Jia Ye, Wei-Lun Chao. 2021

65 citations recorded by the source corpus.

Model-agnostic meta-learning (MAML) is arguably one of the most popular meta-learning algorithms nowadays. Nevertheless, its performance on few-shot classification is far behind many recent algorithms dedicated to the problem. In this paper, we point out several key facets of how to train MAML to excel in few-shot classification. First, we find that MAML needs a large number of gradient steps in its inner loop update, which contradicts its common usage in few-shot classification. Second, we find that MAML is sensitive to the class label assignments during meta-testing. Concretely, MAML meta-trains the initialization of an $N$-way classifier. These $N$ ways, during meta-testing, then have"$N!$"different permutations to be paired with a few-shot task of $N$ novel classes. We find that these permutations lead to a huge variance of accuracy, making MAML unstable in few-shot classification. Third, we investigate several approaches to make MAML permutation-invariant, among which meta-training a single vector to initialize all the $N$ weight vectors in the classification head performs the best. On benchmark datasets like MiniImageNet and TieredImageNet, our approach, which we name UNICORN-MAML, performs on a par with or even outperforms many recent few-shot classification algorithms, without sacrificing MAML's simplicity.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “Few-Shot Learning aims to extract the inductive bias from base classes and generalize it to unseen classes [44], [45], [46], [47], [48].”
Few-Shot Learning With a Strong Teacher (2021, methodology): “Based on the meta-trained model initialization [7], [58], [59], [60], [61], [62], the few-shot learner can adapt it to a good classifier for each few-shot task with a small number of gradient descent steps.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, background): “UNICORN-MAML [154] summarizes the above work and gives a few suggestions on how to train MAML to compete with the state-of-the-art methods.”