Revisiting Meta-Learning as Supervised Learning

By Wei-Lun Chao, Han-Jia Ye, De-chuan Zhan, Murray Campbell, Kilian Q. Weinberger. 2020

24 citations recorded by the source corpus.

Recent years have witnessed an abundance of new publications and approaches on meta-learning. This community-wide enthusiasm has sparked great insights but has also created a plethora of seemingly different frameworks, which can be hard to compare and evaluate. In this paper, we aim to provide a principled, unifying framework by revisiting and strengthening the connection between meta-learning and traditional supervised learning. By treating pairs of task-specific data sets and target models as (feature, label) samples, we can reduce many meta-learning algorithms to instances of supervised learning. This view not only unifies meta-learning into an intuitive and practical framework but also allows us to transfer insights from supervised learning directly to improve meta-learning. For example, we obtain a better understanding of generalization properties, and we can readily transfer well-understood techniques, such as model ensemble, pre-training, joint training, data augmentation, and even nearest neighbor based methods. We provide an intuitive analogy of these methods in the context of meta-learning and show that they give rise to significant improvements in model performance on few-shot learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Qiwen Wang
5. Ziwei Liu
6. Wei-Lun Chao
7. Zhiyuan Qi
8. Lu Ming

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Wei-Lun Chao
5. Qiwen Wang
6. Ziwei Liu
7. Lu Ming
8. Zhiyuan Qi

Citations in the corpus

• Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech (2021, background): “MAML requires the model to learn across tasks to become a good meta-initialization that can adapt to new tasks easily [19].”
• How to Train Your MAML to Excel in Few-Shot Classification (2021, methodology): “, ResNet [40] pre-trained on the meta-training set [13, 67, 79, 110].”
• Few-Shot Learning With a Strong Teacher (2021, methodology): “Some recent methods emphasize the importance of pre-trained features [19], [23], [34], [35], [44], [64].”
• Class-Incremental Learning: A Survey (2023, background): “On the other hand, some works assume the updating rule can be meta-learned [171], [172] from a series of related tasks.”
• Contextualizing Multiple Tasks via Learning to Decompose (2021, background): “Jointly learning their concept probabilities facilitates the learning experience transition between tasks, instead of treating same-class instances independently in standard episodic training [53, 6].”