Meta-SGD: Learning to Learn Quickly for Few Shot Learning

By Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li. 2017

1,259 citations recorded by the source corpus.

Few-shot learning is challenging for learning algorithms that learn each task in isolation and from scratch. In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurately and faster with fewer examples, where the choice of meta-learners is crucial. In this paper, we develop Meta-SGD, an SGD-like, easily trainable meta-learner that can initialize and adapt any differentiable learner in just one step, on both supervised learning and reinforcement learning. Compared to the popular meta-learner LSTM, Meta-SGD is conceptually simpler, easier to implement, and can be learned more efficiently. Compared to the latest meta-learner MAML, Meta-SGD has a much higher capacity by learning to learn not just the learner initialization, but also the learner update direction and learning rate, all in a single meta-learning process. Meta-SGD shows highly competitive performance for few-shot learning on regression, classification, and reinforcement learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Chelsea Finn
  4. Mohit Bansal
  5. Trevor Darrell
  6. Quoc V. Le
  7. Dacheng Tao
  8. Bernt Schiele

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Bernt Schiele
  5. Ming-Hsuan Yang
  6. Xiaogang Wang
  7. Yi Yang
  8. Jitendra Malik

In Meta & Continual Learning

  1. Han-Jia Ye
  2. De-chuan Zhan
  3. Chelsea Finn
  4. Wei-Lun Chao
  5. Sergey Levine
  6. Bernt Schiele
  7. Fei Sha
  8. Yaoyao Liu

Citations in the corpus