Meta-Learning with Latent Embedding Optimization

By Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, Raia Hadsell. International Conference on Learning Representations, 2018

1,522 citations recorded by the source corpus.

Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practical difficulties when operating on high-dimensional parameter spaces in extreme low-data regimes. We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this low-dimensional latent space. The resulting approach, latent embedding optimization (LEO), decouples the gradient-based adaptation procedure from the underlying high-dimensional space of model parameters. Our evaluation shows that LEO can achieve state-of-the-art performance on the competitive miniImageNet and tieredImageNet few-shot classification tasks. Further analysis indicates LEO is able to capture uncertainty in the data, and can perform adaptation more effectively by optimizing in latent space.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

View Paper: https://arxiv.org/abs/1807.05960

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Trevor Darrell
  4. Mohit Bansal
  5. Dacheng Tao
  6. Bernt Schiele
  7. Quoc V. Le
  8. Chelsea Finn

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Bernt Schiele
  5. Ming-Hsuan Yang
  6. Xiaogang Wang
  7. Jitendra Malik
  8. Andrew Zisserman

In Meta & Continual Learning

  1. Han-Jia Ye
  2. De-chuan Zhan
  3. Chelsea Finn
  4. Bernt Schiele
  5. Sergey Levine
  6. Wei-Lun Chao
  7. Qianru Sun
  8. Yaoyao Liu

Citations in the corpus