Auto-Meta: Automated Gradient Based Meta Learner Search

By Jaehong Kim, Youngduck Choi, Moonsu Cha, Jung Kwon Lee, Sangyeul Lee, Sungwan Kim, Yongseok Choi, Jiwon Kim. 2018

44 citations recorded by the source corpus.

Fully automating machine learning pipelines is one of the key challenges of current artificial intelligence research, since practical machine learning often requires costly and time-consuming human-powered processes such as model design, algorithm development, and hyperparameter tuning. In this paper, we verify that automated architecture search synergizes with the effect of gradient-based meta learning. We adopt the progressive neural architecture search \cite{liu:pnas_google:DBLP:journals/corr/abs-1712-00559} to find optimal architectures for meta-learners. The gradient based meta-learner whose architecture was automatically found achieved state-of-the-art results on the 5-shot 5-way Mini-ImageNet classification problem with $74.65\%$ accuracy, which is $11.54\%$ improvement over the result obtained by the first gradient-based meta-learner called MAML \cite{finn:maml:DBLP:conf/icml/FinnAL17}. To our best knowledge, this work is the first successful neural architecture search implementation in the context of meta learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Chelsea Finn
3. Pieter Abbeel
4. Timothy M. Hospedales
5. Aravind Rajeswaran
6. Amos J. Storkey
7. Sham M. Kakade
8. Dacheng Tao

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Sergey Levine
3. Chelsea Finn
4. Timothy M. Hospedales
5. Luc Van Gool
6. Trevor Darrell
7. Yi Yang
8. Bernt Schiele

In Meta & Continual Learning

1. Chelsea Finn
2. Sergey Levine
3. Mikhail Khodak
4. Amos J. Storkey
5. Ameet Talwalkar
6. Timothy M. Hospedales
7. Paul Micaelli
8. Antreas Antoniou

In Neural Architecture Search & AutoML

1. Ameet Talwalkar
2. Timothy M. Hospedales
3. Amos J. Storkey
4. Frank Hutter
5. Aaron Klein

Citations in the corpus

• Adaptive Gradient-Based Meta-Learning Methods (2019, methodology): “GBML is used in a variety of LTL domains such as vision [39, 45, 36], federated learning [16], and robotics [21, 1].”
• Meta-Learning in Neural Networks: A Survey (2020, background): “While few-shot meta-learning is typically addressed from a parameter learning perspective in the context of hand-crafted architectures [19], [20], [87], one can also define NAS meta-objectives to train an architecture suitable for few-shot learning [238], [239].”
• Provable Guarantees for Gradient-Based Meta-Learning (2019, background): “Despite its simplicity, gradient-based meta-learning is a leading approach for LTL in numerous domains including vision (Li et al., 2017; Nichol et al., 2018; Kim et al., 2018), robotics (Al-Shedivat et al., 2018), and federated learning (Chen et al., 2018).”
• Meta-Learning with Implicit Gradients (2019, background): “Note however that architecture tuning can lead to better results for all algorithms [27].”