Optimization as a Model for Few-Shot Learning

By Sachin Ravi, Hugo Larochelle. 2016

3,710 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Do Better ImageNet Models Transfer Better? (2018, methodology): “This problem is closely related to few-shot learning [45, 82, 64, 72, 25, 72, 56], but these methods are typically evaluated with training and test classes from the same distribution.”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “Past techniques have applied such approaches to transfer learning by learning networks that predict models rather than classes [51, 31].”
Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, methodology): “learning. Deep learning models are neither designed to train with very few examples nor to converge very fast. To ﬁx that, optimization-based methods intend to learn with a few examples. Meta-learner [40] exploited an LSTM to satisfy two main desiderata of few-shot learning: quick acquisition of tas”
One-Shot Imitation Learning (2017, background): “One-shot and few-shot learning has been studied for image recognition (Vinyals et al., 2016; Koch, 2015; Santoro et al., 2016; Ravi & Larochelle, 2017), generative modeling (Edwards & Storkey, 2017; Rezende et al., 2016), learning “fast” reinforcement learning agents with recurrent policies (Duan…”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, background): “The usual setup for few-shot out-of-domain transfer learning consists of transfer learning from a “parent” task, with abundant training examples, to a “child” task that only has a few training examples (Thrun, 1996; Ravi and Larochelle, 2017).”
True Few-Shot Learning with Language Models (2021, neutral): “The meta-learning setting assumes access to data from many other tasks in order to perform learning and model selection [20, 21, 64, 65].”
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, methodology): “Since this approach models the learning process as a sequence classification problem, it can be seen as a form of metalearning similar to (Ravi and Larochelle, 2016).”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “The meta-learner either learns to produce new parameters directly from the new data [9, 33, 56, 62, 64, 72, 73], or learns to produce an update rule to iteratively optimize the base learner to fit the new data [2, 6, 8, 38, 63, 97].”
Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes (2019, methodology): “VERSA [5], Proto Nets [4], Matching Nets [24] MAML [7] Meta-LSTM [22]”
Meta-Learning Probabilistic Inference for Prediction (2018, methodology): “Table 3: Accuracy results for different few-shot settings on Omniglot and mini ImageNet.”
Meta-learning with differentiable closed-form solvers (2018, methodology): “On the other side of the spectrum, methods that optimize standard iterative learning algorithms, such as backpropagating through gradient descent (Finn et al., 2017; Nichol et al., 2018) or explicitly learning the learner’s update rule (Hochreiter et al., 2001; Andrychowicz et al., 2016; Ravi & Laro”
Fast Model Editing at Scale (2021, methodology): “Various methods for meta-learning also use gradient transforms to achieve better model updates for few-shot learning (Ravi and Larochelle, 2017; Li et al., 2017; Lee and Choi, 2018; Park and Oliva, 2019; Flennerhag et al., 2020).”
Learning to Learn with Generative Models of Neural Network Checkpoints (2022, background): “They are typically trained with unrolled optimization (Hochreiter et al., 2001; Younger et al., 2001; Gregor & LeCun, 2010; Andrychowicz et al., 2016; Ravi & Larochelle, 2017; Wichrowska et al., 2017; Lv et al., 2017; Metz et al., 2019; 2022) or reinforcement learning (Li & Malik, 2016; 2017).”
Learning Transferable Architectures for Scalable Image Recognition (2017, background): “The concept of having one neural network interact with a second neural network to aid the learning process, or learning to learn or meta-learning [21, 41] has attracted much attention in recent years [1, 13, 14, 18, 32, 38, 52]. Most of these approaches have not been scaled to large problems like Im”