Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace

By Yoonho Lee, Seungjin Choi. 2018

375 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning (2021, methodology): “Third, optimization-based approaches search for good initial parameters during training and adapt the pretrained model for new tasks (Finn et al., 2017; Lee & Choi, 2018; Grant et al., 2018).”
Fast Model Editing at Scale (2021, methodology): “Generally, gradient-based meta-learning algorithms based on MAML (Finn et al., 2017; Lee and Choi, 2018; Park and Oliva, 2019; Flennerhag et al., 2020) rely on modifying the model parameters to provide adaptability, while MEND adds adaptability post-hoc to a pre-trained model by training parameters…”
Meta-Learning with Latent Embedding Optimization (2018, methodology): “Optimization-based methods, such as MAML [6], typically use the full, high-dimensional set of model parameters within the “inner loop”, while [23] learn a layer-wise subspace in which to use gradient-based adaptation.”
Set Transformer (2018, background): “For example, few-shot image clas-siﬁcation (Finn et al., 2017; Snell et al., 2017; Lee & Choi, 2018) operates by building a classiﬁer using a support set of images, which is evaluated with query images.”
Layerwise Optimization by Gradient Decomposition for Continual Learning (2021, positive): “MT-Net [27] enabled the meta-learner to learn on each layer’s activation space, a subspace that the task-speciﬁc learner performed gradient descent on.”
Meta-Transfer Learning for Few-Shot Learning (2018, background): “3) Gradient descent based methods [9, 38, 23, 13, 59] have a specific meta-learner that learns to adapt a specific base-learner (to few-shot examples) through different tasks.”
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML (2019, methodology): “MAML has inspired many other optimization-based algorithms, which utilize the same two-loop structure (Lee and Choi, 2018; Rusu et al., 2018; Finn et al., 2018).”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “set of parameters to meta-learn, for example by subspace [76], [98], by layer [81], [98], [99], or by separating out scale and shift [100].”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, background): “be adapted rapidly using fixed steps of gradient descents over the few-shot training data from unseen categories [3, 15, 32, 37, 43].”
Learning to Self-Train for Semi-Supervised Few-Shot Classification (2019, neutral): “They can be roughly divided into four categories: (1) data augmentation based methods [15, 29, 37, 38] generate data or features in a conditional way for few-shot classes; (2) metric learning methods [36, 31, 33] learn a similarity space of image features in which the classification should be effici”
Meta-Curvature (2019, neutral): “[21] also attempted to transform the gradients.”
Generalizing from a Few Examples (2019, background): “To address this problem, in [79], it learns to choose {θ0} from a subset of a good initialization parameter for a new task.”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “(3) Gradient descent based methods [1, 13,14,20,25,37,38,43,57,70,86] usually employ a meta-learner that learns to fast adapt an NN base-learner to a new task within a few optimization steps.”
Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions (2018, background): “One fruitful idea is the meta-learning framework where the classifiers are optimized in anticipation that a future update due to data from a new task performs well on that task [2, 3, 9, 10, 21, 27, 31, 35], or the classifier itself is directly meta-predicted by the new task data [30, 47].”
On Modulating the Gradient for Meta-learning (2020, methodology): “Methods such as CAVIA [45] and T-Net [21] modulate parameters or use an additional layer to transform gradients.”
Learning Embedding Adaptation for Few-Shot Learning (2018, background): “One fruitful idea is the meta-learning framework where the classifiers are optimized in anticipation that a future update due to data from a new task performs well on that task [2, 10, 23, 29, 33, 37], or the classifier itself is directly meta-predicted by the new task data [32, 47].”
Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation (2019, methodology): “Model-agnostic meta-learners [5, 7, 15, 18, 8, 28, 36, 35] are agnostic to concrete model configurations.”