Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation

By Risto Vuorio, Shao-Hua Sun, Hexiang Hu, Joseph J. Lim. 2019

251 citations recorded by the source corpus.

Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those frameworks demonstrate appealing performance on a variety of domains such as few-shot image classification and reinforcement learning. However, one important limitation of such frameworks is that they seek a common initialization shared across the entire task distribution, substantially limiting the diversity of the task distributions that they are able to learn from. In this paper, we augment MAML with the capability to identify the mode of tasks sampled from a multimodal task distribution and adapt quickly through gradient updates. Specifically, we propose a multimodal MAML (MMAML) framework, which is able to modulate its meta-learned prior parameters according to the identified mode, allowing more efficient fast adaptation. We evaluate the proposed model on a diverse set of few-shot learning tasks, including regression, image classification, and reinforcement learning. The results not only demonstrate the effectiveness of our model in modulating the meta-learned prior in response to the characteristics of tasks but also show that training on a multimodal distribution can produce an improvement over unimodal training. The code for this project is publicly available at https://vuoristo.github.io/MMAML.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, neutral): “We follow the split in [59].”
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches (2022, background): “There are three major swim lanes of the FSL problem: (a) recurrent-based [40, 48] (b) optimisation-based [43, 59], and (c) metric-based frameworks [17, 24].”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “t to provide fast learning for a wide range of potential tasks, or if one is limited to narrow distributions p(T). This has led to variants that model mixtures over multiple initial conditions [100], [103], [104]. Optimizer The above parameter-centric methods usually rely on existing optimizers such”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, methodology): “For example, one line of works uses metalearned discriminative feature embeddings (Snell et al. 2017; Oreshkin et al. 2018; Rusu et al. 2019; Vuorio et al. 2019; Lee et al. 2019; Ye et al. 2020) together with the non-parametric nearest neighbor classifiers, to recognize novel classes given a few…”
Graph Information Aggregation Cross-Domain Few-Shot Learning for Hyperspectral Image Classification (2022, background): “There are three main meta-learning methods to solve the problem of few-shot classiﬁcation (FSC), i.e., the recurrent-based frameworks [28], optimization-based schemes [29], [30], and metric-based methods [31], [32].”
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness (2021, background): “Multi-domain meta learning [53, 54, 58] assume tasks from all domains are available during meta training.”
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning (2022, methodology): “In particular, model-agnostic meta learning (MAML) is one of the most popular optimization-based meta learning frameworks for few-shot learning (Finn et al., 2017; Vuorio et al., 2019; Yin et al., 2020; Obamuyide & Vlachos, 2019).”
Learning to Synthesize Programs as Interpretable and Generalizable Policies (2021, background): “[89] Risto Vuorio, Shao-Hua Sun, Hexiang Hu, and Joseph J Lim.”
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation (2020, methodology): “Second, optimization-based schemes (Finn et al., 2017; Rusu et al., 2019; Tseng et al., 2019; Vuorio et al., 2019) learn to ﬁne-tune the model with the few example images by integrating the ﬁne-tuning process in the meta-training stage.”
Skill-based Meta-Reinforcement Learning (2022, background): “…al., 2018; Rothfuss et al., 2019; Gupta et al., 2018; Vuorio et al., 2018; Nagabandi et al., 2019; Clavera et al., 2019; 2018; Rakelly et al., 2019; Vuorio et al., 2019; Yang et al., 2019; Zintgraf et al., 2019; Humplik et al., 2019; Zintgraf et al., 2020; Liu et al., 2021) hold the promise of…”
How to Train Your MAML to Excel in Few-Shot Classification (2021, background): “Another direction is to enable task-specific initialization [75, 99, 109, 111], which often needs additional sub-networks.”
On Modulating the Gradient for Meta-learning (2020, background): “The modulation can be performed in the feature space [40, 45].”
Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (2020, background): “Sequential variational inference [16] for DP mixtures has been recently integrated with DNNs for image classification [11].”
Contextualizing Multiple Tasks via Learning to Decompose (2021, methodology): “There exist rich semantics inside and between instances — the label of an object could be annotated differently when workers have various preferences [5, 47, 54, 43, 67].”
Online Structured Meta-learning (2020, background): “To handle the heterogeneous task distribution, under stationary setting, a few studies modulate the meta-learner to different tasks [3, 27, 39, 41, 42].”