CAML: Fast Context Adaptation via Meta-Learning

By Luisa M. Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson. 2018

38 citations recorded by the source corpus.

We propose CAML, a meta-learning method for fast adaptation that partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared parameters that are meta-trained and shared across tasks. At test time, the context parameters are updated with one or several gradient steps on a task-specific loss that is backpropagated through the shared part of the network. Compared to approaches that adjust all parameters on a new task (e.g., MAML), our method can be scaled up to larger networks without overfitting on a single task, is easier to implement, and saves memory writes during training and network communication at test time for distributed machine learning systems. We show empirically that this approach outperforms MAML, is less sensitive to the task-specific learning rate, can capture meaningful task embeddings with the context parameters, and outperforms alternative partitionings of the parameter vectors.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes (2019, methodology): “The perspective allows previous work [7, 5, 15, 22, 16, 17, 23, 4, 6, 24, 9, 25, 26] to be organised in terms of i) the choice of the parameterization of the classifier (and in particular the nature of the local parameters), and ii) the function used to compute the local parameters from the training”
Meta-learning of Sequential Strategies (2019, methodology): “…Schulman, 2018], the learning targets [Xu et al., 2018], conditional distributions [Wang et al., 2017, Garnelo et al., 2018, Gordon et al., 2018, Zintgraf et al., 2018, Chen et al., 2018a], or even the entire learning procedure using a memory-based architecture such as a recurrent neural…”
Multi-Step Model-Agnostic Meta-Learning: Convergence and Improved Algorithms (2020, background): “Although having been widely used (Antoniou et al., 2018; Grant et al., 2018; Zintgraf et al., 2018; Nichol et al., 2018), the theoretical convergence of MAML is not well explored except a few attempts recently.”
Variable-Shot Adaptation for Online Meta-Learning (2020, methodology): “…et al., 2017), and optimization-based meta-learners that embed an optimization procedure into the meta-learner (Finn et al., 2017; Li et al., 2017b; Rusu et al., 2018; Zintgraf et al., 2018; Bertinetto et al., 2018; Rajeswaran et al., 2019; Lee et al., 2019; Yoon et al., 2018; Finn et al., 2018).”