The effects of negative adaptation in Model-Agnostic Meta-Learning

By Tristan Deleu, Yoshua Bengio. 2018

23 citations recorded by the source corpus.

The capacity of meta-learning algorithms to quickly adapt to a variety of tasks, including ones they did not experience during meta-training, has been a key factor in the recent success of these methods on few-shot learning problems. This particular advantage of using meta-learning over standard supervised or reinforcement learning is only well founded under the assumption that the adaptation phase does improve the performance of our model on the task of interest. However, in the classical framework of meta-learning, this constraint is only mildly enforced, if not at all, and we only see an improvement on average over a distribution of tasks. In this paper, we show that the adaptation in an algorithm like MAML can significantly decrease the performance of an agent in a meta-reinforcement learning setting, even on a range of meta-training tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “MAML is widely applied into vast learning scenarios, e.g., robotics con-trol [49], neural translation [50] and uncertainty estimation [51], and many improved algorithms [52], [53], [54], [55], [56] have been made to further boost its performance.”
Generalizing from a Few Examples (2019, background): “In this case, directly learning from all these tasks can lead to negative transfer [28].”
Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (2020, background): “In this case, the mutual knowledge transfer in meta-learning algorithms may degrade the generalization performance [7].”
Reconciling meta-learning and continual learning with online mixtures of tasks (2018, background): “On the other hand, transfer in the presence of dissimilar or outlier tasks worsens generalization performance [44, 12].”
Few-shot learning with adaptively initialized task optimizer: a practical meta-learning approach (2019, unknown): “(2018) handlesthehighcomputationalburdenofMAMLwithﬁrst-orderapproximation;Deleuand Bengio(2018)pointsoutpossiblenegativeadaptationwithtoomanyupdates;Antoniouetal.”