A Perspective View and Survey of Meta-Learning

By Ricardo Vilalta, Youssef Drissi, Youssef Drissi. 2002

1,301 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “Meta-learning approaches include t as additional input [Schmidhuber, 1987, Thrun and Pratt, 1998, Thrun, 1998, Vilalta and Drissi, 2002].”
Learning to Optimize Neural Nets (2017, background): “More broadly related is work on “meta-learning” and “learning to learn” (Baxter et al., 1995; Vilalta & Drissi, 2002; Brazdil et al., 2008; Thrun & Pratt, 2012).”
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning (2016, background): “In the broader context of machine learning, there has been a lot of interest in one-shot learning for object classiﬁcation (Vilalta & Drissi, 2002; Fei-Fei et al., 2006; Larochelle et al., 2008; Lake et al., 2011; Koch, 2015).”
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “There is no dearth of literature exploring loss formulations or regularization techniques that unravel and exploit task relatedness, reformulate loss functions to account for adaptation and exploit meta-learning approaches in multitask models (Vilalta and Drissi, 2002; Zhang and Yang, 2017).”
MetaICL: Learning to Learn In Context (2021, methodology): “…objective, MetaICL achieves substantial improvements even with smaller LMs. Meta-training via multi-task learning Our work is broadly inspired by a large body of work in meta-learning (Vilalta and Drissi, 2002; Finn et al., 2017) and multi-task learning (Evgeniou and Pontil, 2004; Ruder, 2017).”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Few-shot learning [25, 35, 46, 53] has recently been primarily addressed in the meta-learning framework [57, 71, 80], where a model learns an update rule for the parameters of a base-learner model [6, 7, 72] through a sequence of training episodes [79, 88].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “The inner loop is used to solve the actual few-shot classiﬁcation problem, while the outer loop serves as a guidance for the former by gradually modifying the inductive bias of the base learner (Vilalta & Drissi, 2002).”
Meta-Learning With Differentiable Convex Optimization (2019, background): “Meta-learning studies what aspects of the learner (commonly referred to as bias or prior) effect generalization across a distribution of tasks [26, 31, 32].”
Efficient Architecture Search by Network Transformation (2017, methodology): “lar to [35], we use a neural network to represent the policy of the agent and train the network with policy gradient methods [33,13,21]. Our work is also related to meta-learning or learning to learn [31], the idea of using acquired knowledge from one task to improve learning in a future task. More ”
Meta Networks (2017, background): “Previous work on meta learning has formulated the problem as a two-level learning, a slow learning of a meta-level model performing across tasks and a rapid learning of a base-level model acting within each task (Mitchell et al., 1993; Vilalta & Drissi, 2002).”
Learning to Optimize (2016, background): “This line of work, known as “learning to learn” or “meta-learning” [1, 27, 5, 26], considers the problem of devising methods that can take advantage of knowledge learned on other related tasks to train faster, a problem that is today better known as multi-task learning and transfer learning.”
Learning Rapid-Temporal Adaptations (2017, background): “Previous work on metalearning has formulated the problem as two-level learning; speciﬁcally, “slow” learning of a meta-level model that occurs across several tasks, and “fast” learning of a base-level model that acts within each task [24, 41, 29, 7, 26, 23].”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “See (Vilalta and Drissi, 2002; Hospedales, Antoniou, Micaelli, and Storkey, 2020) for an in-depth survey of meta-learning.”
Machine Theory of Mind (2018, methodology): “We consider the challenge of building a Theory of Mind as essentially a meta-learning problem (Schmidhuber et al., 1996; Thrun & Pratt, 1998; Hochreiter et al., 2001; Vilalta & Drissi, 2002).”
Practical Network Blocks Design with Q-Learning (2017, methodology): “Our work is also related to the hyper-parameter optimization (Bergstra et al. 2011) , meta-learning (Vilalta and Drissi 2002) and learning to learn methods (Hochreiter, Younger, and Conwell 2001; Andrychowicz et al. 2016).”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “This perspective [27]–[29] views meta-learning as a tool to manage the ‘no free lunch’ theorem [30] and improve generalization by searching for the algorithm (inductive bias) that is best suited to a given problem, or problem family.”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, background): “ well-deﬁned semantic descriptions. Few-shot learning (FSL) propose a more realistic setup, where we have access to a very limited number (instead of zero) of visual exemplars from the “tail” classes [3, 33]. Such approaches usually learn with a meta-learning objective that transfer the task-level i”
Reinforcement Learning for Architecture Search by Network Transformation (2017, background): “Our work is also related to meta-learning or learning to learn [31], the idea of using acquired knowledge from one task to improve learning in a future task.”