Metalearning: a survey of trends and technologies

By Christiane Lemke, Marcin Budka, Bogdan Gabrys. 2013

377 citations recorded by the source corpus.

Metalearning attracted considerable interest in the machine learning community in the last years. Yet, some disagreement remains on what does or what does not constitute a metalearning problem and in which contexts the term is used in. This survey aims at giving an all-encompassing overview of the research directions pursued under the umbrella of metalearning, reconciling different definitions given in scientific literature, listing the choices involved when designing a metalearning system and identifying some of the future research challenges in this domain.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Embracing Change: Continual Learning in Deep Neural Networks. (2020, background): “Historically, a number of ‘ meta-learning ’ or ‘ learning-to-learn ’ approaches have demonstrated that solutions can be improved by automatically learning inductive biases (such as architecture, data, and learning parameters) that would otherwise need to be hand-designed (Figure 1E) [74,75].”
Meta-Learning: A Survey (2018, neutral): “There exists a rich body of earlier work (Brazdil et al., 2009; Lemke et al., 2015; Giraud-Carrier, 2008; Luo, 2016) on building meta-models for algorithm selection (Bensusan and Giraud-Carrier, 2000; Pfahringer et al.”
Meta-Learning in Neural Networks: A Survey (2020, background): “and/or focus on algorithm selection for data mining [29], [33], [35], [36], AutoML [34], or particular applications of meta-learning such as few-shot learning [37] or NAS [38].”
Task-Embedded Control Networks for Few-Shot Imitation Learning (2018, background): “Our work lies at the intersection of imitation learning [8, 9] and meta-learning [10, 11].”
Revisiting Meta-Learning as Supervised Learning (2020, background): “There are excellent overviews and surveys of metalearning (Thrun and Pratt, 1998; Vilalta and Drissi, 2002; Lemke et al., 2015; Vanschoren, 2018; Finn, 2018; Metz et al., 2018).”
Edge-Labeling Graph Neural Network for Few-Shot Learning (2019, background): “A lot of interest in meta-learning [1] has been recently arisen in various areas including especially taskgeneralization problems such as few-shot learning [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], learn-to-learn [16, 17, 18], non-stationary reinforcement learning[19, 20, 21], and continual ”
Towards Knowledgeable Supervised Lifelong Learning Systems (2020, background): “, 2016), multitask learning (Caruana, 1998), metalearning (Brazdil et al., 2008; Lemke et al., 2015), lifelong machine learning (Chen & Liu, 2016, 2018) and continual learning (Parisi et al.”
On the predictive power of meta-features in OpenML (2017, background): “Among them, one that has been the focus of research for long is meta-learning (Brazdil et al., 2008; Bilalli et al., 2017; Lemke et al., 2015).”
How to Train Your MAML to Excel in Few-Shot Classification (2021, background): “Meta-learning, or learning to learn, is a sub-field of machine learning that attempts to search for the best learning strategy as the learning experiences increases [28, 49, 92, 95, 97].”
A Meta-Learning Perspective on Cold-Start Recommendations for Items (2017, methodology): “In this paper, we propose to view recommendations from a meta-learning perspective [28, 18].”
P4ML: A Phased Performance-Based Pipeline Planner for Automated Machine Learning (2018, background): “…(Feurer et al., 2015a) ﬁnd marked improvement from using ensembles of models (Caruana et al., 2004) and warm-starting hyperparameter settings based on dataset meta-features (Feurer et al., 2015b), ﬁtting within the wider tradition of meta-learning (Brazdil et al., 2008; Lemke et al., 2015).”
Meta-learning with neural networks and landmarking for forecasting model selection an empirical evaluation of different feature sets applied to industry data (2016, methodology): “In this context, meta-learning is a promising approach for model selection [8], [9], [10], [11] describing how to deploy knowledge from past tasks in order to select predictive models for new tasks with certain characteristics.”
Small Sample Learning in Big Data Era (2018, background): “– Meta knowledge: Some high-level knowledge beyond data, and can be helpful to compensate each concept learning with not sufficient samples (Lemke et al., 2015; Lake et al., 2017).”
Scalable Gaussian process-based transfer surrogates for hyperparameter optimization (2017, background): “According to Lemke et al. (2015), a meta-learning system must fulfill two properties: 1.”
Meta-SR: A Magnification-Arbitrary Network for Super-Resolution (2019, background): “The detailed survey of the metalearning can be found in [16].”
Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, methodology): “, meta-learning [24], [44] and transfer learning [26], they both try to make use of external knowledge to enhance basic ones for the optimizer and evaluator.”
Beyond Manual Tuning of Hyperparameters (2015, background): “The problem of predicting the best performing model for a given dataset/task has been studied extensively as special case of algorithm selection within the wider area of metalearning [42, 60, 69].”
Meta Multi-Task Learning for Sequence Modeling (2018, background): “One goal of meta-learning is to ﬁnd efﬁcient mechanisms to transfer knowledge across domains or tasks (Lemke, Budka, and Gabrys 2015).”