Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-. hook

By J. Urgen Schmidhuber. 1987

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “Meta-learning approaches include t as additional input [Schmidhuber, 1987, Thrun and Pratt, 1998, Thrun, 1998, Vilalta and Drissi, 2002].”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, methodology): “The method that we propose in this paper addresses the general problem of meta-learning (Thrun & Pratt, 1998; Schmidhuber, 1987; Naik & Mammone, 1992), which includes few-shot learning.”
Learning to learn by gradient descent by gradient descent (2016, neutral): “Different behavior on each coordinate is achieved by using separate activations for each objective function parameter.”
Learning with Latent Language (2017, neutral): “Closely related meta-learning approaches (e.g. Schmidhuber, 1987; Santoro et al., 2016; Vinyals et al., 2016) make use of the same data, but collapse the inner optimization over θ ( c ) and subsequent prediction of y ( e ) into a single learned model.”
One-Shot Imitation Learning (2017, methodology): “Like our algorithm, many of the aforementioned approaches are a form of meta-learning (Thrun & Pratt, 1998; Schmidhuber, 1987; Naik & Mammone, 1992), where the algorithm itself is being learned.”
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning (2021, background): “Transfer learning (Schmidhuber, 1987; Pratt et al., 1991; Caruana et al., 1995) has been the cornerstone of recent progress in natural language processing (Ruder et al., 2019; Devlin et al., 2019; Raffel et al., 2020).”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Few-shot learning [25, 35, 46, 53] has recently been primarily addressed in the meta-learning framework [57, 71, 80], where a model learns an update rule for the parameters of a base-learner model [6, 7, 72] through a sequence of training episodes [79, 88].”
Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes (2019, methodology): “This general approach relates to methods for meta-learning [1, 2] and few-shot learning [3].”
Meta-Learning Probabilistic Inference for Prediction (2018, background): “Second, we share information between tasks about how to learn and perform inference using meta-learning (Naik and Mammone, 1992; Thrun and Pratt, 2012; Schmidhuber, 1987).”
Meta-learning with differentiable closed-form solvers (2018, background): “learning to learn) [14, 15, 29] has been of great importance in the Machine Learning community for several decades.”
Learning to Learn with Generative Models of Neural Network Checkpoints (2022, background): “Of course, the concept of learning improved optimizers is not new and dates back to the 1980s, if not earlier, following early work from Schmidhuber (1987) and Bengio et al. (1991).”
How to train your MAML (2018, background): “Meta-learning (Schmidhuber, 1987; Vilalta & Drissi, 2002), or learning to learn (Thrun & Pratt, 1998), can instead be used to automatically learn across-task knowledge usually referred to as across-task (or sometimes slow) knowledge such that our model can, at inference time, quickly acquire…”
Toward Causal Representation Learning (2021, background): “The contexts can likewise be interpreted as different tasks, which provides a connection to metalearning [23], [68], [213].”
Meta-Learning with Latent Embedding Optimization (2018, background): “The problem of few-shot adaptation has been approached in the context of fast weights (Hinton & Plaut, 1987; Ba et al., 2016), learning-to-learn (Schmidhuber, 1987; Thrun & Pratt, 1998; Hochre-iter et al., 2001; Andrychowicz et al., 2016), and through meta-learning .”
Meta-Learning With Differentiable Convex Optimization (2019, positive): “We ﬁrst derive the meta-learning framework for few-shot learning following prior work ( e.g ., [28, 22, 8]) and then discuss how convex base learners, such as linear SVMs, can be incorporated.”
Deep learning in neural networks: An overview (2014, methodology): “Around 1960: visual cortex provides inspiration for DL (Sections”
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch (2020, background): “Early approaches have proposed to search for programs that improve themselves (Lenat, 1983; Schmidhuber, 1987).”