A neural network that embeds its own meta-levels

By J. Urgen Schmidhuber. 1993

Subfields: Model Architectures, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Meta & Continual Learning

Citations in the corpus

CrossTransformers: spatially-aware few-shot transfer (2020, background): “The meta-learner either learns to produce new parameters directly from the new data [9, 33, 56, 62, 64, 72, 73], or learns to produce an update rule to iteratively optimize the base learner to fit the new data [2, 6, 8, 38, 63, 97].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “This technique has been first introduced by Schmidhuber [35, 36] and recently revamped by Bertinetto et al.”
Meta Networks (2017, background): “The base and the meta-level models can be framed in a single learner (Schmidhuber, 1993) or in separate learners (Bengio et al., 1990; Hochreiter et al., 2001).”
Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, neutral): “Meta Learning [51] focuses on rapidly adapting to new unseen tasks by learning prior knowledge through training on a large number of similar tasks.”
Meta-Learning: A Survey (2018, neutral): “An early meta-learning approach is to create recurrent neural networks (RNNs) able to modify their own weights [148, 149].”
Large-Scale Long-Tailed Recognition in an Open World (2019, background): “It is often formulated as meta learning [46, 5, 37, 42, 12, 57].”
Meta-Learning in Neural Networks: A Survey (2020, background): “Schmidhuber et al .continued to explore self-referential systems and meta-learning in subsequent work [49], [50].”
Learning What Data to Learn (2017, background): “In that sense, NDF belongs to the category of meta learning (Schmidhuber, 1987; 1993), a.k.a., learning to learn (Thrun & Pratt, 2012; Li & Malik, 2016; Andrychowicz et al., 2016).”
A survey of deep meta-learning (2020, background): “Schmidhuber developed a theory of self-referential learning, where the weights of a neural network can serve as input to the model itself, which then predicts updates (Schmidhuber 1987, 1993).”
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness (2021, background): “Meta Learning: Meta learning [49] focuses on rapidly adapting to unseen tasks by learning on a large number of similar tasks.”
Reinforcement Learning with Self-Modifying Policies (1998, background): “Self-modifying recurrent neural nets [18, 41] are able to run their own weight change algorithms.”
Learning to Model the Tail (2017, methodology): “Specifically, our approach is inspired by early work on parameter prediction that modifies the weights of one network using another [32, 33, 34, 35, 36, 37, 38, 26, 39].”
Meta-learning with backpropagation (2001, result): “The self-modifying neural networks of Schmidhuber et al. [ 3 ], which run their own learning algorithms, are similar to our meta-learning method.”