Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement

By J. Urgen Schmidhuber, Jieyu Zhao, Marco A. Wiering. 1997

Subfields: Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

Diversity With Cooperation: Ensemble Methods for Few-Shot Classification (2019, background): “The dominant paradigm in few-shot learning builds upon meta-learning [9, 24, 30, 32, 31, 33], which is formulated as a principle to learn how to adapt to new learning problems.”
TADAM: Task dependent adaptive metric for improved few-shot learning (2018, methodology): “rature. The ﬁrst one, which is used in this work, is the meta-learning approach, i.e. learning a model that, given a task (set of labeled data), produces a classiﬁer that generalizes across all tasks [31, 25]. This is the case of Matching Networks [33], which optionally use a Recurrent Neural Networ”
One-shot Learning with Memory-Augmented Neural Networks (2016, methodology): “Although the term has been used in numerous senses (Schmidhuber et al., 1997; Caruana, 1997; Schweighofer & Doya, 2003; Brazdil et al., 2003), meta-learning generally refers to a scenario in which an agent learns at two levels, each associated with different time scales.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “One early promising example was the Success Story Algorithm (SSA) (Schmidhuber, Zhao, and Wiering, 1997), which leveraged backtracking for improvements that maximize a long-term average reward per step objective.”
Meta-Learning: A Survey (2018, neutral): “Later work used reinforcement learning across tasks to adapt the search strategy (Schmidhuber et al., 1997) or the learning rate for gradient descent (Daniel et al.”
Learning to Generalize: Meta-Learning for Domain Generalization (2017, positive): “This is related to the long standing idea of learning to learn (Thrun and Pratt 1998; Schmidhuber, Zhao, and Wiering 1997), which has recently seen a resurgence of popularity with applications to few-shot learning (Finn, Abbeel, and Levine 2017; Ravi and Larochelle 2017) and learning optimizers…”
First Experiments with PowerPlay (2012, background): “learned a version of s called s i 1 able to classify i 1 previously invented training patterns (i > 1). Then the next task is deﬁned by a simple enumerative search in the style of universal search [10, 26, 21], which combines task simpliﬁcation and systematic run-time growth (see Alg. 2). Algorit”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “General methods for incremental multitask RL and algorithmic transfer learning that are not NN-specific include the evolutionary ADATE system [182], the Success-Story Algorithm for SelfModifying Policies running on general-purpose computers [233, 252, 251], and the Optimal Ordered Problem Solver [23”
Optimal Ordered Problem Solver (2002, neutral): “Compare Levin (1973, 1984), Solomonoﬀ (1986), Schmidhuber et al. (1997b), Li and Vit´anyi (1997), Hutter (2002a) (Levin also attributes similar ideas to Al-lender): Method 2.1 ( Lsearch) Set current time limit T=1.”
Learning to Learn: Meta-Critic Networks for Sample Efficient Learning (2017, background): “Learning eﬀectively from few examples is a key capability of humans that machine learning has yet to replicate, despite a long history of research [35, 25, 30] and explosive recent interest [9, 27, 22, 4].”
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning (2018, background): “Meta-learning, or learning to learn, has been a topic of continued interest in machine learning research [41, 38].”
A survey of deep meta-learning (2020, background): “, 2019) (see Figure 4), hyperparameter optimization (Antoniou et al., 2019; Schmidhuber et al., 1997), meta-learning learning rules (Bengio et al.”