Exploring the predictable

By J. Urgen Schmidhuber. 2003

86 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Active learning of inverse models with intrinsically motivated goal exploration in robots (2013, neutral): “owing general computer programs to actively and adaptively self-generate abstract computational problems, or goals, of increasing complexity were studied in a theoretical computer science perspective [98, 99, 102]. While the high expressivity of these formalisms allows in principle to tackle a wide ”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, neutral): “Previous approximative implementations based on traditional RL [24, 40] and on co-evo lution of program-like task generators and task solvers [29, 30] also did not have a built-in guarant ee that they cannot forget solutions to previously solved tasks, while P OWERPLAY as in Section 2 does (and the ”
First Experiments with PowerPlay (2012, background): “However, there is a phase transition to more complex non-linear functions after a few tasks, indicating a new developmental stage [14, 19, 12].”
Optimal Ordered Problem Solver (2002, neutral): “In what follows we will discuss to which extent “no free lunch theorems” are relevant to oops (Section 5.1), which are the essential limitations of oops (Section 5.2), and how to use oops for reinforcement learning (Section 5.3).”
Some Considerations on Learning to Explore via Meta-Reinforcement Learning (2018, background): “eedy exploration [15], or perhaps a simple scheme like Boltzmann exploration [6]. As a result of these shortcomings, a variety of new approaches to exploration in deep RL have recently been suggested [34, 9, 31, 18, 4, 19, 17, 8, 23, 17, 8, 26, 25, 32, 33, 12, 24]. In spite of these numerous efforts”
Evolved Policy Gradients (2018, background): “This ability is likely required to achieve truly intelligent agents (28).”
Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes (2008, methodology): “Towards this end, in the spirit of our previous work since 1990 [57, 58, 61, 59, 60, 108, 68, 72, 76, 81, 88, 87, 89] we split the reward signal r ( t ) into two scalar real-valued components: r ( t ) = g ( r ext ( t ) , r int ( t )) , where g maps pairs of real values to real values, e.g., g ( a,…”
Maximizing Fun by Creating Data with Easily Reducible Subjective Complexity (2013, methodology): “According to the theory of creativity, in both cases, the data is not surprising but boring (Schmidhuber 2002a, 2007c) as it does not allow for further encoding progress—there is no novel pattern.”
Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints (2013, methodology): “Indeed, we measure the evolution of the relative frequency of the use of the different actuators and motor primitives (analogous measures were also used to study the behavioural structure of intrinsically motivated exploration in (Schmidhuber, 2002).”
Optimal Artiﬁcial Curiosity, Creativity, Music, and the Fine Arts (2005, methodology): “Towards this end, in the spirit of our previous work [23, 22,49, 29, 31], we split the reward signal r ( t ) into two scalar real-valued components: r ( t ) = g ( r ext ( t ) , r int ( t )) , where g maps pairs of real values to real values, e.g., g ( a, b ) = a + b .”
Empowerment - an Introduction (2013, background): “Artificial curiosity (Schmidhuber 2002, 1991) is one of the earlier models, where an agent receives an internal reward depending on how “boring” the environment is which it currently tries to learn.”
Abandoning Objectives: Evolution Through the Search for Novelty Alone (2011, background): “…related to the concept of curiosity and seeking novelty in reinforcement learning research and developmental robotics (Kaplan and Hafner, 2006; Oudeyer et al., 2005, 2007; Schmidhuber, 2003, 2006), though novelty search operates on an evolutionary timescale and is motivated by open-ended evolution.”
Evolving plastic neural networks with novelty search (2010, background): “This idea is also related to the concept of curiosity and seeking novelty in reinforcement learning research (Schmidhuber, 2003, 2006).”
Model-Based Active Exploration (2018, methodology): “Our work is inspired by the framework developed in Schmidhuber (1997, 2002), in which two adversarial reward-maximizing modules called the left brain and the right brain bet on outcomes of experimental protocols or algorithms they have collectively generated and agreed upon.”
Artificial General Intelligence: Concept, State of the Art, and Future Prospects (2009, background): “Many of the foundations of this research area were laid by Juergen Schmidhuber’s work in the 1990s (Schmidhuber, 1991b,a, 1995, 2003), but now with more powerful computers and robots the area is leading to more impressive practical demonstrations.”
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) (2010, background): “Note that some actions may actually trigger learning algorithms that compute changes of the compressor and the controller’s policy, such as in Section III-D [79].”