Curious model-building control systems

By J. Urgen Schmidhuber. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

What can the brain teach us about building artificial intelligence? (2016, background): “ AI problems. These mechanisms include the interaction between several systems that guide active exploration in large and open environments: curiosity, intrinsically motivated reinforcement learning (Schmidhuber, 1991; Oudeyer et al., 2007; Barto, 2013), and goal exploration (Baranes and Oudeyer, 20”
Recurrent World Models Facilitate Policy Evolution (2018, methodology): “In particular, we can augment the reward function based on improvement in compression quality [77, 80, 81, 83].”
Prioritized Experience Replay (2015, background): “Some transitions may not be immediately useful to the agent, but might become so when the agent competence increases (Schmidhuber, 1991).”
Toward Causal Representation Learning (2021, background): “Particularly relevant for causal leaning are generative world models that capture some of the causal relations underlying the environment and serve as Lorenzian imagined spaces (see I NTRODUCTION above) to train RL agents [48], [99], [127], [178], [214], [231], [248], [268], [271].”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “). From a modelling perspective, it has been proposed that the intrinsically motivated exploration of the environment, e.g., driven by the maximization of the learning progress (Oudeyer et al., 2007; Schmidhuber, 1991), can lead to the self-organization of human-like developmental structures where t”
Weight Agnostic Neural Networks (2019, background): “…single WANN capable of encoding many different useful tasks in its environment, one might consider developing a WANN with a strong intrinsic bias for intrinsic motivation [82, 84, 94], and continuously optimize its architecture to perform well at pursuing novelty in an open-ended environment [60].”
Deep learning in neural networks: An overview (2014, methodology): “, Schmidhuber, 2006a, 2013b) formalizes driving forces and value functions behind such curious and exploratory behavior: A measure of the learning progress of M becomes the intrinsic reward of C (Schmidhuber, 1991a); compare (Oudeyer, Baranes, & Kaplan, 2013; Singh, Barto, & Chentanez, 2005).”
Towards Causal Representation Learning (2021, background): “world models that capture some of the causal relations underlying the environment and serve as Lorenzian imagined spaces (see INTRODUCTION above) to train RL agents [127, 248, 98, 47, 271, 178, 232, 214, 268].”
Causally Correct Partial Models for Reinforcement Learning (2020, methodology): “One approach to capture this ability is via model-based reinforcement learning (MBRL) (Munro, 1987; Werbos, 1987; Nguyen & Widrow, 1990; Schmidhuber, 1991).”
Shaping Belief States with Generative Environment Models for RL (2019, background): “The idea of learning general world models to support decision-making is probably one of the most pervasive ideas in AI research, [30, 11, 41, 42, 14, 43, 44, 45, 46, 33, 30, 47, 48, 3, 49, 50, 51, 52].”
World Models (2018, methodology): “Using RNNs to develop internal models to reason about the future has been explored as early as 1990 in a paper called Making the World Differentiable (Schmidhuber, 1990a), and then further explored in (Schmidhuber, 1990b; 1991a; 1990c).”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Interestingly, by rewarding novel behaviors, novelty search validates the importance of exploration or curiosity, previously proposed in Schmidhuber (1991, 2006), also from an evolutionary viewpoint.”
Active learning of inverse models with intrinsically motivated goal exploration in robots (2013, positive): “Two communities of researchers, the first one in reinforcement learning [38,39,33,40], the second one in developmental robotics [41–43 As argumented in [39,33,51,52], architectures based on intrinsically motivated learning can be conceptualized as active learning mechanisms which, in addition to…”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “Prioritized experience replay is based on the assumption that some transitions between experiences may be more surprising to the agent and that some experiences might not be immediately relevant to an agent and should be replayed at a later point during training (Schmidhuber, 1991).”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, methodology): “This is generally achieved by defining a heuristic based on some measurement of surprise, information gain, compression, or empowerment (Schmidhuber, 1991; Achiam and Sastry, 2017; Schmidhuber, 1990; Klyubin, Polani, and Nehaniv, 2005; Kaplan and Oudeyer, 2006; Schmidhuber, 2008; Frank, Leitner,…”
Continual Learning of Control Primitives: Skill Discovery via Reset-Games (2020, background): “Autonomous acquisition of skills is related to the learning with intrinsic motivation, which considers the unsupervised setting where an agent must learn without extrinsic reward [39, 9, 26, 4, 3].”
Exploration by Random Network Distillation (2018, methodology): “Another class of exploration methods rely on errors in predicting dynamics (Schmidhuber, 1991b; Stadie et al., 2015; Achiam & Sastry, 2017; Pathak et al., 2017; Burda et al., 2018).”
Reinforcement Learning: A Survey (1996, methodology): “…several reinforcement learning algorithms including the interval exploration method (Kaelbling, 1993b) (described shortly), the exploration bonus in Dyna (Sutton, 1990), curiosity-driven exploration (Schmidhuber, 1991a), and the exploration mechanism in prioritized sweeping (Moore & Atkeson, 1993).”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, background): “One line in this direction is curiosity-driven exploration (Schmidhuber, 1991).”