Optimal Artiﬁcial Curiosity, Creativity, Music, and the Fine Arts

By J. Urgen Schmidhuber. 2005

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reward learning from human preferences and demonstrations in Atari (2018, background): “Reward-free learning attempts to avoid reward functions and instead use measures of intrinsic motivation, typically based on information theory, as a training signal (Chentanez et al., 2005; Schmidhuber, 2006; Orseau et al., 2013).”
Recurrent World Models Facilitate Policy Evolution (2018, background): “An exciting research direction is to look at ways to incorporate artificial curiosity and intrinsic motivation [61, 64, 77, 80, 81] and information seeking [23, 86] abilities in an agent to encourage novel exploration [47].”
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models (2015, methodology): “have developed a curiosity driven approach to exploration which uses model predictors to aid in control [17].”
Deep learning in neural networks: An overview (2014, background): “The contents of this paper may be used for educational and noncommercial purposes, including articles for Wikipedia and similar sites.”
World Models (2018, methodology): “In particular, we can augment the reward function based on improvement in compression quality (Schmidhuber, 2010; 2006; 1991b; 2015a).”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Interestingly, by rewarding novel behaviors, novelty search validates the importance of exploration or curiosity, previously proposed in Schmidhuber (1991, 2006), also from an evolutionary viewpoint.”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, methodology): “…reward, P OWER P LAY is greedy, always trying to ﬁnd the simplest (easiest to ﬁnd and validate) task to add to the repertoire, or the simplest way of improving the efﬁciency or compressibility of previous solutions, instead of looking further ahead, as a universal RL method [42, 45] would do.”
First Experiments with PowerPlay (2012, methodology): “POWERPLAY for SLIM RNN represents a greedy implementation of central aspects of the Formal Theory of Fun and Creativity [22, 23].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “They seem to get intrinsic reward for creating experiments leading to observations that obey a previously unknown law that allows for better compression of the observations—corresponding to the discovery of a temporarily interesting, subjectively novel regularity [224, 239, 241] (compare also [261, ”
One Big Net For Everything (2018, methodology): “Like in previous work since 2006 [70, 72, 77], to be able to retrain ONE on all observations ever made, we should store ONE’s entire, growing, lifelong sensory-motor interaction history including all inputs and goals and actions and reward signals observed during all successful and failed trials…”
Go-Explore: a New Approach for Hard-Exploration Problems (2019, background): “IM is often motivated by psychological concepts such as curiosity [12, 13] or novelty-seeking [7, 14], which play a role in how humans explore and learn.”
Artificial General Intelligence (2016, methodology): “The nonuniformity of affective interactions can be quantified in terms of information integration, a concept developed by Tononi [13].”
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) (2010, methodology): “They embody approximations of a simple, but general, formal theory of creativity and curiosity and inte restingness and fun, explaining essential aspects of human or no nhuman intelligence, including selective attention, scien e, art, music, humor [85], [92], [97], [96], [94].”
Online Learning with Stochastic Recurrent Neural Networks using Intrinsic Motivation Signals (2017, background): “1 Introduction The idea of developmental robots [1, 2, 3, 4, 5] that adapt autonomously through lifelong-learning is still one of the big challenges in robotics.”
Artificial General Intelligence: Concept, State of the Art, and Future Prospects (2009, background): “Juergen Schmidhuber’s Goedel Machine (Schmidhuber, 2006) operates differently in detail, but the concept is similar.”
An intrinsic value system for developing multiple invariant representations with incremental slowness learning (2013, background): “The recently formulated PowerPlay can be viewed as a greedy variant of the Formal Theory of Creativity.”
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules (2018, background): “…have studied these ideas in the context of curiosity (Schmidhuber, 2010), information gain during active exploration (Storck et al., 1995; Schmidhuber, 2006; Sun et al., 2011; Little and Sommer, 2013; Friston et al., 2016), and via formal definitions of surprise (Shannon, 1948; Storck et…”
Intrinsically motivated information foraging (2010, background): “Perhaps the closest to our work are those of [25] and [26], who focus on minimizing prediction error as an intrinsic motivation.”