Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes

By J. Urgen Schmidhuber. 2008

208 citations recorded by the source corpus.

I argue that data becomes temporarily interesting by itself to some self-improving, but computationally limited, subjective observer once he learns to predict or compress the data in a better way, thus making it subjectively simpler and more beautiful. Curiosity is the desire to create or discover more non-random, non-arbitrary, regular data that is novel and surprising not in the traditional sense of Boltzmann and Shannon but in the sense that it allows for compression progress because its regularity was not yet known. This drive maximizes interestingness, the first derivative of subjective beauty or compressibility, that is, the steepness of the learning curve. It motivates exploring infants, pure mathematicians, composers, artists, dancers, comedians, yourself, and (since 1990) artificial systems.

Subfields: Computational Theory and Mathematics, Experimental and Cognitive Psychology, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “…or empowerment (Schmidhuber, 1991; Achiam and Sastry, 2017; Schmidhuber, 1990; Klyubin, Polani, and Nehaniv, 2005; Kaplan and Oudeyer, 2006; Schmidhuber, 2008; Frank, Leitner, Stollenga, Förster, and Schmidhuber, 2014; Mohamed and Rezende, 2015; Bellemare, Srinivasan, Ostrovski, Schaul,…”
The Uncertainty Bellman Equation and Exploration (2017, background): “The resulting algorithm shares an intimate connection to the existing literature of both OFU and intrinsic motivation [31, 30].”
One Big Net For Everything (2018, neutral): “We may call this a consolidation phase or sleep phase [71] or dream phase or regularity detection phase.”
Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings (2018, background): “Hierarchical reinforcement learning is a well studied area in reinforcement learning (Sutton et al., 1999; Dayan & Hinton, 1992; Schmidhuber, 2008; Parr & Russell, 1997; Dietterich, 1998).”
Learning Awareness Models (2018, background): “…works include Oudeyer & Kaplan (2008; 2009); Sequeira et al. (2011); Still & Precup (2012); Bellemare et al. (2016); Martius et al. (2013); Schmidhuber (2008); Mohamed & Rezende (2015); Haber et al. (2018b;a) Some of the ideas here, in particular the notion of choosing actions speciﬁcally…”
On scientific understanding with artificial intelligence (2022, background): “[103] Jürgen Schmidhuber, “Driven by compression”
Model-Based Active Exploration (2018, background): “Schmidhuber (2009) suggested a general form of learning progress as compression progress which can be used as an extra intrinsic reward for curious RL systems.”
On the influence of sensor morphology on eye motion coordination (2010, background): “between the agent and the environment in the mathematical definition of recent developmental algorithms [4]-[8], these algorithms do not take in account the interrelations among the sensor system, the body, the environment, and the controller.”
Intrinsically Motivated Learning in Natural and Artificial Systems (2013, background): “6Schmidhuber (2009) would argue that it is the other way around—that control is a result of behavior directed to improve predictive models, which in this author’s opinion is at odds with what we know about evolution.”
Unsupervised Curricula for Visual Meta-Reinforcement Learning (2019, background): “A large body of prior work has studied exploration and intrinsic motivation objectives [51, 48, 43, 22, 8, 5, 35, 42].”
Unsupervised Meta-Learning for Reinforcement Learning (2018, background): “Exploration methods that seek out novel states are also closely related to goal generation methods (Pathak et al., 2017; Schmidhuber, 2009; Bellemare et al., 2016; Osband et al., 2016; Stadie et al., 2015), but do not by themselves aim to generate new tasks or learn to adapt more quickly to new…”
The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour (2016, methodology): “…treatment of surprise and its use in motivating exploration and acquisition of world knowledge can be found, for instance, in the work of Schmidhuber (2009b), who ties surprise to the ability to compress data and to predict novel inputs (cf. Luciw, Kompella, Kazerounian, & Schmidhuber,…”
Meta-learning curiosity algorithms (2020, background): “, 2018), learning progress (Oudeyer et al., 2007; Schmidhuber, 2008; Azar et al., 2019), generating diverse skills (Eysenbach et al.”
Maximizing Fun by Creating Data with Easily Reducible Subjective Complexity (2013, methodology): “2.6; Hutter 2005; Schmidhuber 2009e) to maximize not only external reward for achieving goals such as the satisfaction of hunger and thirst but also intrinsic reward or wow-effect reward for learning a better data encoder or model, by creating/discovering/learning novel patterns in the growing…”
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) (2010, methodology): “They embody approximations of a simple, but general, formal theory of creativity and curiosity and inte restingness and fun, explaining essential aspects of human or no nhuman intelligence, including selective attention, scien e, art, music, humor [85], [92], [97], [96], [94].”
Hierarchical consciousness: the Nested Observer Windows model (2024, background): “In principle, other characterizations of information integration/compression, e.g. (Fanelli 2019) or (Schmidhuber 2008), might be equally, or perhaps even more amenable to the task.”