Intrinsically Motivated Reinforcement Learning

By Satinder Singh, Andrew Barto, Nuttapong Chentanez. 2004

879 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Reward learning from human preferences and demonstrations in Atari (2018, background): “Reward-free learning attempts to avoid reward functions and instead use measures of intrinsic motivation, typically based on information theory, as a training signal (Chentanez et al., 2005; Schmidhuber, 2006; Orseau et al., 2013).”
Learning Parameterized Skills (2012, background): “, [5, 63, 4], where intrinsic motivation signals are used (for the first time, to the best of our knowledge) to autonomously identify and construct a set of skills.”
Bridging the Gap Between Value and Policy Based Reinforcement Learning (2017, background): “, 2016), information theoretic notions of curiosity (Singh et al., 2004; Schmidhuber, 2010), and underappreciated reward exploration (Nachum et al.”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, background): “extrinsic rewards – to drive curiosity within an agent, inﬂuenced by classic ideas from psychology (White, 1959). To sketch out some recurring themes, these novelty signals might be prediction error (Singh et al., 2004; Stadie et al., 2015), value error (Simsek and Barto, 2006), learning progress (S”
Deep learning in neural networks: An overview (2014, methodology): “While HRL frameworks such as Feudal RL (Dayan and Hinton, 1993) and options (Sutton et al., 1999b; Barto et al., 2004; Singh et al., 2005) do not directly address the problem of automatic subgoal discovery, HQ-Learning (Wiering and Schmidhuber, 1998a) automatically decomposes POMDPs (Sec.”
Causality for Machine Learning (2019, background): “ich components are robust. This search can be viewed and potentially analyzed as a form of intrinsic motivation, a concept related to latent learning in Ethology that has been gaining traction in RL (Chentanez et al., 2005). Finally, a large open area in causal learning is the connection to dynamics”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “In the absence of an explicit reward shaping and expert demonstrations, agents can use intrinsic rewards or intrinsic motivation [141] to evaluate if their actions were good or not.”
Curiosity-Driven Exploration by Self-Supervised Prediction (2017, background): “…the agent to perform actions that reduce the error/uncertainty in the agent’s ability to predict the consequence of its own actions (i.e. its knowledge about the environment) (Houthooft et al., 2016; Mohamed & Rezende, 2015; Schmidhuber, 1991; 2010; Singh et al., 2005; Stadie et al., 2015).”
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, background): “[27] explored agents with intrinsic reward structures in order to learn generic options that can apply to a wide variety of tasks.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “This concept is foundational to work on intrinsic curiosity (Schmidhuber, 1991; Chentanez, Barto, and Singh, 2005; Singh, Lewis, and Barto, 2009; Barto, 2013; Achiam and Sastry, 2017) and learning an agent’s own curriculum (Schmidhuber, 2013; Justesen and Risi, 2018; Srinivasan, Bahdanau,…”
Continual Learning of Control Primitives: Skill Discovery via Reset-Games (2020, background): “Autonomous acquisition of skills is related to the learning with intrinsic motivation, which considers the unsupervised setting where an agent must learn without extrinsic reward [39, 9, 26, 4, 3].”
The Uncertainty Bellman Equation and Exploration (2017, methodology): “The resulting algorithm shares a connection to the existing literature of both OFU and intrinsic motivation (Singh et al., 2004; Schmidhuber, 2009; White & White, 2010).”
Emergent Tool Use From Multi-Agent Autocurricula (2019, methodology): “[35] Nuttapong Chentanez, Andrew G Barto, and Satinder P Singh.”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, background): “There is a large body of work on intrinsic motivation (Barto, 2013; Singh et al., 2004; Klyubin et al., 2005; Schmidhuber, 1991) for self-supervised learning agents.”
Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning (2015, background): “…on prediction errors of future states such predicted L 1 change, predicted mode change or probability gain [16], or salient event prediction [23]; and measures based on information-theoretic quantities such as predicted information gain (PIG) [14], causal entropic forces [28] or empowerment…”
Noisy Networks for Exploration (2017, background): “[32] Satinder P Singh, Andrew G Barto, and Nuttapong Chentanez.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “They seem to get intrinsic reward for creating experiments leading to observations that obey a previously unknown law that allows for better compression of the observations—corresponding to the discovery of a temporarily interesting, subjectively novel regularity [224, 239, 241] (compare also [261, ”