Intrinsic Motivation and Reinforcement Learning

By Andrew Barto. 2013

272 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Information Systems, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Information Systems

In Reinforcement Learning

Citations in the corpus

What can the brain teach us about building artificial intelligence? (2016, background): “e interaction between several systems that guide active exploration in large and open environments: curiosity, intrinsically motivated reinforcement learning (Schmidhuber, 1991; Oudeyer et al., 2007; Barto, 2013), and goal exploration (Baranes and Oudeyer, 2013), social learning and natural interact”
Learning Parameterized Skills (2012, background): “, [5, 63, 4], where intrinsic motivation signals are used (for the first time, to the best of our knowledge) to autonomously identify and construct a set of skills.”
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (2017, background): “Finally, for intrinsically difficult games, agents may need some form of intrinsic motivation (Barto, 2013; Oudeyer, Kaplan, & Hafner, 2007) to keep playing despite the apparent impossibility of scoring in the game.”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, background): “Closest to our work is Houthooft et al. (2016)’s variational approach to intrinsic motivation, which is equivalent to a second order Taylor approximation to prediction gain.”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “In this case, it is necessary to introduce additional factors such as intrinsic motivation (Barto, 2013; Oudeyer, Kaplan, & Hafner, 2007), where indicators of learning progress are used as reward signals to encourage exploration (see Section 4.”
Reinforcement Learning in Healthcare: A Survey (2019, neutral): “[26].”
Automated Curriculum Learning for Neural Networks (2017, background): “Doing so brings us into contact with the intrinsic motivation literature (Barto, 2013), where various indicators of learning progress have been used as reward signals to encourage exploration, including compression progress (Schmidhuber, 1991), information acquisition (Storck et al., 1995),…”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “This concept is foundational to work on intrinsic curiosity (Schmidhuber, 1991; Chentanez, Barto, and Singh, 2005; Singh, Lewis, and Barto, 2009; Barto, 2013; Achiam and Sastry, 2017) and learning an agent’s own curriculum (Schmidhuber, 2013; Justesen and Risi, 2018; Srinivasan, Bahdanau,…”
Continual Learning of Control Primitives: Skill Discovery via Reset-Games (2020, background): “Autonomous acquisition of skills is related to the learning with intrinsic motivation, which considers the unsupervised setting where an agent must learn without extrinsic reward [39, 9, 26, 4, 3].”
Variational Bayesian Reinforcement Learning with Regret Bounds (2018, methodology): “In the non-tabular case methods that estimate the pseudo-count have proven successful in some instances, indicating a possibility of combining K-learning with pseudo-counts in a deep RL setting [94, 11, 97, 80, 13, 79].”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, methodology): “See Appendix B for all the hyperparameter values used in the experiments.”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, neutral): “4; compare (Barto, 2013; Dayan, 2013; Nehmzow et al., 2013; Oudeyer et al., 2013)), PowerPlay provably (by design) does not have any problems with online learning – it cannot forget previously learned skills, automatically segmenting its life into a sequence of clearly identified tasks with explicit”
First Experiments with PowerPlay (2012, background): “[1, 4, 13, 11]), P OWER P LAY provably (by design) does not have any problems with online learning—it cannot forget previously learned skills, automatically segmenting its life into a sequence of clearly identiﬁed tasks with explicitly recorded solutions.”
Curiosity Driven Exploration of Learned Disentangled Goal Spaces (2018, background): “More involved methods that value states or actions in terms of novelty, information gain, or prediction errors are limited the presence of “distractors” that cannot be controlled [9, 10, 11, 12, 13].”
Go-Explore: a New Approach for Hard-Exploration Problems (2019, background): “Prior to Go-Explore, the typical approach to sparse reward problems has been intrinsic motivation (IM) [4, 9–11], which supplies the RL agent with intrinsic rewards (IRs) that encourage exploration (augmenting or replacing extrinsic reward that comes from the environment).”
Reinforcement learning for control: Performance, stability, and deep approximators (2018, background): “Other directed exploration methods include those that add to the original rewards an extra exploration-inducing term, called intrinsic reward, when visiting states that are deemed interesting (Barto, 2013).”
How Evolution May Work Through Curiosity-Driven Developmental Process (2016, background): “…2013; Gottlieb et al., 2013; Oudeyer et al., 2007), in models about the evolutionary origins of intrinsic motivation systems (Singh et al., 2010; Barto, 2013), and in recent findings in neuroscience linking intrinsic motivation with attention (Gottlieb et al., 2013), as well as in new formal…”
Intrinsic Motivation for Encouraging Synergistic Behavior (2020, background): “(Barto, 2013; Oudeyer et al., 2007; Bellemare et al., 2016); intrinsic motivation can even be used on its own when no extrinsic reward is provided (Pathak et al.”
Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. (2016, background): “As curiosity-driven spontaneous exploration may have evolved as a mean to acquire information and skills in rapidly changing environments (Barto, 2013), it appears that heuristics based on searching for novelty and complexity are suboptimal (Schmidhuber, 2001; Oudeyer et al., 2007).”