Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective

By Satinder Singh, Richard L. Lewis, Andrew Barto, Jonathan Sorg. 2010

456 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Reinforcement Learning, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Reinforcement Learning

In Evolutionary Computation

Citations in the corpus

Value Iteration Networks (2016, background): “In a different study, Singh et al. (2010) used evolutionary algorithms to learn a cost function that accelerates learning.”
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, background): “[26] take an evolutionary perspective to optimize over the space of reward functions for the agent, leading to a notion of extrinsically and intrinsically motivated behavior.”
Emergent Tool Use From Multi-Agent Autocurricula (2019, methodology): “Intrinsic motivation methods have been widely studied in the literature [35, 36].”
Learning to Play with Intrinsically-Motivated Self-Aware Agents (2018, background): “Intrinsic and extrinsic reward structures have been used to learn generic options for a variety of tasks (Chentanez et al., 2005; Singh et al., 2010).”
Human-level performance in 3D multiplayer games with population-based reinforcement learning (2018, methodology): “In our approach, we operationalise the idea that each agent has a dense internal reward function ( 60,61,74 ), by specifying r t = w ( ρ t ) based on the available game points signals ρ t (points are registered for events such as capturing a ﬂag), and, crucially, allowing the agent to learn the…”
Machine Teaching for Bayesian Learners in the Exponential Family (2013, background): “The third thread is the research on better ways to training machine learners such as curriculum learning or easy-to-hard ordering of training items [3, 15, 16], and optimal reward design in reinforcement learning [21].”
Active Reward Learning (2014, background): “The term reward shaping has been used to describe efforts to adapt rewards functions such that the resulting policy remains invariant but learning speed is increased [27, 10, 4, 32] or”
Evolved Policy Gradients (2018, background): “See also (49; 50) and (1) for additional evolutionary perspectives on reward learning.”
Cooperation and communication in multiagent deep reinforcement learning (2016, background): “Early work on optimal rewards (Singh et al., 2010) used an evolutionary approach to create reward functions that lead to success across environments.”
Active reward learning with a novel acquisition function (2015, background): “The term reward shaping describes efforts to adapt reward to increase learning speed while not affecting the policy (Ng et al. 1999; Devlin and Kudenko 2012; Bratman et al. 2012; Singh et al. 2010) or to accelerate learning of a new task (Konidaris and Barto 2006).”
The Routledge Handbook of Philosophy and Science of Addiction (2018, background): “Mathematical modelers have begun to explore the notion that reward can be generated internally, but they still assume that such rewards come from innately programmed outcomes, “hard-wired from the start of the agent’s life” (Singh et al. 2010, p. 73).”
Active Learning and Intrinsically Motivated Exploration in Robots : Advances and Challenges (2010, background): “The complexity of the concept of intrinsic motivation is also illustrated by [29], which shows that searching for hard and fast features for discriminating intrinsic motivation from extrinsic motivation might be impossible.”
Reward Design via Online Gradient Ascent (2010, background): “The optimal reward parameters are given by the solution to the optimal reward problem [16, 17, 18]:”
Learning and exploration in action-perception loops (2013, background): “For example, Singh et al. (2010) have demonstrated that RL guided by saliency, an intrinsic motivation derived from changes in stimulus intensity, can promote the learning of reusable skills.”
Reinforcement learning improves behaviour from evaluative feedback (2015, background): “One can turn to evolutionary optimization 47 to generate reward functions if there is an effective way to evaluate the appropriateness of the resulting behaviour.”