Reward Design via Online Gradient Ascent

By Jonathan Sorg, Satinder Singh, Richard L. Lewis. 2010

113 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement learning in robotics: A survey (2013, background): “In (Sorg et al., 2010; Zucker and Bagnell, 2012), complex policies are derived by adapting a reward function for simple optimal control problems using policy search techniques.”
Inverse Reward Design (2017, background): “Sorg et al. (2010) and subsequent work has studied the computational problem of selecting an optimal proxy reward.”
Evolved Policy Gradients (2018, background): “Other reward learning methods include the work of Guo et al. (14), which focuses on learning reward bonuses, and the work of Sorg et al. (51), which focuses on learning reward functions through gradient descent.”
Self-Correcting Models for Model-Based Reinforcement Learning (2016, unknown): “For instance, Sorg et al. (2010) and Joseph et al. (2013) both pointed out that the most accurate model by this measure is not necessarily the best for planning.”
LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning (2019, background): “The optimal intrinsic reward problem has been introduced in [13] for single agent learning scenarios and studied in some recent RL approaches [14, 15, 16].”
What Can Learned Intrinsic Rewards Capture? (2019, methodology): “…return ( G life ) given any randomly initialised agents and possibly non-stationary tasks drawn from some distribution p ( T ) . been follow-up works (Sorg et al., 2010; Guo et al., 2016) that use a gradient-based method, they consider a non-parameteric policy using Monte-Carlo Tree Search…”
Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition (2018, background): “They observe that a proxy reward that is distinct from the ﬁtness function might be optimal under certain settings, and Sorg et al. (2010) study the problem of how this optimal proxy reward can be selected.”
Generative Adversarial Self-Imitation Learning (2018, background): “[25] introduced optimal reward problem under the motivation that the true reward function defined in the environment may not be optimal for learning, and there exists an optimal reward function that allows learning the desired behavior much quickly.”
Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learning (2019, background): “The optimal reward design problem [35, 36] concerns about ﬁnding the proxy reward function can obtains high expected return according to the true reward function.”
Enforcing Constraints for Time Series Prediction in Supervised, Unsupervised and Reinforcement Learning (2019, background): “The speciﬁcation of the reward function is an important aspect of AC meth-ods and reinforcement learning in general [23, 11].”
Agnostic System Identification for Monte Carlo Planning (2015, background): “Sorg, Lewis, and Singh (2010) and Joseph et al. (2013) both address the agnostic setting by adapting model parameters to directly improve agent performance (rather than model accuracy) via policy gradient algorithms.”
On Learning Intrinsic Rewards for Policy Gradient Methods (2018, background): “Sorg et al. [2010] introduced PGRD (Policy Gradient for Reward Design), a scalable algorithm that only works with lookahead-search (UCT) based planning agents (and hence the agent itself is not a learning-based agent; only the reward to use with the ﬁxed planner is learned).”
Cooperation and communication in multiagent deep reinforcement learning (2016, background): “The recent work of Liu et al (Liu et al., 2014, 2012; Sorg et al., 2010) describe a multiagent architecture where each agent employs a gradient-based algorithm to learn its own reward function.”
Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games (2016, methodology): “Our work builds on PGRD (policy-gradient for reward-design; Sorg et al. 2010a), a method for learning reward-bonus functions for use in planning.”