Active Learning for Reward Estimation in Inverse Reinforcement Learning

By Manuel Lopes, Francisco S. Melo, Luis Montesano. 2009

217 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

End-to-End Robotic Reinforcement Learning without Reward Engineering (2019, background): “Recent work on active learning for inverse RL has sought to reduce the required number of demonstrations [2, 6, 25, 5], but still requires some number of demonstrations to be provided manually.”
An Algorithmic Perspective on Imitation Learning (2018, neutral): “[2012], Lopes et al. [2009] proposed methods for IRL with active learning.”
Active Reward Learning (2014, background): “[24] propose a framework where IRL is combined with active learning such that the agent can decide when and where to ask for demonstrations.”
Maximum Entropy Deep Inverse Reinforcement Learning (2015, background): “A number of early works from (Ziebart et al., 2008), (Abbeel & Ng, 2004), (Lopes et al., 2009) and (Ratliff et al., 2006), ex- ar X iv :1 50 7.”
Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges (2010, methodology): “Nevertheless, examples that consider robotic problems already exist for a large variety of problems: mapping [21], [22], reinforcement learning [23], body schema learning [25], imitation [26], [27], exploration of objects and body properties [7], manipulation [28], among many others.”
Active Reward Learning from Critiques (2018, methodology): “[15], which asks for demonstrations at states with high entropy over actions.”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, methodology): “Most notably, the IRL problem is fundamentally ill-posed—infinitely many reward functions exist that result in the same optimal policy [275].”
Abstraction Levels for Robotic Imitation: Overview and Computational Approaches (2010, background): “Active learning approaches applied to imitation are very recent [81, 132].”
A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress (2018, background): “[33] allow the learner to query the expert for actions at select states.”
Active reward learning with a novel acquisition function (2015, background): “[26] propose a framework where IRL is combined with active learning such that the agent can decide when and where to ask for demonstrations.”
Comparing Action-Query Strategies in Semi-Autonomous Agents (2011, methodology): “We conducted our first set of experiments in the Puddle World, as used in the evaluations of Lopes et al. (2009).”
Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications (2018, methodology): “Our ﬁrst application is to provide a lower bound on the sample complexity of learning a reward function via active queries (Lopes, Melo, and Montesano 2009; Cui and Niekum 2017; Brown, Cui, and Niekum 2018).”
Apprenticeship Learning About Multiple Intentions (2011, background): “In Active Learning (Lopes et al., 2009), transitions are provided dynamically.”
The strategic student approach for life-long exploration and learning (2012, background): “A third family of approaches have considered internal mechanisms for active choice of either training examples [8]– [10], tasks to be explored [11], learning methods to be used [12], [13] or questions to pose to a teacher [14].”
Active Learning and Intrinsically Motivated Exploration in Robots : Advances and Challenges (2010, methodology): “Nevertheless, examples that consider robotic problems already exist for a large variety of problems: mapping [21], [22], reinforcement learning [23], body schema learning [25], imitation [26], [27], exploration of objects and body properties [7], manipulation [28], among many others.”
Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs (2008, unknown): “Much of the work in inverse reinforcement learning assumes that states are fully-observable [18, 47, 48, 49, 50].”
Towards Resolving Unidentifiability in Inverse Reinforcement Learning (2016, background): “Previous applications of active learning to IRL have considered settings where, in a single environment , the learner can query the agent for its action in some state [8], or for information about its reward [13].”