A Game-Theoretic Approach to Apprenticeship Learning

By Umar Syed, Robert E. Schapire. 2007

358 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Q-learning From Demonstrations (2017, background): “Another popular paradigm is to setup a zero-sum game where the learner chooses a policy and the adversary chooses a reward function (Syed and Schapire 2007; Syed, Bowling, and Schapire 2008; Ho and Ermon 2016).”
Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards (2017, background): “For instance, in [16, 17] the inverse RL problem is cast into a two-player zero-sum game where one player chooses policies and the other chooses reward functions.”
Learning from Demonstrations for Real World Reinforcement Learning (2017, background): “Another popular paradigm is to setup a zero-sum game where the learner chooses a policy and the adversary chooses a reward function Syed & Schapire (2007); Syed et al. (2008); Ho & Ermon (2016).”
Inverse Reward Design (2017, background): “In inverse reinforcement learning (IRL) (Ng & Russell, 2000; Ziebart et al., 2008; Evans et al., 2016; Syed & Schapire, 2007) the agent observes demonstrations of (approximately) optimal behavior and infers the reward function being optimized.”
Maximum Entropy Deep Inverse Reinforcement Learning (2015, background): “A number of approaches have been proposed to tackle the IRL problem (see, for example, (Abbeel and Ng 2004), (Neu and Szepesvári 2012), (Ratliff, Bagnell, and Zinkevich 2006), (Syed and Schapire 2007)).”
Imitation Learning as f-Divergence Minimization (2019, background): “Thedegeneracy issues of IRL can be alleviated by a game theoretic framework where an adversary selects a reward function and the learner must compete to do as well as the expert [29, 30].”
Model-Free Imitation Learning with Policy Optimization (2016, background): “Game-theoretic approaches Syed & Schapire (2007); Syed et al. (2008) proposed two apprenticeship learning algorithms, Multiplicative Weights Apprenticeship Learning (MWAL) and Linear Programming Apprenticeship Learning (LPAL), that also use basis cost functions, but with the weights restricted to…”
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations (2019, background): “Syed & Schapire (2008) proved that, given prior knowledge about which features contribute positively or negatively to the true reward, an apprenticeship policy can be found that is guaranteed to outperform the demonstrator.”
A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress (2018, methodology): “[25] Multiplicative Weights for Apprenticeship Learning (mwal); the initial model and input are the same in both methods.”
Apprenticeship learning using linear programming (2008, methodology): “Our experiments were run in a car driving simulator modeled after the environments in (Abbeel & Ng, 2004) and ( Syed & Schapire, 2008 ).”
Bayesian Multitask Inverse Reinforcement Learning (2011, methodology): “For reference, we include the performance of MWAL and the imitator.”
Generative Adversarial Imitation Learning (2016, methodology): “[10] using the cost function class Cconvex (10) of Syed and Schapire [26]”
Imitating driver behavior with generative adversarial networks (2017, background): “These BC approaches are conceptually sound [11], but tend to fail in practice as small inaccuracies compound during simulation.”
Towards Resolving Unidentifiability in Inverse Reinforcement Learning (2016, background): “Prior work in IRL has mostly focused on inferring an agent’s reward function from data acquired from a fixed environment [10, 2, 4, 20, 11, 18, 14].”