Apprenticeship learning using linear programming

By Umar Syed, Michael H. Bowling, Robert E. Schapire. 2008

294 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Deep Q-learning From Demonstrations (2017, methodology): “ovide additional feedback to the agent. Another popular paradigm is to setup a zero-sum game where the learner chooses a policy and the adversary chooses a reward function (Syed & Schapire, 2007; Syed et al., 2008; Ho & Ermon, 2016). Demonstrations have also been used for inverse optimal con”
Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards (2017, background): “For instance, in [16, 17] the inverse RL problem is cast into a two-player zero-sum game where one player chooses policies and the other chooses reward functions.”
Learning from Demonstrations for Real World Reinforcement Learning (2017, background): “Another popular paradigm is to setup a zero-sum game where the learner chooses a policy and the adversary chooses a reward function (Syed & Schapire, 2007; Syed et al., 2008; Ho & Ermon, 2016).”
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction (2017, methodology): “These methods (Abbeel & Ng, 2004; Syed et al., 2008; Ratliff et al., 2006; Ziebart et al., 2008; Finn et al., 2016; Ho & Ermon, 2016) learn either a policy π̂∗ or Q̂∗ from a fixed-size dataset pre-collected from the oracle.”
Inverse Reward Design (2017, methodology): “Trajectory optimization in this case can be done via the linear programming approach described in Syed et al. (2008).”
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (2018, methodology): “Following Syed et al. (2008), and by treating imitation learning as an occupancy matching problem, Ho & Ermon (2016) proposed a Generative Adversarial Imitation Learning (GAIL) framework for learning a policy from demonstrations, which bypasses the need to recover the expert’s reward function.”
Model-Free Imitation Learning with Policy Optimization (2016, methodology): “Comparing against globally optimal methods As mentioned in Section 3.1, LPAL (Syed et al., 2008) ﬁnds a global optimum for the apprenticeship problem (3) with C convex in ﬁnite state and action spaces.”
A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress (2018, background): “[65] presents a linear program imitating the expert’s policy, that makes solving MDP less expensive in large state spaces with many basis functions (φ for RE = w φ).”
Relative Entropy Inverse Reinforcement Learning (2011, methodology): “Along these lines, both the LPAL algorithm (Syed et al., 2008) and the Maximum Entropy algorithm (Ziebart et al., 2008, 2010) used an MDP model for calculating a probability distribution on the state-actions.”
Bayesian Robust Optimization for Imitation Learning (2020, methodology): “To address negative side-effects, most prior work on safe inverse reinforcement learning takes a minmax approach and seeks to optimize a policy with respect to the worst-case reward function [23, 25, 53]; however, treating the world as if it is completely adversarial (e.”
Energy-Based Imitation Learning (2020, background): “Formally, denoting the set of policies as Π and the set of valid occupancy as measures D , {ρπ > 0 : π ∈ Π}, Lemma 1 shows the one-to-one correspondence between Π and D: Lemma 1 (Theorem 2 of [44]).”
Policy Gradient Bayesian Robust Optimization for Imitation Learning (2021, methodology): “…function of an MDP. Prior work on robust imitation learning has primarily focused on maxmin approaches which seek to optimize a policy for an adversarial worst-case reward function (Syed et al., 2008; Ho & Ermon, 2016; Regan & Boutilier, 2009; HadfieldMenell et al., 2017; Huang et al., 2018).”
Policy Optimization with Demonstrations (2018, methodology): “Following this line, Syed & Schapire (2008); Syed et al. (2008) casted IRL problems as a two-player zero-sum game which is then solved by alternating between ﬁtting the reward function and selecting the policy.”
Boosted and reward-regularized classification for apprenticeship learning (2014, methodology): “This problem is used as benchmark in the IRL literature [1, 21, 9].”
Generative Adversarial Imitation Learning (2016, methodology): “Our approach builds upon a vast line of work on IRL [29, 1, 27, 26], and hence, just like IRL, our approach does not interact with the expert during training.”
Besting the Quiz Master: Crowdsourcing Incremental Classification Games (2012, methodology): “To learn a policy, we use a standard reinforcement learning technique (Langford and Zadrozny, 2005; Abbeel and Ng, 2004; Syed et al., 2008): given a representation of the state space, learn a classifier that can map from a state to an action.”
Inferring The Latent Structure of Human Decision-Making from Raw Visual Inputs (2017, background): “AL, on the other hand, has the advantage of learning a reward function that can be used to score entire trajectories (Abbeel & Ng, 2004; Syed et al., 2008; Ho et al., 2016), but is typically expensive to run because it requires solving a reinforcement learning (RL) problem inside a learning loop.”