Maximum margin planning

By Ashesh Jain, Michael Hu, Nathan D. Ratliff, Drew Bagnell, Drew Bagnell, Martin A. Zinkevich. 2006

815 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Software, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Software

In Reinforcement Learning

Citations in the corpus

Learning Trajectory Preferences for Manipulators via Iterative Improvement (2013, methodology): “We quantify closeness of trajectories by the l2−norm of difference in their feature representations, and choose the regularization parameter C for training SSVM in hindsight, to give an unfair advantage to MMP-online.”
Value Iteration Networks (2016, methodology): “Previous approaches to the structured-prediction problem, such as inverse reinforcement learning (IRL) address the underlying planning process in the learning algorithm (Abbeel & Ng, 2004; Ratliff et al., 2006; Ziebart et al., 2008; Ross et al., 2011). Our contribution is therefore complimentary to ”
Certifying Some Distributional Robustness with Principled Adversarial Training (2017, background): “Robust optimization and adversarial training The standard robust-optimization approach minimizes losses of the form supu∈U `(θ; z+u) for some uncertainty set U (Ratliff et al., 2006; Ben-Tal et al., 2009; Xu et al., 2009).”
Reinforcement learning in robotics: A survey (2013, methodology): “This reward function does not necessarily correspond to the true reward function, but provides guarantees on the resulting performance of learned behaviors (Abbeel and Ng, 2004; Ratliff et al., 2006b).”
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation (2018, background): “Imitation learning [1, 12, 19, 28, 44] utilizes the indicator signal to estimate a reward function for training robots by supposing the indicator signal is optimal, which is not in line with the clinical reality.”
End-to-End Robotic Reinforcement Learning without Reward Engineering (2019, background): “Data-driven approaches for reward specification [29, 1, 11, 17, 12, 48, 33, 9, 27, 4, 18] seek to overcome this issue, but typically require demonstration data to acquire rewards.”
Learning from Physical Human Corrections, One Feature at a Time (2018, methodology): “Interestingly, (7) is the same update rule from co-active learning [11] and online maximummargin planning [18], shown by [3] to be an approximate solution to the partially observableMarkov decision process that treats θ∗ as the hidden state and optimizes the cost parametrized by θ∗.”
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (2016, methodology): “…with unknown dynamics on real robots (Boularias et al., 2011; Kalakrishnan et al., 2013) and some have proposed using nonlinear cost functions (Ratliff et al., 2006; 2009; Levine et al., 2011), to our knowledge no prior method has been demonstrated that can provide all of these beneﬁts in the…”
Cooperative Inverse Reinforcement Learning (2016, background): “gorithms can be derived as state estimation for a best response to different ˇH, where the state includes the unobserved reward parametrization . Ng & Russell (2000), Abbeel & Ng (2004), and Ratliff et al. (2006) compute constraints that characterize the set of reward functions so that the o”
Third-Person Imitation Learning (2017, background): “Recently, there has been a signiﬁcant amount of work on IRL, and current algorithms can infer a reward function from a very modest number of demonstrations (e.g,. Abbeel & Ng (2004); Ratliff et al. (2006); Ziebart et al. (2008); Levine et al. (2011); Ho & Ermon (2016); Finn et al. (2016)).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “…additional objective functions to be optimized, such as margin between the optimal policy and others [Ng and Russell, 2000, Abbeel and Ng, 2004, Ratliff et al., 2006b,a, 2009, Silver et al., 2010] and to maximize the entropy [Ziebart et al., 2008, Ziebart, 2010, Kitani et al., 2012, Shiarlis…”
Slow Learners are Fast (2009, background): “This occurs, for instance, in the case of conditional random fields [15, 18], in planning [14], and in ranking [19].”
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning (2017, background): “Previous IRL approaches have included maximum margin approaches (Abbeel & Ng, 2004; Ratliff et al., 2006), and probabilistic approaches such as Ziebart et al.”
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction (2017, methodology): “These methods (Abbeel & Ng, 2004; Syed et al., 2008; Ratliff et al., 2006; Ziebart et al., 2008; Finn et al., 2016; Ho & Ermon, 2016) learn either a policy ˆ π ∗ or ˆ Q ∗ from a ﬁxed-size dataset pre-collected from the oracle.”
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (2018, methodology): “To recover the underlying policy, IRL performs an intermediate step of estimating the reward function followed by RL on this function (Abbeel & Ng, 2004; Ratliff et al., 2006).”
Occam's razor is insufficient to infer the preferences of irrational agents (2017, background): “Moreover, supervised learning of long-range and goal-directed behavior is often difficult without the reward function [Ratliff et al., 2006].”
Active Reward Learning (2014, methodology): “To avoid specifying reward functions, Inverse RL (IRL) extracts a reward function from demonstrations [30, 31, 36].”
Unsupervised Perceptual Rewards for Imitation Learning (2016, methodology): “Inverse reinforcement learning can be performed with a variety of algorithms (Ng et al., 2000), ranging from margin-based methods (Abbeel & Ng, 2004; Ratliff et al., 2006) to methods based on probabilistic models (Ramachandran & Amir, 2007; Ziebart et al., 2008).”
Maximum Entropy Deep Inverse Reinforcement Learning (2015, background): “A number of approaches have been proposed to tackle the IRL problem (see, for example, (Abbeel & Ng, 2004), (Neu & Szepesv´ari, 2012), (Ratliff et al., 2006), (Syed & Schapire, 2007)).”