Boosting Structured Prediction for Imitation Learning

By Nathan D. Ratliff, David M. Bradley, J. Bagnell, Joel E. Chestnutt. 2006

181 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (2010, methodology): “In future work, we will consider more sophisticated strategies than simple greedy forward decoding for structured prediction, as well as using base classifiers that rely on Inverse Optimal Control (Abbeel and Ng, 2004; Ratliff et al., 2006) techniques to learn a cost function for a planner to aid pr”
Reinforcement learning in robotics: A survey (2013, methodology): “Another example is Crusher (Ratliff et al., 2006a), an outdoor robot, where the human designer was interested in a combination of minimizing time and risk to the robot.”
Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “In the Imitation Learning (IL) setting (Pomerleau, 1991; Bagnell et al., 2007; Ross and Bagnell, 2010), the environment does not come with a reward, but instead with a number of demonstrations which are trajectories of an agent interacting with the environment of interest.”
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (2016, background): “o impose structure on the cost, they substantially increase the engineering burden. Several methods have proposed to learn nonlinear costs using Gaussian processes (Levine et al., 2011) and boosting (Ratliff et al., 2007; 2009), but even these methods generally operate on features rather than raw st”
Third-Person Imitation Learning (2017, background): “p; Ng, 2004; Ratliff et al., 2006; Ramachandran & Amir, 2007; Ziebart et al., 2008; Boularias et al., 2011; Kalakrishnan et al., 2013; Doerr et al., 2015), or could also involve feature learning (Ratliff et al., 2007; Levine et al., 2011; Wulfmeier et al., 2015; Finn et al., 2016; Ho & Ermon”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “In GPIRL, the kernel hyperparameters θ define the shape of the reward function, manually chosen feature coordinates φu correspond to input coordinates, outputs correspond to demonstrated actions, and a GP models the probability distribution over true actions u.”
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning (2017, methodology): “networks, but only evaluate on simple domains where analytically solving the problem with value iteration is tractable. Previous methods which aim to learn nonlinear cost function have used boosting (Ratliff et al., 2007) and gaussian processes (Levine et al., 2011), but still suffer from the featur”
Imitation learning for locomotion and manipulation (2007, methodology): “[1], [2] This multi-class imitation learning approach is especially suited to robot applications because demonstration provides a natural method for an operator to specify optimality as well as to specify actions that the operator considers as “close” or equivalent (due to symmetries, for example).”
CHOMP: Covariant Hamiltonian optimization for motion planning (2013, background): “CHOMP, nevertheless, can exploit the parameter-performance mapping to train the cost function in a manner reminiscent of imitation learning techniques [62, 63].”
Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion (2007, background): “In addition, there has been subseque nt work [12] that extends the maximum”
Learning to search: Functional gradient techniques for imitation learning (2009, methodology): “In prior work ( Ratliff et al. 2006b ), we developed a technique that utilized the linear MMP algorithm in the inner loop of the functional gradient algorithm to ensure adherence to the positivity constraints.”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, methodology): “Behavioral cloning is often used as a stand-alone learning method [316, 19, 390, 288, 55, 6, 96, 151, 327], as well as a way to provide a better starting point for reinforcement learning [289], though this additionally requires the specification of a reward function for RL to optimize.”
Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy (2010, background): “A feature boosting approach (Ratliff et al., 2007) partially addresses this sub-optimal behavior problem by creating a larger (non-linear) class of features to explain demonstrated behavior as optimal.”
Online) Subgradient Methods for Structured Prediction (2007, methodology): “The method studied in ( Ratli et al., 2006c ), is built atop a maximum margin structured classification algorithm applied to learning MDPs, known as Maximum Margin Planning (MMP).”
A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress (2018, methodology): “A generalization of mmp that focuses on IRL when the feature vector is known to be insufficient to explain the expert’s behavior is mmpboost [75].”
Contextual classification with functional Max-Margin Markov Networks (2009, methodology): “The contribution of this paper is to adapt a recent functional gradient technique [20] for learning such random fields.”