Learning to search: Functional gradient techniques for imitation learning

By Nathan D. Ratliff. 2009

268 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Colorful Image Colorization (2016, methodology): “Deshpande et al. recently established a benchmark for colorization using a subset of the SUN dataset and reported top results using an algorithm based on LEARCH [29].”
Learning Dynamic Robot-to-Human Object Handover from Human Feedback (2016, background): “One approach is inverse reinforcement learning (IRL), also called inverse optimal control, which learns a reward function from expert demonstration [22, 24].”
Value Iteration Networks (2016, background): “The success of reactive policies in sequential problems is due to the learning algorithm , which essentially trains a reactive policy to select actions that have good long-term consequences in its training domain.”
Generative Adversarial Imitation Learning (2016, background): “Accordingly, IRL has succeeded in a wide range of problems, from predicting behaviors of taxi drivers [29] to planning footsteps for quadruped robots [20].”
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (2016, methodology): “…of a cost (Todorov, 2006), solving an approximate local con-trol problem (Levine & Koltun, 2012; Dragan & Srinivasa, 2012), generating a discrete graph of states (Byravan et al., 2015; Monfort et al., 2015), or only obtaining an optimal trajectory rather than policy (Ratliff et al., 2006; 2009).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “…the optimal trajectory τ ∗i = arg min c̃iµ Compute the state-action frequency couts µ∗i end for Compute the subgradient g ∈ ∂LMMP(w) w ← w − αtg (Optional) Project w on to any additional constraint t ← t + 1 end while return w stochastic policy and approximations in planning [Ratliff et al., 2009].”
Active Reward Learning (2014, methodology): “To avoid specifying reward functions, Inverse RL (IRL) extracts a reward function from demonstrations [30, 31, 36].”
Model-Free Imitation Learning with Policy Optimization (2016, background): “Inverse reinforcement learning (IRL) methods (Russell, 1998; Ng & Russell, 2000; Ratliff et al., 2006; Ziebart et al., 2008), which are some of the most successful approaches to imitation learning, assume that the behavior the learner desires to imitate is generated by an expert behaving optimally w”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, unknown): “Subsequent advancements by Bagnell et al. [58] and Ratliff et al. [59] further extend these methodologies to nonlinear hypothesis spaces, employing various functional gradient techniques for enhanced performance.”
Imitation learning for natural language direction following through unknown environments (2013, methodology): “Our learning approach is also similar to search-based structured prediction [10] and Learning to Search [11], which learn classifiers and planners (respectively) parameterized by cost functions.”
Learning Large-Scale Automatic Image Colorization (2015, methodology): “In contrast, we extend LEARCH [17], a framework for learning an objective function from examples, in a manner that allows us to control long spatial scales and provides a positive definite Hessian without difficulty.”
Recent Advances in Robot Learning from Demonstration (2020, methodology): “Depending on the complexity of the problem of interest, IRL methods assume that the reward function is either linear (114, 117–119) or nonlinear (58, 116, 120).”
An Invitation to Imitation (2015, methodology): “[Ratliff et al., 2009 a] blends the advantages of IOC-based methods with methods that directly learn to predict actions.”
Stacked Hierarchical Labeling (2010, methodology): “We repeat this procedure until convergence and then return 𝜙 = ∑ ℎ 𝑡 , where is the step size.”
Differentiable Gaussian Process Motion Planning (2019, methodology): “Machine learning has been used to accelerate motion planning by combining reinforcement learning (RL) with sampling based planning [5], learning cost functions from demonstration [6], learning efﬁcient heuristics [7], and learning collision checking policies [8].”