Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion

By J. Kolter, Pieter Abbeel, Andrew Y. Ng. 2007

166 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement learning in robotics: A survey (2013, methodology): “, 2007), and quadruped locomotion (Ratliff et al., 2006a, 2007; Kolter et al., 2007).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “Konidaris et al. [2011] proposed an online algorithm for constructing skill trees from demonstrations.”
APRIL: Active Preference-learning based Reinforcement Learning (2012, neutral): “Let us discuss Ppl with respect to inverse reinforcement learning (IRL) [1,18].”
Multi-Level Discovery of Deep Options (2017, background): “Since then, the focus of research has shifted towards discovery of the hierarchical structure itself, by: trading off value with description length [30], identifying transitional states [31, 32, 33, 34, 6], inference from demonstrations [8, 35, 5, 12], iteratively expanding the set of solvable…”
SWIRL: A sequential windowed inverse reinforcement learning algorithm for robot tasks with delayed rewards (2018, background): “Early work in hierarchical control demonstrated the advantages of hierarchical structures by handcrafting hierarchical policies (Brooks, 1986) and learning them, given various manual specifications: state abstractions (Dayan and Hinton, 1992; Hengst, 2002; Kolter et al., 2007; Konidaris and Barto, 2”
Learning to search: Functional gradient techniques for imitation learning (2009, background): “Sections 5 and 6 present some of these case studies; more recently, extensions to hierarchical learning problems for quadrupedal locomotion (Kolter et al., 2008) have additionally supported inverse optimal control’s status as a preferred approach to imitation learning.”
Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations (2010, background): “In [11], an apprenticeship learning approach is introduced that has enabled a quadruped robot to traverse challenging, previously unseen terrain [19] and helicopters to autonomously perform aerobatic maneuvers.”
Probabilistic model-based imitation learning (2013, background): “, parking lot navigation [1] or quadruped locomotion [20]).”
Recent Advances in Robot Learning from Demonstration (2020, background): “LfD has also enjoyed some successes with quadrupedal locomotion (122, 180, 181).”
Observational Learning by Reinforcement Learning (2017, methodology): “e behaviour of other agents [16, 17] and then use this, in conjunction with RL techniques, to optimize this inferred function. While these methods have been successfully applied to a variety of tasks [14, 18, 19, 20, 21, 22, 23, 24, 25], one problematic aspect of both these scenarios is that they al”
Neural Programmer-Interpreters (2015, background): “…learning ( e.g. , Dietterich (2000); Andre & Russell (2001); Sutton et al. (1999) and Schaul et al. (2015)), imitation and apprenticeship learning ( e.g. , Kolter et al. (2008) and Rothkopf & Ballard (2013)) and elicitation of options through human interaction (Subramanian et al., 2011).”
A survey of inverse reinforcement learning techniques (2012, background): “Using this idea, the control of a quadruped robot was divided into two levels and the expert can give instructions more easily (Kolter et al., 2008).”
Learning objective functions for manipulation (2013, background): “The performance of these systems is critically dependent on the definition of this objective function, which may require extensive manual tuning in order for the planner or optimal controller to achieve acceptable real-world performance [6].”
Learning manipulation skills from a single demonstration (2018, background): “Many successful applications in different areas have demonstrated the capabilities of this idea, including the learning of quadruped locomotion (Kolter et al., 2008), helicopter acrobatics (Abbeel et al.”
Learning locomotion over rough terrain using terrain templates (2009, methodology): “deviation of the terrain on various spatial scales [4], [9].”