Stochastic policy gradient reinforcement learning on a simple 3D biped

By Russ Tedrake, Teresa Weirui Zhang, H. Seung. 2004

308 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biomedical Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Reinforcement Learning

Citations in the corpus

Trust Region Policy Optimization (2015, neutral): “This is in contrast with most prior methods for learning locomotion, which typically rely on hand-architected policy classes that explicitly encode notions of balance and stepping (Tedrake et al., 2004; Geng et al., 2006; Wampler & Popović, 2009).”
High-Dimensional Continuous Control Using Generalized Advantage Estimation (2015, neutral): “This is in contrast with most prior methods for learning locomotion controllers, which typically rely on hand-architected policy classes that explicitly encode notions of balance and stepping [15, 4, 18].”
End-to-End Training of Deep Visuomotor Policies (2015, background): “They have been applied to a range of robotic tasks, including manipulation [2, 13] and locomotion [5, 7, 15, 39].”
Reinforcement learning in robotics: A survey (2013, methodology): “For example, a visionbased mobile robot docking task can be learned faster with such a basic behavior than using Q-learning alone, as demonstrated by Martínez-Marín and Duckett (2005) Providing hand-coded, stable initial gaits can significantly help in learning robot locomotion, as shown on a six-le”
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots (2018, background): “Much research has applied reinforcement learning to locomotion tasks [2, 47, 11, 22, 31].”
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation (2017, background): “While more traditional RL success stories in robotics (e.g., [45], [46], [47]) work in state space, more recent work has been able to learn deep neural net policies from pixels to actions (e.g., [17], [18], [20]).”
Deep spatial autoencoders for visuomotor learning (2015, methodology): “Reinforcement learning and policy search methods have been applied to perform a variety of robotic tasks, such as table tennis [9], object manipulation [10], [11], and locomotion [12], [13], [14], [15].”
Collective robot reinforcement learning with distributed asynchronous guided policy search (2016, background): “Policy search techniques show promising ability to learn feedback control policies for robotic tasks with highdimensional sensory inputs through trial and error [1, 2, 3, 4].”
Path integral guided policy search (2016, methodology): “Reinforcement learning (RL) and policy search methods have shown considerable promise for enabling robots to automatically learn a wide range of complex skills [1, 2, 3, 4], and recent results in deep reinforcement learning suggest that this capability can be extended to learn nonlinear policies tha”
Learning contact-rich manipulation skills with guided policy search (2015, methodology): “Previous policy search methods in robotics generally use one or more of the following techniques to achieve good performance and reasonable training times: the use of example demonstrations [15], [9], [16], the use of a prior controller that allows the policy to only set high level targets, such as ”
Learning Dexterous Manipulation Policies from Experience and Imitation (2016, methodology): “Indeed RL has been successfully applied to a range of robotic tasks (Tedrake, Zhang and Seung 2004; Kober, Oztop and Peters 2010; Pastor, Hoffmann, Asfour and Schaal 2009; Deisenroth, Rasmussen and Fox 2011), however dimensionality and sample complexity have presented major challenges (Kober,…”
Learning agile and dynamic motor skills for legged robots (2019, background): “Direct application of learning methods to physical legged systems is therefore complicated and has only been demonstrated on relatively simple and stable platforms [19] or in a limited context [20].”
A Tour of Reinforcement Learning: The View from Continuous Control (2018, background): “Moreover, seminal work by Tedrake, Zhang, and Seung demonstrated that direct policy search could rapidly ﬁnd feedback control policies for certain constrained legged robot designs [78].”
Deep reinforcement learning for tensegrity robot locomotion (2016, methodology): “These methods, however, typically require hand-engineered policy classes, such as a linear function approximator using a set of handdesigned features as input [2].”
Guided Policy Search (2013, methodology): “Previous locomotion methods often rely on hand-crafted components to introduce prior knowledge or ensure stability (Tedrake et al., 2004; Whitman & Atkeson, 2009), while our method used only general purpose neural networks.”
Toward fast policy search for learning legged locomotion (2012, background): “There have been a series of applications of machine learning approaches in legged locomotion such as Imitation Learning [20] and Reinforcement Learning (RL) [24], [22].”
Learning Locomotion Skills for Cassie: Iterative Design and Sim-to-Real (2019, methodology): “Reinforcement learning has also been applied to bipedal locomotion, with results on physical robots demonstrated on either 2D bipeds [21, 22] or small bipeds with large feet [23, 24].”
HumanPlus: Humanoid Shadowing and Imitation from Humans (2024, background): “…humanoid robots [13, 32, 37, 54, 65, 89], learning-based meth-ods can achieve robust locomotion performance for humanoids [39, 67, 68, 85, 91, 94, 103] and biped robots [4, 42, 44, 45, 83, 84, 93, 100] due to their training on highly randomized environments in simulation and their ability to adapt.”
Hierarchical Control Using Networks Trained with Higher-Level Forward Models (2014, background): “Similar ideas have also been applied to robot walking where a target state is deﬁned at a time slice of the walker’s dynamical orbit or Poincar´e return map (Tedrake, Zhang, & Seung, 2004).”
SIMBICON: simple biped locomotion control (2007, methodology): “Searching directly in the often highdimensional parameter space is known as policy search, and this has been applied with some success[Taga et al. 1991; van de Panne and Fiume 1993; Auslander et al. 1995; van de Panne et al. 1994; Tedrake et al. 2004; Sharon and van de Panne 2005].”