Terrain-adaptive locomotion skills using deep reinforcement learning

By Xue Bin Peng, Glen Berseth, Michiel van de Panne. 2016

267 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Emergence of Locomotion Behaviours in Rich Environments (2017, background): “Terrain-adaptive locomotion with RL has been demonstrated by [28], but they still impose considerable structure on their solution.”
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots (2018, methodology): “It was further improved using a mixture of actor-critic experts [33].”
Maximum a Posteriori Policy Optimisation (2018, background): “Nonetheless, recent years have seen significant progress, with improvements to different aspects of learning algorithms including stability, data-efficiency and speed, enabling notable results on a variety of domains, including locomotion (Heess et al., 2017; Peng et al., 2016), multi-agent behaviou”
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization (2017, background): “The framework has enabled simulated agents to develop highly dynamic motor skills [4], [5], [6], [7].”
Preparing for the Unknown: Learning a Universal Policy with Online System Identification (2017, background): “In recent years, researchers have used deep reinforcement learning to train highly dynamic motor skills in simulated environments that have high-dimensional state and action spaces [23, 31, 30, 18, 20, 10].”
EPOpt: Learning Robust Neural Network Policies Using Model Ensembles (2016, background): “…function approximators like deep neural networks (deep RL) has recently demonstrated remarkable success in a wide range of tasks like games (Mnih et al., 2015; Silver et al., 2016), simulated control problems (Lillicrap et al., 2015; Mordatch et al., 2015b), and graphics (Peng et al., 2016).”
DeepLoco (2017, background): “Learning of high-level controllers for physics-based characters has been successfully demonstrated for several locomotion and obstacle-avoidance tasks [Coros et al. 2009; Peng et al. 2015, 2016].”
Learning symmetric and low-energy locomotion (2018, methodology): “Two major classes of policy search methods used in the previous work include sampling-based methods [Geijtenbeek et al. 2013; Tan et al. 2014] and gradient-based methods [Levine and Koltun 2014; Liu et al. 2016; Peng et al. 2016, 2017].”
MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies (2019, background): “One approach for constructing a hierarchical policy is by using a mixture-of-experts model [15, 19, 28, 31, 42], where the composite policy’s action distribution π(a|s, g) is represented by a weighted sum of distributions from a set of primitives πi(a|s, g) (i.”
Mode-adaptive neural networks for quadruped motion control (2018, methodology): “…as torques, momentum and feasibility [Levine and Popović 2012; Wampler and Popović 2009; Wampler et al. 2014], and torque-based approaches where the body is directly driven by torques [Coros et al. 2011; Liu and Hodgins 2017; Peng et al. 2015, 2016; Raibert and Hodgins 1991; van de Panne 1996].”
DeepMimic (2018, methodology): “In this work, we will use a similar termination condition as [Peng et al. 2016], where an episode is terminated whenever certain links, such as the torso or head, makes contact with the ground.”
Learning scheduling algorithms for data processing clusters (2018, methodology): “We use standard policy gradient methods that have recently been applied to multiple domains [29; 36; 40] for our learning algorithm.”
Catch & Carry (2019, methodology): “Complementary to the use of motion capture, Deep RL approaches enable learning from scratch [Peng et al. 2016; Heess et al. 2017], with behavior emerging from the constraints of the physics and task.”
Legged Locomotion in Challenging Terrains using Egocentric Vision (2022, methodology): “Recent methods skip 248 foothold planning and directly train a deep RL policy that takes the elevation map as input and outputs 249 either low-level motor primitives [9, 70] or raw joint angles [14, 71, 72, 73].”
The Effects of Memory Replay in Reinforcement Learning (2017, background): “Measures other than TD-error are also in literature to weight experience; examples include rewards [15] and the transition property [10].”
Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning (2019, background): “The behaviors and the selection rule can be learned separately [16], [14] or simultaneously [8], [19].”
Learning Agile Robotic Locomotion Skills by Imitating Animals (2020, background): “In particular, reinforcement learning (RL) can be an effective and general approach for developing controllers that can perform a wide range of sophisticated skills [7, 43, 25, 44, 34].”
Learning locomotion skills using DeepRL: does the choice of action space matter? (2016, methodology): “Finite state machines (FSM) have been successfully applied to locomotion for human and nonhuman characters [Coros et al. 2008; Peng et al. 2016; Yin et al. 2007].”