Optimal control with learned local models: Application to dexterous manipulation

By Vikash Kumar, E. Todorov, Sergey Levine. 2016

252 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Solving Rubik's Cube with a Robot Hand (2019, neutral): “[53, 52] and Falco et al.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Learning dexterous manipulation skills with multi-ﬁngered hands, for which model-based (Kumar et al., 2016; Gupta et al., 2016) and model-free (Popov et al., 2017) DRL algorithms have been proposed and demonstrated in real robotic experiments, is quite challenging.”
MOPO: Model-based Offline Policy Optimization (2020, methodology): “To approach this question, we first hypothesize that model-based RL methods [64, 12, 42, 38, 29, 44] make a natural choice for enabling generalization, for a number of reasons.”
When to Trust Your Model: Model-Based Policy Optimization (2019, background): “Gaussian processes and time-varying linear dynamical systems provide excellent performance in the low-data regime (Deisenroth & Rasmussen, 2011; Levine & Koltun, 2013; Kumar et al., 2016).”
Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning (2019, background): “Recent years have seen reinforcement learning (RL) successfully applied to a number of robotics tasks such as in-hand manipulation [1], grasping [2] and door opening [3].”
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost (2018, positive): “s. In simulation, [10], [11], [12] have shown the ability to synthesize complex behaviors when provided with a ground truth model. This model is rarely available in the real world. Techniques such as [13], [14] get around this problem by learning locally-linear models and deploying model-based RL me”
Learning dexterous in-hand manipulation (2018, background): “Conversely, due to the difficulty in modeling such complex systems, there has also been work in approaches that only train on a physical robot [16, 67, 29, 30].”
Model-Ensemble Trust-Region Policy Optimization (2018, methodology): “…possible model parameterization, namely linear models (Bagnell & Schneider, 2001; Abbeel et al., 2006; Levine & Abbeel, 2014; Watter et al., 2015; Levine et al., 2016a; Kumar et al., 2016), where the model either operates directly over the raw state, or over a feature representation of the state.”
Towards Generalization and Simplicity in Continuous Control (2017, background): “Recent success of RL [2, 8, 9, 10, 11] has been enabled largely due to engineering efforts such as large scale data collection [1, 2, 11] or careful systems design [8, 9] with well behaved robots.”
Dynamics-Aware Unsupervised Discovery of Skills (2019, background): “There also have been multiple approaches addressing the planning component of MBRL including linear controllers for local models [44, 40, 10], uncertainty-aware [12, 22] or deterministic planners [49] and stochastic optimization methods [73].”
Learning Dexterous Manipulation Policies from Experience and Imitation (2016, methodology): “(van Hoof, Hermans, Neumann and Peters 2015) report results for simple in-hand manipulation with a 3-ﬁnger hand, and our work reports learning of simple in-hand manipulation skills, such as rotating a cylinder, using time-varying linear-Gaussian controllers (Kumar, Todorov and Levine 2016).”
Learning to Solve a Rubik’s Cube with a Dexterous Hand (2019, background): “[8] learn a control policy and implement the policy on a physical humanoid robot hand.”
Trends and challenges in robot manipulation (2019, background): “A growing trend in providing training data is to test the algorithms in simulation first and then refine the learning on a real platform; e.g., for learning dexterous in-hand manipulation ( 81 – 83 ).”
A System for General In-Hand Object Re-Orientation (2021, methodology): “Model-based RL works learned a linear [4, 7] or deep neural network [5] dynamics model from the rollout data, and used online optimal control to rotate a pen or Baoding balls on a Shadow hand.”
Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators (2019, methodology): “The state-of-the-art approaches using variations of this idea are concerned with using learned local models [20], compound multistep controllers [21], and deep representation of the control policies [22].”
Manipulators and Manipulation in high dimensional spaces (2016, unknown): “[112] report results for simple in-hand manipulation with a 3-finger hand, and our work reports learning of simple in-hand manipulation skills, such as rotating a cylinder, using time-varying linear-Gaussian controllers [45].”
Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interaction (2019, methodology): “Leveraging real robot data for model learning was proposed in [18], but the method is limited to slow inhand manipulation of a pole.”