A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems

By E. Todorov, Weiwei Li. 2005

766 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Physical Therapy, Sports Therapy and Rehabilitation, Surgery, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Physical Therapy, Sports Therapy and Rehabilitation

In Surgery

In Reinforcement Learning

Citations in the corpus

Continuous control with deep reinforcement learning (2015, background): “The second baseline is iLQG [14], a planning based solver with full access to the underlying physical model and its derivaties.”
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images (2015, background): “A prominent class of algorithms that aim to solve this problem are model-based locally optimal (stochastic) control algorithms such as iLQG [1, 2], which approximate the general non-linear control problem via local linearization.”
MOReL : Model-Based Offline Reinforcement Learning (2020, methodology): “Furthermore, MBRL can draw upon the rich literature on model-based planning including model predictive control (MPC) [23, 24, 61, 62], search based planning [25, 63], dynamic programming [26, 64], and policy optimization [32, 65, 33, 27, 53].”
Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards (2017, methodology): “For instance, iterative Linear Quadratic Gaussian (iLQG) [18] is a method for nonlinear stochastic systems where the dynamics is known and the reward has to be quadratic (and thus entails hand-crafted task designs).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “Iterative LQR (iLQR) is often employed to con- 108 Behavioral Cloning trol a system of which the dynamics is not accurately known [Todorov and Li, 2005, Abbeel et al., 2010, Tassa et al., 2012]. iLQR learns a linear feedback controller to follow a trajectory through an iterative learning process.”
Stochastic Neural Networks for Hierarchical Reinforcement Learning (2016, background): “GPS trains with iLQG Todorov & Li (2005) in state space, and in parallel trains a neural net that only receives raw sensory inputs that has to agree with the iLQG controller.”
Exploring Model-based Planning with Policy Networks (2019, methodology): “GPS uses iLQG [25, 42, 40] as the local controller, and distill the knowledge into a policy neural network.”
PLATO: Policy learning using adaptive trajectory optimization (2016, methodology): “The details of this maximum entropy variant of iLQG may be found in prior work [19], [20], [14].”
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control (2018, methodology): “However, for problems in continuous control, MPC based on local dynamic programming methods [15, 16] provide an efficient way to approximately realize B , which can be used to accelerate and stabilize value function learning.”
SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning (2018, methodology): “From this, we devise an efﬁcient local model method based on the linear-quadratic regulator (LQR) [8, 40, 24] that utilizes linear models for gradient directions for policy improvement.”
Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control (2017, methodology): “For linear dynamics the proposed method is a generalisation of iLQG [46]: The moment matching transition fMM implicitly linearises the transition dynamics at each time step, whereas in iLQG an explicit local linear approximation is made.”
Generating Legible Motion (2013, background): “IV), echoing earlier works in motion planning [9, 21, 22, 27, 29, 33, 35], now with legibility as an optimization criterion.”
Predictor-Corrector Policy Optimization (2018, methodology): “A classical example of pure model-based methods is optimal control [5, 6, 7, 8], which has recently been extended to model abstract latent dynamics with structured neural networks [9, 10].”
From Motor Learning to Interaction Learning in Robots (2010, background): “The chapter highlights the interesting robustness properties of the resulting framework and highlights deep relationships with the optimal control framework that provides convincing computational principles for motor control [22], [23], [24].”
Probabilistic Differential Dynamic Programming (2014, methodology): “In the last decade, variations of DDP have been proposed in both control and machine learning communities [2][3][4][5][6].”
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning (2018, unknown): “method that automatically encourages learning representations where linear models better ﬁt the data. From this, we devise an efﬁcient local model method based on the linear-quadratic regulator (LQR) [8, 40, 24] that utilizes linear models for gradient directions for policy improvement. Our motivati”
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control (2017, unknown): “control, we would like to do long-term planning using model based techniques such as iterative LQG [23] to leverage the full strength the latent pose space, i.”