Differential dynamic programming

By H. Rosenbrock, D. Jacobson, David Q. Mayne. 1972

Subfields: Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Optimization & Training

Citations in the corpus

Learning Continuous Control Policies by Stochastic Value Gradients (2015, background): “Writing the noise variables as exogenous inputs to the system to allow direct differentiation with respect to the system state (equation 7) is a known device in control theory [10, 7] where the model is given analytically.”
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images (2015, background): “A prominent class of algorithms that aim to solve this problem are model-based locally optimal (stochastic) control algorithms such as iLQG [1, 2], which approximate the general non-linear control problem via local linearization.”
Reinforcement learning in robotics: A survey (2013, methodology): “This approach combines initial human demonstration of trajectories, machine learning to extract approximate models from multiple trajectories, and classical locally optimal control methods (Jacobson and Mayne, 1970) to achieve state-of-the-art acrobatic flight.”
Cognitive Mapping and Planning for Visual Navigation (2017, methodology): “Unlike past work [12] that uses human operators to implement such macro-actions for such simulation to real transfer, we implement these macro-actions using an iterative linearquadratic regulator (iLQR) controller [41, 55].”
Automatic differentiation in machine learning: a survey (2015, background): “Note the difference from differential dynamic programming (Mayne and Jacobson, 1970) in optimal control.”
Learning dexterous manipulation for a soft robotic hand from human demonstrations (2016, background): “ t)=N (K jtx t +k jt;C jt) where K jt is a feedback term and k jt is an open loop term. Given this form, the maximum entropy objective (Eq. 3), can be optimized using differential dynamic programming [27], [28]. As d i(t¯) is a multivariate Gaussian N (m i;S i), we can rewrite the optimization probl”
Neural probabilistic motor primitives for humanoid control (2018, methodology): “For linear-quadratic-Gaussian control (Athans, 1971) or differential dynamic programming (Mayne, 1966; Jacobson & Mayne, 1970), we obtain feedback policies where the feedback terms are computed from the value function, amounting effectively to feedback-stabilized plans.”
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control (2018, methodology): “However, for problems in continuous control, MPC based on local dynamic programming methods [15, 16] provide an efficient way to approximately realize B , which can be used to accelerate and stabilize value function learning.”
SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning (2018, methodology): “As shown in prior work [16, 40], these optimal policies can be computed in closed form using dynamic programming by computing the ﬁrst and second derivatives of the Q (cost-to-go) and value functions: Here, similar to prior work [39, 24], we use subscripts to denote derivatives, and we use ˜ s to…”
CHOMP: Covariant Hamiltonian optimization for motion planning (2013, methodology): “Although the authors in this case used a variety of other optimizers such as DDP [55], AICO [77], and RPROP [34], we view the trajectory prediction research as complementary in that it establishes that informed priors can overcome local minima and trade-off offline learning for online optimization.”
A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems (2005, methodology): “DDP, ILQR, and the new ILQG are dynamic programming methods that use quadratic approximations to the optimal cost-to-go function.”
Model Predictive Path Integral Control using Covariance Variable Importance Sampling (2015, methodology): “etting. After deriving the model predictive path integral control (MPPI) algorithm, we compare it with an existing model predictive control formulation based on differential dynamic programming (DDP) [6], [13], [18]. DDP is one of the most powerful techniques for trajectory optimization, it relies o”
Minimax Differential Dynamic Programming: An Application to Robust Biped Walking (2002, methodology): “To improve reinforcement learning, we propose using differential dynamic programming (DDP) which is a second order local trajectory optimization method to generate locally optimal plans and local models of the value function[2, 4].”
Approximate real-time optimal control based on sparse Gaussian process models (2014, methodology): “Iterative LQR [5] is a powerful trajectory optimization method from the family of differential dynamic programming (DDP) [12] algorithms.”
Probabilistic Differential Dynamic Programming (2014, background): “DDP: Originally introduced in the 70’s, the classical DDP [1] is still one of the most effective and efficient trajectory optimization approaches.”