Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming

By Christopher G. Atkeson. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement learning in robotics: A survey (2013, methodology): “Teacher Atkeson and Schaal (1997), Bentivegna et al.”
Dual Policy Iteration (2018, methodology): “The instantiation shares similar spirit from some previous works from robotics and control literature, including works from [17, 18] and Guided Policy Search (GPS) [19] (and its variants (e.”
Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping (2008, methodology): “The first, due simultaneously to Peng and Williams (1993) and to Moore and Atkeson (1993), which we call PWMA prioritized sweeping, adds the predecessors of every state encountered in real experience to the priority queue whether or not the value of the encountered state was significantly changed. T”
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control (2018, methodology): “Related to this, Atkesnon [30] uses an offline trajectory library for action selection in real-time, but do not explicitly consider learning parametric value functions.”
A comparison of direct and model-based reinforcement learning (1997, methodology): “This task involves swinging up a single pendulum with a motor at the base from the hanging down position to the inverted position , and balancing in the up position (Atkeson, 1994; Doya, 1996).”
Exploiting Model Uncertainty Estimates for Safe Dynamic Control Learning (1996, methodology): “It has been suggested Atkeson, 1995, Atkeson, 1993 that techniques used in global linear control, including caution and probing, may also be applicable in the local case.”
Reinforcement learning of motor skills with policy gradients (2008, methodology): “Extensions based on learned, approximate models originated in the literature on optimizing government decision policies, seeWerbos (1979), and have also been applied in control (Atkeson, 1994;Morimoto & Atkeson, 2003).”
Reinforcement Learning in Continuous Time and Space (2000, methodology): “The performance of the proposed methods is rst evaluated in nonlinear control tasks of swinging up a pendulum with limited torque (Atkeson, 1994; Doya, 1996) using normalized Gaussian basis function networks for representing the value function, the policy, and the model.”
Linear Bellman combination for control of character animation (2009, methodology): “A related non-parametric representation was proposed by Atkeson et al [2002]. In our case, example trajectories need not be optimized for the same task.”
Learning from Demonstration (1996, methodology): “The V–function and the dynamics model were incrementally learned by a nonlinear function approximator, Receptive Field Weighted Regression (RFWR) (Schaal & Atkeson (1996)). Differing from Doya’s (1996) implementation, we used the optimal action suggested by CTD to learn a model of the policy π (an “”
Nonparametric Model-Based Reinforcement Learning (1997, background): “2 CONSISTENT LOCAL PLANNING An e cient approach to dynamic programming, a form of global planning, is to use local trajectory optimizers (Atkeson, 1994).”
Locally Weighted Learning for Control (1997, methodology): “This is done by the shifting setpoint algorithm (SSA) (Schaal and Atkeson 1994a).”
Variational Bayesian learning of nonlinear hidden state-space models for model predictive control (2009, methodology): “Dynamic programming can be used to solve complex control problems in two different ways: Atkeson et al. [4] [5] used dynamic programming in the state space by discretising the state space into cells and solving locally optimal trajectories for each cell.”
Learning perceptual coupling for motor primitives (2008, unknown): “, using differential-dynamic programming [14]).”
Learning motor primitives for robotics (2009, unknown): “, using differential-dynamic programming [18]).”
Gaussian process dynamic programming (2009, methodology): “Similarly, many reinforcement learning algorithms are based on dynamic programming techniques comprising value iteration and policy iteration methods, details of which are given by Sutton and Barto (1998) and Bert-sekas and Tsitsiklis (1996).”
A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, background): “This is especially critical for training, as training an agent in an imprecise model will result in a system that will not transfer to the real-world without significant modifications [152], [197].”