Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics

By Sergey Levine, Pieter Abbeel. 2014

528 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning (2016, background): “Notable examples include guided policy search with unknown dynamics (Levine & Abbeel, 2014) and PILCO (Deisenroth & Ras-mussen, 2011).”
Emergence of Locomotion Behaviours in Rich Environments (2017, background): “[10, 7, 11, 12, 6, 13]), although impressive, have focused on comparatively small problems, and the use of large-scale, distributed optimization is less widespread and the corresponding algorithms are less well developed (but see e.”
End-to-End Training of Deep Visuomotor Policies (2015, methodology): “The derivation follows prior work [19], and we describe novel modiﬁcations for improving performance and sample efﬁciency at the end.”
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, methodology): “, 2017a,b), model-based algorithms based on the guided policy search framework (Levine and Abbeel, 2014; Levine and Koltun, 2014; Levine et al., 2016), and a variety of methods based on soft Q-learning (Haarnoja et al.”
Mastering Atari, Go, chess and shogi by planning with a learned model (2019, background): “[24] Sergey Levine and Pieter Abbeel.”
Data-efficient Deep Reinforcement Learning for Dexterous Manipulation (2017, background): “And indeed, in simulation, recent work has used neural networks to learn solutions to a variety of control problems from scratch (e.g. [7, 20, 32, 31, 11, 17]).”
Reinforcement Learning with Deep Energy-Based Policies (2017, background): “This distinction is crucial, since the maximum entropy objective can be shown to maximize the entropy of the entire trajectory distribution for the policy π , while the greedy Boltz-mann exploration approach does not (Ziebart et al., 2008; Levine & Abbeel, 2014).”
Value Iteration Networks (2016, methodology): “ces of this domain with different start state, goal, and obstacle positions, would generalize to an unseen domain. For training we chose the guided policy search (GPS) algorithm with unknown dynamics [15], which is suitable for learning policies for continuous dynamics with contacts, and we used the”
Generative Adversarial Imitation Learning (2016, background): “Guided cost learning [6], for instance, builds upon guided policy search [12] and inherits its sample efficiency, but also inherits its requirement that the model is well-approximated by iteratively fitted time-varying linear dynamics.”
Continuous Deep Q-Learning with Model-based Acceleration (2016, methodology): “…including neural networks (Heess et al., 2015), Gaussian processes (Deisenroth & Rasmussen, 2011), and locally-weighted regression (Atkeson et al., 1997), we found that we could obtain good results by using iteratively reﬁtted time-varying linear models, as proposed by Levine & Abbeel (2014).”
Learning to Optimize (2016, methodology): “We refer interested readers to [17] for details.”
Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search (2018, methodology): “We chose a particular algorithm to start from to make a connection to the previously proposed Guided Policy Search algorithm (Levine & Koltun, 2013; Levine & Abbeel, 2014), but we think a larger class of MBRL algorithms can be generalized in an analogous manner.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “einforcement learning while it can identify better policies, plans, and trajectories. Even the dynamics model is unknown, a constructed linear-Gaussian controller can be used for policy optimization (Levine and Abbeel, 2014). Then KL-divergence was leveraged to control the updating bound of the poli”
Sim-to-Real Robot Learning from Pixels with Progressive Nets (2016, background): “While recent advances in simulation-driven deep RL are impressive [8, 19, 6, 12, 20, 15, 5], demonstrating learning capabilities on real robots remains the bar by which we must measure the practical applicability of these methods.”
The limits and potentials of deep learning for robotics (2018, background): “Deep reinforcement learning is a new learning paradigm that is capable of learning end-to-end robotic control tasks, but the accomplishments have been demonstrated primarily in simulation, rather than on actual robot platforms [37], [43], [62], [64], [77], [106], [107].”
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees (2018, neutral): “…from Gaussian process (Deisenroth & Rasmussen, 2011b; Ko & Fox, 2009), time-varying linear models (Levine & Koltun, 2013; Lioutikov et al., 2014; Levine & Abbeel, 2014; Yip & Camarillo, 2014), mixture of Gaussians (Khansari-Zadeh & Billard, 2011), to neural networks (Hunt et al., 1992;…”