Guided Policy Search

By Sergey Levine, Vladlen Koltun. 2013

899 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Flow Contrastive Estimation of Energy-Based Models (2019, neutral): “In the context of inverse reinforcement learning, [44] proposes a guided policy search method, and [11] connects it to GAN.”
Sample Efficient Actor-Critic with Experience Replay (2016, background): “With richer realistic environments, the capabilities of our agents have increased and improved.”
Benchmarking Deep Reinforcement Learning for Continuous Control (2016, methodology): “…difﬁculty: Swimmer (Purcell, 1977; Coulom, 2002; Levine & Koltun, 2013; Schulman et al., 2015a), Hopper (Murthy & Raibert, 1984; Erez et al., 2011; Levine & Koltun, 2013; Schulman et al., 2015a), Walker (Raibert & Hodgins, 1991; Erez et al., 2011; Levine & Koltun, 2013; Schulman et al., 2015a),…”
End-to-End Training of Deep Visuomotor Policies (2015, methodology): “The training set for this supervised learning procedure can be constructed from example demonstrations [20], trajectory optimization under known dynamics [21, 22, 28], and trajectory-centric reinforcement learning methods that operate under unknown dynamics [19, 23], which is the approach taken in…”
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, methodology): “g algorithms (Dvijotham and Todorov, 2010). More recently, entropy maximization has been used as a component in algorithms based on model-free policy search with importance sampling and its variants (Levine and Koltun, 2013a; Nachum et al., 2017a,b), model-based algorithms based on the guided policy”
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018, background): “In guided policy search (Levine & Koltun, 2013; Levine et al., 2016), the maximum entropy distribution is used to guide policy learn-Soft ing towards high-reward regions.”
Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic (2016, methodology): “Prior methods that combine off-policy data with policy gradients either introduce bias (Sutton et al., 1999; Silver et al., 2014) or use importance weighting, which is known to result in degenerate importance weights in high dimensions, resulting in very high variance (Precup, 2000; Levine & Koltun,”
PGQ: Combining policy gradient and Q-learning (2016, methodology): “Vanilla actor-critic methods are on-policy only, although some attempts have been made to extend them to off-policy data (Degris et al., 2012; Levine & Koltun, 2013).”
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning (2015, methodology): “Another re-lated method is that of guided policy search (Levine & Koltun, 2013), which combines a regularized importance-sampled policy gradient with guiding trajectory samples generated using differential dynamic programming.”
Continuous Deep Q-Learning with Model-based Acceleration (2016, methodology): “Employing the maximum entropy objective (Levine & Koltun, 2013), we can also construct a linear-Gaussian controller, where c is a scalar to adjust for arbitrary scaling of the reward magnitudes: When the dynamics are not known, a particularly effective way to use iLQG is to combine it with learned…”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “While the reinforcement learning algorithms have been an active area of research for decades, the introduction of effective high-capacity function approximators – deep neural networks – into reinforcement learning, along with effective algorithms for training them, has allowed reinforcement learning”
Solving Rubik's Cube with a Robot Hand (2019, background): “Guided policy search [56, 58] learns simple local policies directly on the robot and distills them into a global policy represented by a neural network.”
Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search (2018, methodology): “We refer to this algorithm as Guided Policy Search-like (GPS-like), as it roughly corresponds to the algorithm presented by Levine & Abbeel (2014), as discussed in greater detail in sec.”
Verifiable Reinforcement Learning via Policy Extraction (2018, methodology): “Instead, we used an approach inspired by guided policy search [21].”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “GPS starts from guiding samples generated from some initial optimal control policies and augmented from samples generated from the current policy, from which at every iteration a set of training trajectories are sampled to optimize the current policy with supervised learning.”
Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning (2016, methodology): “Furthermore, often the goal of OPE is not to produce conﬁdence bounds, but to produce the best estimate of v ( π e ) possible, in order to determine whether π e should be used instead of the current behavior policy or as an internal mechanism in a policy search algorithm (Levine & Koltun, 2013).”
Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation (2018, background): “It is known that pretraining a policy network with expert policies if they are available leads to better training stability and performance [24].”
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning (2017, methodology): “Among these importance sampling methods, Levine & Koltun (2013) evaluates on tasks that are the most similar to our paper, but the focus is on using importance sampling to include demonstrations, rather than to speed up learning from scratch.”
Adaptive Online Planning for Continual Lifelong Learning (2019, background): “[19, 22] propose to use priors that make the planner stay close to policy outputs, which is problematic in changing worlds, when the policy is not accurate.”