Policy Gradients with Parameter-Based Exploration for Control

By Frank Sehnke, Christian Osendorfer, Thomas Rückstieß, Alex Graves, Jan Peters, J. Urgen Schmidhuber. 2008

83 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks (2016, methodology): “Compared to algorithms for eﬃcient direct exploration in parameter space (see e.g., [18, 27, 37, 25]), the novelty of our algorithm is to mix exploration through parameter randomization and plain gradient descent.”
Hierarchical Variational Models (2015, methodology): “A similar methodology also arises in the policy search literature (Rückstieß et al., 2008; Sehnke et al., 2008).”
Multi-Dimensional Deep Memory Atari-Go Players for Parameter Exploring Policy Gradients (2010, methodology): “We therefore train our Go-playing networks using the novel Policy Gradients with Parameter-based Exploration (PGPE, [8]), which have recently been shown to be very successful at optimizing the parameters of large Neural Network controllers [9].”
Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularization (2021, background): “In particular, we focus on the parameter-based PO2 in which the policy parameter θ is sampled from a hyperpolicy νρ belonging, in turn, to a parametric set NP “ tνρ : ρ P P Ď R2u (Sehnke et al. 2008).”
A Survey on Policy Search for Robotics (2013, methodology): “Such an approach was first introduced by [74, 75] with the Parameter Exploring Policy Gradient (PEPG) algorithm.”
Partial Policy-Based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images (2018, background): “Due to the stochasticity in the experienced rewards, direct policy search methods suffer from the high variance problem [9], [10].”
Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes (2008, methodology): “…efﬁciently, without frequent expensive compressor performance evaluations on the entire history so far; (3) study the applicability of recent improved RL techniques in the ﬁelds of policy gradients [110, 119, 118, 56, 100, 117], artiﬁcial evolution [43, 20, 21, 19, 22, 23, 24], and others [71, 75].”
Black-box data-efficient policy search for robotics (2017, background): “Both the model-based PGPE [30] and the PILCO [13] algorithm use gradient-based policy updates.”
Exploring parameter space in reinforcement learning (2010, methodology): “Furthermore, this introduces much less variance in the rollouts, which facilitates credit assignment and generally leads to faster convergence [18, 22].”
Adaptive Batch Size for Safe Policy Gradients (2017, background): “In the last years, several RL studies [1, 2, 3, 4, 5, 6, 7] have shown that policy-search methods can effectively be employed to solve complex control tasks (e.”