A Survey on Policy Search for Robotics

By Marc Peter Deisenroth, Gerhard Neumann, Jan Peters. 2013

1,025 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Trust Region Policy Optimization (2015, background): “These algorithms are scalable and can optimize nonlinear policies with tens of thousands of parameters, which have previously posed a major challenge for model-free policy search (Deisenroth et al., 2013).”
Benchmarking Deep Reinforcement Learning for Continuous Control (2016, methodology): “Relative Entropy Policy Search (REPS) (Peters et al., 2010): This algorithm limits the loss of information per iteration and aims to ensure a smooth learning progress (Deisenroth et al., 2013).”
Learning Dynamic Robot-to-Human Object Handover from Human Feedback (2016, background): “Policy search algorithms parametrize robot control policies and search for the best parameter values by maximizing a reward function that captures the policy performance.”
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates (2016, methodology): “Neural networks are powerful function approximators, but they have significantly more parameters than the simpler linear models that are often used in robotic learning [23], [8].”
Model-Based Reinforcement Learning for Atari (2019, background): “1 Outside of games, model-based reinforcement learning has been investigated at length for applications such as robotics (Deisenroth et al., 2013).”
Continuous Deep Q-Learning with Model-based Acceleration (2016, methodology): “…following sections, it will be useful to also deﬁne the value function V π ( x t , u t ) and advantage function A π ( x t , u t ) of a given policy π : (Deisenroth et al., 2013; Kober & Peters, 2012), two are particularly relevant for this work: iterative LQG (iLQG) (Li & Todorov, 2004) and…”
Reinforcement Learning in Healthcare: A Survey (2019, background): “There has been a significant achievement in the continuous control using AC [74] and PS meth-ods [42] in the past few years, particularly from the area of robotic control and DRL.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Popular approaches for cautious exploration include avoiding signiﬁ-cant changes in the policy, or explicitly discouraging entering undesired regions in the state space (Deisenroth et al., 2013).”
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning (2017, background): “Model-based algorithms are known in general to outperform model-free learners in terms of sample complexity [4], and in practice have been applied successfully to control both simulated and real-world robotic systems, such as pendulums [6], legged robots [18], swimmers [19], and manipulators [20].”
Uncertainty-Aware Reinforcement Learning for Collision Avoidance (2017, neutral): “Reinforcement learning has been applied to a wide range of robotic problems, ranging from locomotion and manipulation to autonomous helicopter ﬂight [14, 4].”
PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos (2019, methodology): “For a review of policy search methods, see (Deisenroth et al., 2013).”
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, methodology): “Model-based RL algorithms are generally known to be more efﬁcient than model-free methods [10], and have been used with both low-dimensional [11] and high-dimensional [12] model classes.”
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, methodology): “In the context of reinforcement learning (RL), model-based RL explored optimizing policies using learned models [12].”
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees (2018, methodology): “Indeed, empirical results (Deisenroth & Rasmussen, 2011b; Deisenroth et al., 2013; Levine et al., 2016; Nagabandi et al., 2017; Kurutach et al., 2018; Pong et al., 2018a) have shown strong improvements in sample efﬁciency.”
Boosting the Actor with Dual Critic (2017, background): “Actor-critic methods combine the beneﬁts of these two classes, and have proved successful in a number of challenging problems such as robotics (Deisenroth et al., 2013), meta-learning (Bello et al., 2016), and games (Mnih et al., 2016).”
Deep spatial autoencoders for visuomotor learning (2015, methodology): “The controller can be optimized using a variety of modelbased and model-free methods [31].”