Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics

Subfields: Robotics & Control

Citations in the corpus

• ColdGANs: Taming Language GANs with Cautious Sampling Strategies (2020, background): “Such unstructured exploration causes a large variance which grows with the number of time steps, and perturbs actions too frequently [35, 18].”
• Benchmarking Deep Reinforcement Learning for Continuous Control (2016, methodology): “For neural network policies with tens of thousands of parameters or more, generic Natural Policy Gradient incurs prohibitive computation cost by forming and inverting the empirical FIM.”
• Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning (2019, background): “Importance sampling can be incorporated into RWR to reuse data from previous iterations, but at the cost of larger variance from the importance sampling estimator (Kober & Peters, 2009).”
• Learning Trajectory Preferences for Manipulators via Iterative Improvement (2013, background): “Most of the past research has focused on mimicking expert’s demonstrations, for example, autonomous helicopter flights [1], ball-in-a-cup experiment [17], planning 2-D paths [27, 25, 26], etc.”
• Learning Parameterized Skills (2012, methodology): “We combine DMPs with PoWER (Kober & Peters, 2008), a policy search technique that collects sample path executions and updates the policy’s parameters towards ones that induce a new success-weighted path distribution.”
• Reinforcement Learning in Healthcare: A Survey (2019, methodology): “There has been a significant achievement in the continuous control using AC methods and PS methods in the past years, particularly from the area of robotic control [20] and DRL [12].”
• Reinforcement learning in robotics: A survey (2013, background): “approaches that can be applied in simulation may work similarly in the real-world (Kober and Peters, 2010).”
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models (2018, background): “ learning algorithms are quite expensive to train, which often limits their application to simulated domains [Mnih et al., 2015, Lillicrap et al., 2016, Schulman et al., 2017], with a few exceptions [Kober and Peters, 2009, Levine et al., 2016]. A promising direction for reducing sample complexity i”
• Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations (2017, methodology): “Some methods like PoWER overcome this limitation through the use of simple policy representations such as DMPs and demonstrate impressive results [36], [22].”
• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward (2019, methodology): “This ranges from simple techniques such as demonstration-based pre-training and initialization [28, 19, 21, 20, 35] to more complex methods that incorporate both demonstration data and reward information in the loop of training [40, 3, 37, 14, 38, 29, 26, 22].”
• Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost (2018, methodology): “Similar methods have also been used for whole-arm manipulation [16].”
• Maximum a Posteriori Policy Optimisation (2018, methodology): “Existing EM policy search approaches perform this step typically by reweighting trajectories with sample returns (Kober & Peters, 2009) or via local trajectory optimization (Levine & Koltun, 2013).”
• Overcoming Exploration in Reinforcement Learning with Demonstrations (2017, methodology): “Initializing policies from demonstrations for RL has been used for learning to hit a baseball [32] and for underactuated swingup [33].”
• Optimal control as a graphical model inference problem (2009, positive): “This idea is further developed into an ExpectationMaximization (EM) type algorithm (Dayan and Hinton 1997) in recent work (Peters et al. 2010; Kober and Peters 2011) using a relative entropy term.”
• Parameter Space Noise for Exploration (2017, background): “Their main limitation, however, is that they are either only proposed and evaluated for the on-policy setting with relatively small and shallow function approximators [10] or disregard all temporal structure and gradient information [9, 11, 12].”
• Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning (2011, methodology): “In [11], a model-free policy learning method is presented, which relies on rollouts sampled from the system.”
• Deep predictive policy training using reinforcement learning (2017, background): “These approaches, known as dynamic movement primitives (DMP), have found applications in robot learning by demonstration [25] and also reinforcement learning [3], [26].”
• Learning dexterous manipulation for a soft robotic hand from human demonstrations (2016, methodology): “This has been applied to reaching motions [28] and dynamic manipulation tasks, such as ballin-a-cup [29] or pancake-flipping [30].”
• Dynamic policy programming (2010, neutral): “Likewise, our formalism is superficially similar to PoWER (Kober and Peters, 2008) and SAEM (Vlassis and Toussaint, 2009), which rely on EM algorithm to maximize a lower bound for the expected return in an iterative fashion.”
• Learning by Playing - Solving Sparse Reward Tasks from Scratch (2018, background): “…et al. (2017); Tobin et al. (2017); Rusu et al. (2017) for recent examples), learning from demonstrations (Ross et al., 2011; Vecerik et al., 2017; Kober & Peters, 2011; Sermanet et al., 2017; Nair et al., 2017), learning with model guidance, see e.g. Montgomery & Levine (2016), or inverse RL…”