Robot motor skill coordination with EM-based Reinforcement Learning

By Petar Kormushev, Sylvain Calinon, Darwin G. Caldwell. 2010

303 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Learning human activities and object affordances from RGB-D videos (2012, methodology): “None of these methods explicitly consider the role of objects or object affordances that not only help in identifying sub-activities and high-level activities, but are also important for several robotic applications (e.g. Kormushev et al., 2010).”
End-to-End Robotic Reinforcement Learning without Reward Engineering (2019, background): “For example, using thermal cameras for tracking fluids [35], mocap sensors [21] or computer vision systems [34] for tracking objects, and accelerometers for determining the state of a door [44].”
Watch, Try, Learn: Meta-Learning from Demonstrations and Reward (2019, methodology): “This ranges from simple techniques such as demonstration-based pre-training and initialization [28, 19, 21, 20, 35] to more complex methods that incorporate both demonstration data and reward information in the loop of training [40, 3, 37, 14, 38, 29, 26, 22].”
Learning dexterous manipulation for a soft robotic hand from human demonstrations (2016, methodology): “This has been applied to reaching motions [28] and dynamic manipulation tasks, such as ballin-a-cup [29] or pancake-flipping [30].”
Active Reward Learning (2014, background): “It does not rely on exhaustive sampling of the state-action space and, as a result, many recent advances in robot learning rely on policy search [22, 20, 26].”
Learning motion primitive goals for robust manipulation (2011, background): “In recent years, model-free reinforcement learning algorithms that search directly in policy parameter space have been able to learn a variety of motor skills, such as baseball batting [13], dart throwing [6], table tennis [6], pancake flipping [7], and archery [8].”
Probabilistic Movement Primitives (2013, methodology): “MPs have been used to solve many complex tasks, including ‘Ball-in-the-Cup’ [4], Ball-Throwing [5, 6], Pancake-Flipping [7] and Tetherball [8].”
Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions (2014, background): “Imitation learning (also known as learning from demonstrations) is one of the most common and intuitive ways of building an estimate of DS motions from a set of demonstrations [6, 13, 7, 8].”
Few-Shot Goal Inference for Visuomotor Learning and Planning (2018, background): “Robotic learning methods often sidestep this issue and instead hand-engineer task objectives [15, 16], often using manually instrumented environments (e.g., a thermal camera or scale to evaluate pouring [17, 18, 19], mocap to track a pancake [20], and an accelerometer on a door handle [21]).”
Imitation Learning of Positional and Force Skills Demonstrated via Kinesthetic Teaching and Haptic Input (2011, methodology): “Successful applications of this torque-based kinesthetic teaching process have been developed recently for the Barrett WAM and KUKA/DLR robotic arms [8, 15].”
Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresight (2019, background): “Prior work has leveraged demonstrations to accelerate model-free reinforcement learning either in simulation or the real world, overcoming the well-known exploration problem [37, 33, 21, 22].”
A probabilistic approach to robot trajectory generation (2013, methodology): “They have been used successfully to solve many complex tasks, including the ‘Ball-in-the-Cup’ game [4], Ball-Throwing [5], Pancake-Flipping [6] and bipedal gait generation [7].”
Generalized Cylinders for Learning, Reproduction, Generalization, and Refinement of Robot Skills (2017, methodology): “It has also been shown that integrating LfD with Reinforcement Learning can be used for refinement [17].”
Active reward learning with a novel acquisition function (2015, background): “It does not rely on exhaustive sampling of the state-action space and, as a result, many recent advances in robot learning are based on PS (Kormushev et al. 2010; Kober et al. 2008; Ng and Coates 1998).”
Model-based contextual policy search for data-efficient generalization of robot skills (2017, methodology): “Policy Search (PS) methods are one of the most successful Reinforcement Learning (RL) algorithms for learning complex movement tasks in robotics [31, 38, 22, 23, 21, 8, 30, 20, 27, 13].”
Learning force control policies for compliant manipulation (2011, background): “Acquisition of manipulation skills using reinforcement learning has been previously demonstrated [1], [2], [3], [4], [5].”
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training (2021, background): “…application area in which HiL methods could have signiﬁcant real-world impact is robotic manipulation, since learning often requires extensive engineering in the real world (Yahya et al., 2017; Schenck & Fox, 2017; Kormushev et al., 2010; Rusu et al., 2017; Akkaya et al., 2019; Peng et al., 2020).”
Socially guided intrinsic motivation for robot learning of motor skills (2013, background): “– Beginning of learning A limited number of examples is given to initialise the learning, as a basic behaviours repertoire (Argall et al. 2008, 2011), or a sample behaviour to be optimised (Peters and Schaal 2008; Kormushev et al. 2010).”
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning (2019, methodology): “Using these ideas, RL has been applied to wide variety of robotic tasks such as stacking blocks in a tower [2], [3], [4], opening doors [5], ﬂipping pan-cakes [6], hitting a ball, orienting a cube [7] and other dexterous manipulation tasks [8].”