Learning force control policies for compliant manipulation

By Mrinal Kalakrishnan, Ludovic Righetti, Peter Pastor, Stefan Schaal. 2011

176 citations recorded by the source corpus.

Subfields: Biomedical Engineering, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

End-to-End Training of Deep Visuomotor Policies (2015, methodology): “one component of the robot control pipeline, which often sits on top of a hand-designed controller, such as a PD controller, and accepts processed input, for example from an existing vision pipeline (Kalakrishnan et al., 2011). Our method learns policies that map visual input and joint encoder readi”
Data-efficient Deep Reinforcement Learning for Dexterous Manipulation (2017, background): “…have been routinely applied in robotics, in part because they straightforwardly handle continuous and high-dimensional action spaces [3] and applications include manipulation [26, 13, 25, 37, 18, 5, 39, 8], locomotion e.g. [16, 21], and a range of other challenges such as helicopter ﬂight [1].”
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates (2016, methodology): “Previous work has demonstrated learning of door opening policies using example demonstration provided by a human expert [6].”
Reinforcement learning in robotics: A survey (2013, background): “…and Schaal, 2008c,b), reaching tasks (Guenter et al., 2007; Bitzer et al., 2010), ball in a cup (Kober and Peters, 2009), flipping a light switch (Buchli et al., 2011), playing pool and manipulating a box (Pastor et al., 2011), and opening a door and picking up objects (Kalakrishnan et al., 2011).”
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, methodology): “Reinforcement learning offers a promising avenue for tackling this problem, but current work on reinforcement learning tackles the problem of mastering individual skills, such as hitting a ball [1], opening a door [2, 3], or throwing [4].”
Path integral guided policy search (2016, positive): “In [23], PI(2) is used to learn force/torque profiles for door opening and picking tasks to improve imperfect kinesthetic demonstrations.”
Learning contact-rich manipulation skills with guided policy search (2015, background): “While contact-rich manipulation tasks such as the ones considered in this work are less common, several prior methods have addressed opening doors and picking up objects [13], as well as using force sensors during manipulation [14].”
Learning dexterous manipulation for a soft robotic hand from human demonstrations (2016, background): “Although DMPs are often used to describe the kinematic state of a system, they can be used to generate compliant behavior for picking up small objects or opening doors [15].”
Learning robot in-hand manipulation with tactile features (2015, methodology): “The problems of grasping and door opening have been addressed in [36].”
Learning manipulation skills from a single demonstration (2018, background): “Another approach is proposed by Kalakrishnan et al. (2011) that learns force control policies for manipulations.”
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks (2018, background): “Most of them do not combine the two modalities and do not work on full manipulation tasks [4,5,6,7].”
Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks (2019, methodology): “[13] proposed a method to refine given trajectories with additional force profiles using PI2[44] and readings from a force-torque sensor.”
Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets (2017, background): “There has also been a separate line of work on learning from demonstration, which is then iteratively improved through reinforcement learning [17, 6, 21].”
Stable reinforcement learning with autoencoders for tactile and visual data (2016, background): “Reinforcement learning (RL) has been shown to be useful in optimizing, among others, reaching, grasping, and manipulation skills [3]–[5].”
Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks (2019, methodology): “In addition, compliance makes the peg insertion task easier to accomplish under position uncertainty, as the robot can slide on the surface of the box while pushing downward [18], [77], [78].”
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020, methodology): “Robotic manipulation tasks that have been possible with DRL range from learning peg-in-hole tasks [40] to deformable object manipulation [6], and including more dexterous manipulation with multi-fingered hands [5], or learning force control policies [78].”
Scaling data-driven robotics with reward sketching and batch reinforcement learning (2019, background): “RL has a long history in robotics [37, 53, 34, 25, 42, 43, 35].”
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning (2019, background): “Lastly, while a number of prior works have studied RLbased learning of door opening (Kalakrishnan et al., 2011; Chebotar et al., 2017), our method is the first to demonstrate autonomous learning of door opening without a userprovided reward function.”
Helping Robots Learn: A Human-Robot Master-Apprentice Model Using Demonstrations via Virtual Reality Teleoperation (2020, background): “Policy search methods [14] can thereby enable learning new behavior from experience [15], [16], [14] or even jointly training perception and control systems [17].”