Efficient Bimanual Manipulation Using Learned Task Schemas

By Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, Abhinav Kumar Gupta. 2019

89 citations recorded by the source corpus.

We address the problem of effectively composing skills to solve sparse-reward tasks in the real world. Given a set of parameterized skills (such as exerting a force or doing a top grasp at a location), our goal is to learn policies that invoke these skills to efficiently solve such tasks. Our insight is that for many tasks, the learning process can be decomposed into learning a state-independent task schema (a sequence of skills to execute) and a policy to choose the parameterizations of the skills in a state-dependent manner. For such tasks, we show that explicitly modeling the schema’s state-independence can yield significant improvements in sample efficiency for model-free reinforcement learning algorithms. Furthermore, these schemas can be transferred to solve related tasks, by simply re-learning the parameterizations with which the skills are invoked. We find that doing so enables learning to solve sparse-reward tasks on real-world robotic systems very efficiently. We validate our approach experimentally over a suite of robotic bimanual manipulation tasks, both in simulation and on real hardware. See videos at http://tinyurl.com/chitnis-schema.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

Learning Multi-Arm Manipulation Through Collaborative Teleoperation (2020, background): “Prior work has tried to address the exploration burden by assuming access to parametrized skills such as reaching and twisting [7], by encouraging efﬁcient exploration via intrinsic motivation [8], and leveraging movement primitives from human demonstrations [3].”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “Learning more complex behaviors, e.g. bimanual (148) or dynamic tasks like table tennis (149), is another important avenue for future work (11, 19).”
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation (2024, unknown): “…biases, such as distinguishing two arms for stabilization and functionality (Grannen et al., 2023b), parameterizing movement primitives (Batinica et al., 2017; Amadio et al., 2019; Chitnis et al., 2020; Franzese et al., 2023), or using voxel representations (Grotz et al., 2024; Liu et al., 2024).”
Intrinsic Motivation for Encouraging Synergistic Behavior (2020, background): “These environments are suggested as bimanual manipulation tasks by Chitnis et al. (2019).”