Learning to Manipulate Deformable Objects without Demonstrations

By Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, Pieter Abbeel. 2019

231 citations recorded by the source corpus.

In this paper we tackle the problem of deformable object manipulation through model-free visual reinforcement learning (RL). In order to circumvent the sample inefficiency of RL, we propose two key ideas that accelerate learning. First, we propose an iterative pick-place action space that encodes the conditional relationship between picking and placing on deformable objects. The explicit structural encoding enables faster learning under complex object dynamics. Second, instead of jointly learning both the pick and the place locations, we only explicitly learn the placing policy conditioned on random pick points. Then, by selecting the pick point that has Maximal Value under Placing (MVP), we obtain our picking policy. This provides us with an informed picking policy during testing, while using only random pick points during training. Experimentally, this learning framework obtains an order of magnitude faster learning compared to independent action-spaces on our suite of deformable object manipulation tasks with visual RGB observations. Finally, using domain randomization, we transfer our policies to a real PR2 robot for challenging cloth and rope coverage tasks, and demonstrate significant improvements over standard RL techniques on average coverage.

Subfields: Building and Construction, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Building and Construction

Jia Pan

In Control and Systems Engineering

Citations in the corpus

SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation (2020, background): “[4] V.”
Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, background): “, logistics, agriculture, household) – leading to renewed interest in leveraging learned models to achieve general pick-and-place policies [23, 24, 25, 26, 27, 28, 29] that can handle unseen objects.”
CLIPort: What and Where Pathways for Robotic Manipulation (2021, background): “On the other hand, end-to-end perception-toaction models can learn precise sequential policies [2, 4, 6, 33, 34, 35], but these methods have limited understanding of semantic concepts and rely on goal-images to condition policies.”
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, background): “End-to-End Manipulation approaches [26, 27, 28, 29] make the least assumptions about objects and tasks, but are often formulated as an image-to-action prediction task.”
Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks (2020, methodology): “For example, IL is used with human [54] and scripted [53] demonstrators for fabric smoothing, while RL is applied for smoothing and folding in [44], [29], [63].”
Challenges and Outlook in Robotic Manipulation of Deformable Objects (2021, background): “have proposed a reinforcement learning scheme for DOM, which does not require initial demonstrations [61].”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, methodology): “Tasks like cloth folding (134, 135, 136) and assistive dressing (137) have thus received considerable attention in RL.”
Cable manipulation with a tactile-reactive gripper (2019, methodology): “Most work in fabric manipulation similarly uses the quasistatic assumption and incremental pick-and-place movements (Ganapathi et al., 2020; Hoque et al., 2020; Wu et al., 2019).”