SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation

By Xingyu Lin, Yufei Wang, Jake Olkin, David Held. 2020

291 citations recorded by the source corpus.

Manipulating deformable objects has long been a challenge in robotics due to its high dimensional state representation and complex dynamics. Recent success in deep reinforcement learning provides a promising direction for learning to manipulate deformable objects with data driven methods. However, existing reinforcement learning benchmarks only cover tasks with direct state observability and simple low-dimensional dynamics or with relatively simple image-based environments, such as those with rigid objects. In this paper, we present SoftGym, a set of open-source simulated benchmarks for manipulating deformable objects, with a standard OpenAI Gym API and a Python interface for creating new environments. Our benchmark will enable reproducible research in this important area. Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable. The experiments and analysis indicate the strengths and limitations of existing methods in the context of deformable object manipulation that can help point the way forward for future methods development. Code and videos of the learned policies can be found on our project website.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Challenges and Outlook in Robotic Manipulation of Deformable Objects (2021, methodology): “work, Softgym [52] was proposed for bench-marking DOM based on Nvidia Flex.”
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments (2021, unknown): “While benchmarks for stationary manipulation [55, 29, 28, 30, 56, 31, 32] fare well on physical realism, they commonly fall short on diversity (of scenes, objects, tasks) and complexity (e.g., simple activities that take a few seconds).”
BEHAVIOR-1K: A Benchmark for Embodied AI with 1, 000 Everyday Activities and Realistic Simulation (2022, background): “While signiﬁcant progress in realism has been made in speciﬁc domains [43–45], achieving realism for a diverse set of activities remains a tremendous challenge, due to the effort required to provide realistic models and simulation features.”
Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks (2020, methodology): “[39] propose SoftGym, a set of environments with cables, fabrics, and liquids simulated using NVIDIA FleX, and use it to benchmark standard reinforcement learning algorithms.”
Modeling, learning, perception, and control methods for deformable object manipulation (2021, background): “The maturity and accessibility of deformable object simulators make it now possible to generate large amounts of relevant data (39, 121) and, to some extent, compensate the need for analytical modeling.”
Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments (2023, methodology): “On the other hand, frameworks for deformable bodies [4], [8] mainly employ Bullet [18] or FleX [19], which use particle-based dynamics for soft bodies and cloth simulation.”
ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills (2023, background): “Other environments, like SoftGym (Lin et al., 2020) and ThreeDWorld (Gan et al.”