SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark

By Linxi (Jim) Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei. 2018

167 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Evaluation & Benchmarks, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Evaluation & Benchmarks

In Robotics & Control

Citations in the corpus

What Matters in Learning from Offline Human Demonstrations for Robot Manipulation (2021, methodology): “Robot Manipulation Benchmarks : Several benchmark robot manipulation tasks have been proposed before [73–76, 24, 77].”
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation (2020, background): “A variety of new environments have been created recently to benchmark reinforcement learning algorithms [15, 34, 35, 36, 37].”
robosuite: A Modular Simulation Framework and Benchmark for Robot Learning (2020, methodology): “The development of this framework was initiated from the SURREAL project [3] on distributed reinforcement learning for robot manipulation, and is now part of the broader Advancing Robot Intelligence through Simulated Environments (ARISE) Initiative, with the aim of lowering the barriers of entry for”
Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “SURREAL (Fan et al., 2018) expresses continuous control agents in a distributed training framework.”
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data (2019, methodology): “Robosuite Lift - Suboptimal Demonstrations from a Human: We collected human demonstrations from a single human using RoboTurk [26] on the Robosuite Lifting task [9].”
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning (2019, background): “Similarly, there exists a number of robotics simulation benchmarks including visual navigation [39, 40, 41, 42, 43], autonomous driving [44, 45, 46], grasping [47, 48, 49], single-task manipulation [50], among others.”
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation (2018, methodology): “This simple method for doing reinforcement learning with demonstrations has been motivated theoretically by [45] and used in practice in prior and concurrent work [46, 47, 39].”
Continual Learning for Robotics (2019, neutral): “, 2013) for Atari games, SURREAL (Fan et al., 2018) for robot manipulation and RoboTurk for robotic skill learning through imitation (Mandlekar et al.”
Physically Embedded Planning Problems: New Challenges for Reinforcement Learning (2020, background): “This includes, in particular, a signiﬁcant body of work on robotic arm manipulation at diﬀerent levels of complexity (e.g., Lee et al., 2019a; Popov et al., 2017; Riedmiller et al., 2018; Zhu et al., 2018).”
Learning Visually Guided Latent Actions for Assistive Teleoperation (2021, methodology): “We use the Surreal Robotics Suite (Robosuite) (Fan et al., 2018), a simulation toolkit based off the MuJoCo physics engine (Todorov et al., 2012) to render environments.”
BC + RL : Imitation Learning From Non-Optimal Demonstrations (2019, methodology): “To generate the episodes that we use to perform the BC segment of our method, we train a policy using Surreal distributed PPO using end effector control with 8 actors to achieve a policy that attains an average sum of rewards of 180 which differs from the results from the Surreal paper which was abl”
Online Continual Learning on Sequences (2020, methodology): “• Reinforcement Learning: Arcade Learning Environment (ALE) [3] for Atari games, SURREAL [11] for robot manipulation and RoboTurk for robotic skill learning through imitation [49], CRLMaze extension of VizDoom [39] and DeepMind Lab [50].”
Rearrangement: A Challenge for Embodied AI (2020, background): “…tasks focusing on standardized real-world object sets [13], and competition-style setups, such as RoboCup [43, 67] and the International Planning Competition [53], as well as simulated benchmarks focusing on speciﬁc algorithmic approaches such as reinforcement learning [5, 9, 24, 79, 26].”
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks (2019, methodology): “g and placing [20, 21], in-hand dexterous manipulation [22, 23], door opening [5], and peg inserting [24]. Recent advancements in simulators have made a push towards more complex and realistic tasks. [4] takes a step towards a comprehensive manipulation simulator by offering a variety of manipulatio”
Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning (2019, methodology): “We demonstrate the effectiveness of the proposed method on the 2D CoinRun (Cobbe et al., 2019) game, the 3D DeepMind Lab exploration task (Beattie et al., 2016), and the 3D robotics control task (Fan et al., 2018).”
Intrinsic Motivation for Encouraging Synergistic Behavior (2020, methodology): “Bimanual manipulation tasks are built on the Surreal Robotics Suite (Fan et al., 2018).”
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback (2023, background): “More broadly, building a simulator environment to enable low-cost experimentation is common in the field of reinforcement learning and robotics [11, 67, 66, 64, 17, 23, 18].”
VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning (2019, background): “Typical systems include 3D game environments [6, 21, 23], and robot control platforms [11, 13, 37, 51].”
CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability (2021, background): “…1998), CIFAR (Krizhevsky, 2009), Core50 (Lomonaco & Maltoni, 2017), learning unique sounds (Gemmeke et al., 2017) or skills (Mandlekar et al., 2018; Fan et al., 2018) in sequence, or simply creating a chain across multiple datasets for natural language processing (McCann et al., 2018; Wang et al.,…”