iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks

By Chengshu Li, Fei Xia, Roberto Mart'in-Mart'in, Michael Lingelbach, Siddharth Srivastava, Bokui (William) Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain and 6 more. Conference on Robot Learning, 2021

329 citations recorded by the source corpus.

Recent research in embodied AI has been boosted by the use of simulation environments to develop and train robot learning approaches. However, the use of simulation has skewed the attention to tasks that only require what robotics simulators can simulate: motion and physical contact. We present iGibson 2.0, an open-source simulation environment that supports the simulation of a more diverse set of household tasks through three key innovations. First, iGibson 2.0 supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks. Second, iGibson 2.0 implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked. Additionally, given a logic state, iGibson 2.0 can sample valid physical states that satisfy it. This functionality can generate potentially infinite instances of tasks with minimal effort from the users. The sampling mechanism allows our scenes to be more densely populated with small objects in semantically meaningful locations. Third, iGibson 2.0 includes a virtual reality (VR) interface to immerse humans in its scenes to collect demonstrations. As a result, we can collect demonstrations from humans on these new types of tasks, and use them for imitation learning. We evaluate the new capabilities of iGibson 2.0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new research in embodied AI. iGibson 2.0 and its new dataset are publicly available at http://svl.stanford.edu/igibson/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/aeae048c7d8d4ea03bb9cb1b75c65903c915909a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots (2024, background): “RoboCasa is one of the few frameworks to feature photorealistic rendering [21, 40, 27, 10, 6] and multiple robot embodiments [21, 26, 27, 6, 34].”
A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers (2022, background): “The ROS ecosystem is already integrated in [220], whereas [218] requires the use of external libraries.”
OpenEQA: Embodied Question Answering in the Era of Foundation Models (2024, unknown): “The key technologies enabling this are: (1) videos and scans of real-world environments like ScanNet [6], Gibson [43], and HM3D [36], as well as simulators capable of rending these scenes [11, 21, 24, 32, 39]; and crucially (2) LLMs capable of scoring open-ended answers.”
BEHAVIOR-1K: A Benchmark for Embodied AI with 1, 000 Everyday Activities and Realistic Simulation (2022, methodology): “BEHAVIOR-100 is implemented in iGibson 2.0 [59]; however, realistic simulation of the diverse activities in BEHAVIOR-1K is beyond the capability of iGibson 2.0.”
Advances, challenges and opportunities in creating data for trustworthy AI (2022, background): “In robotics, although real-world challenges provide the ultimate testbed, high-fidelity simulated environments are widely used to enable faster, safer learning on complex and long-horizon tasks 36,37 .”
Objaverse: A Universe of Annotated 3D Objects (2022, methodology): “To put the scale of O BJAVERSE -Interiors in perspective, the number of scenes in O BJAVERSE -Interiors is significantly larger than the 400 or so existing hand-built interactive embodied AI scenes [21,36,39,67].”
Diffusion-based Generation, Optimization, and Planning in 3D Scenes (2023, background): “Planning in 3D Scenes The ability to act and plan in 3D scenes is critical for an intelligent agent and has led to the recent culmination of embodied AI research [28, 30, 33, 36, 54, 55, 79].”
Simple but Effective: CLIP Embeddings for Embodied AI (2021, background): “Simulators are photo-realistic, simulate physics, and enable interaction between agents and their environments [7, 8, 15, 16, 25, 26, 30, 36].”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “Table adapted from [188], [189].”
Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments (2023, background): “c) Scene designing and asset handling: Frameworks support scene creation procedurally [1], [2], [11], via mesh scans [6], [7] or through game-engine style interfaces [9], [26].”
TidyBot: Personalized Robot Assistance with Large Language Models (2023, methodology): “Below, we describe how we use the LLM to infer personalized rules for both receptacle selection and manipulation primitive selection, and also how we deploy the approach on a real-world mobile manipulation system for household cleanup.”
VIMA: General Robot Manipulation with Multimodal Prompts (2022, background): “For example, iGibson (Shen et al., 2020; Li et al., 2021; Srivastava et al., 2021) simulates interactive household scenarios.”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, unknown): “Restrictions apply.”