Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered Environments

By Fei Xia, Bokui (William) Shen, Chengshu Li, Priya Kasimbeg, Micael E. Tchapmi, Alexander Toshev, Roberto Mart'in-Mart'in, Silvio Savarese. 2019

213 citations recorded by the source corpus.

We present <italic>Interactive Gibson Benchmark</italic>, the first comprehensive benchmark for training and evaluating <italic>Interactive Navigation</italic> solutions. Interactive Navigation tasks are robot navigation problems where physical interaction with objects (e.g., pushing) is allowed and even encouraged to reach the goal. Our benchmark comprises two novel elements: 1) a new experimental simulated environment, the <italic>Interactive Gibson Environment</italic>, that generate photo-realistic images of indoor scenes and simulates realistic physical interactions of robots and common objects found in these scenes; 2) the <italic>Interactive Navigation Score</italic>, a novel metric to study the interplay between navigation and physical interaction of Interactive Navigation solutions. We present and evaluate multiple learning-based baselines in Interactive Gibson Benchmark, and provide insights into regimes of navigation with different trade-offs between navigation, path efficiency and disturbance of surrounding objects. We make our benchmark publicly available<xref ref-type="fn" rid="fn1">1</xref><fn id="fn1"><label>1</label>[Online]. Available: <uri>https://sites.google.com/view/interactivegibsonenv</uri>.</fn> and encourage researchers from related robotics disciplines (e.g., planning, learning, control) to propose, evaluate, and compare their Interactive Navigation solutions in Interactive Gibson Benchmark.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Evaluation & Benchmarks, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Evaluation & Benchmarks

In Robotics & Control

Citations in the corpus

Habitat 2.0: Training Home Assistants to Rearrange their Habitat (2021, background): “How are interactive simulators built today? Either by relying on game engines [7, 51, 52] or via a ‘homebrew’ integration of existing rendering and physics libraries [8, 35, 37, 53].”
A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “Currently, AI2-THOR [19], Habitat Sim [23], iGibson [24] are the front runner ups to bridging the gaps between simulation and real-world.”
robosuite: A Modular Simulation Framework and Benchmark for Robot Learning (2020, background): “In recent years, advances in physics-based simulations and graphics have led to a series of simulated platforms and toolkits [1, 14, 8, 2, 16] that have accelerated scientific progress on robotics and embodied AI.”
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments (2021, unknown): “The Interactive Gibson Benchmark [27] trades off some visual realism for physically realistic object manipulation in order to benchmark interactive visual navigation.”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “Gibson [190] 572/- - - Pybullet Navi RGB, D, N, S - - Navi only - iGibson [189], [191], [192] 15/108 Object States, Task Planning.”
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes (2020, methodology): “This process is radically different from the annotation we performed for the Interactive Gibson Benchmark [35].”
Pushing it out of the Way: Interactive Visual Navigation (2021, methodology): “Our task is closer to existing tasks that consider navigation among movable obstacles [4, 18, 23, 31, 32, 43, 44].”
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget (2020, background): “Galvanized by fast simulation platforms [15, 20, 21, 29], large rich 3D datasets [4, 25, 30], and the success of deep reinforcement learning [18, 24, 26], training virtual robots (embodied agents) in simulation has garnered considerable interest in recent years.”
ReLMoGen: Integrating Motion Generation in Reinforcement Learning for Mobile Manipulation (2021, methodology): “Interactive Navigation (IN) Tasks: In these tasks the robot needs to interact with the environment to change the environment’s state in order to facilitate or enable navigation [45].”
iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks (2021, background): “Recent years, we have seen the emergence of many simulation environments for robotics and embodied AI research [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].”
ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation (2020, methodology): “Platform Scene (I,O) Physics (R/R,S,C,F) Acoustic (E,P) Interaction (D,A,H) Models (L,E) Deepmind Lab [6] D, A MuJuCo [36] R, C, S D, A PyBullet [9] R, C, S E D, A HoME [40] R E VirtualHome [31] I D, A Gibson [42] I iGibson [41] I R D, A ,H L Sapien [43] I R D, A L Habitat [33] I E AI2-THOR [21] I R”
The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark Towards Physically Realistic Embodied AI (2021, unknown): “iGibson [46, 45] made initial progress in this area, introducing an interactive navigation task with partial observations in a 3D cluttered environment.”
Exploiting Scene-specific Features for Object Goal Navigation (2020, background): “er diﬀerent research areas connected to VisualNavigation. 2.1 Simulators and 3D Datasets for Visual Navigation In recent years a large number of diﬀerent simulators have been developed. GibsonEnv [32][33] and AI2Thor [14] both allow to simulate multi-agent situations and to interact with objects, fo”
AllenAct: A Framework for Embodied AI Research (2020, methodology): “…focused on manipulation (such as RL-Bench [5], Sapien [6], and Meta-world [7]), using advanced physics simulations (such as MuJoCo [8] and ThreeDWorld [9]), and also physical counterparts to simulation environments (RoboTHOR [10] and iGibson [11]) to enable research in simulation-to-real transfer.”
iGibson, a Simulation Environment for Interactive Tasks in Large Realistic Scenes (2020, methodology): “This process is radically different from the annotation we performed for the Interactive Gibson Benchmark [35].”
Large Batch Simulation for Deep Reinforcement Learning (2021, background): “…game engines (Kolve et al., 2017; Dosovitskiy et al., 2017; Lee et al., 2019; Gan et al., 2020; James et al., 2020), and those built from scratch using open-source 3D graphics and physics libraries (Savva et al., 2017; 2019; Xia et al., 2018; 2020; Xiang et al., 2020; Zeng et al., 2020).”