Gibson Env: Real-World Perception for Embodied Agents

By Fei Xia, Amir Zamir, Zhi-Yang He, Alexander Sax, Jitendra Malik, Silvio Savarese. 2018

1,020 citations recorded by the source corpus.

Developing visual perception models for active agents and sensorimotor control in the physical world are cumbersome as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a question on whether the results transfer to real-world. In this paper, we investigate developing real-world perception for active agents, propose Gibson Environment for this purpose, and showcase a set of perceptual tasks learned therein. Gibson is based upon virtualizing real spaces, rather than artificially designed ones, and currently includes over 1400 floor spaces from 572 full buildings. The main characteristics of Gibson are: I. being from the real-world and reflecting its semantic complexity, II. having an internal synthesis mechanism "Goggles" enabling deploying the trained models in real-world without needing domain adaptation, III. embodiment of agents and making them subject to constraints of physics and space.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

  1. Jitendra Malik
  2. Abhinav Kumar Gupta
  3. Silvio Savarese
  4. Kristen Grauman
  5. Vladlen Koltun
  6. Andreas Geiger
  7. Saurabh Gupta
  8. Sergey Levine

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Jitendra Malik
  3. Trevor Darrell
  4. Dacheng Tao
  5. Xiaogang Wang
  6. Kristen Grauman
  7. Ming-Hsuan Yang
  8. Abhinav Kumar Gupta

In Control and Systems Engineering

  1. Sergey Levine
  2. Chelsea Finn
  3. Pieter Abbeel
  4. Abhinav Kumar Gupta
  5. Kristen Grauman
  6. Dieter Fox
  7. Silvio Savarese
  8. Li Fei-Fei

In Robotics & Control

  1. Sergey Levine
  2. Dhruv Batra
  3. Abhinav Kumar Gupta
  4. Silvio Savarese
  5. Vladlen Koltun
  6. Kristen Grauman
  7. Chelsea Finn
  8. Jitendra Malik

Citations in the corpus