The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation

By Kaichun Mo, Haoxiang Li, Zhe L. Lin, Joon-Young Lee. 2018

30 citations recorded by the source corpus.

Deep reinforcement learning (DRL) demonstrates its potential in learning a model-free navigation policy for robot visual navigation. However, the data-demanding algorithm relies on a large number of navigation trajectories in training. Existing datasets supporting training such robot navigation algorithms consist of either 3D synthetic scenes or reconstructed scenes. Synthetic data suffers from domain gap to the real-world scenes while visual inputs rendered from 3D reconstructed scenes have undesired holes and artifacts. In this paper, we present a new dataset collected in real-world to facilitate the research in DRL based visual navigation. Our dataset includes 3D reconstruction for real-world scenes as well as densely captured real 2D images from the scenes. It provides high-quality visual inputs with real-world scene complexity to the robot at dense grid locations. We further study and benchmark one recent DRL based navigation algorithm and present our attempts and thoughts on improving its generalizability to unseen test targets in the scenes.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Karen Simonyan
2. Sergey Levine
3. Justin Fu
4. Andrew Zisserman
5. Ofir Nachum
6. Raia Hadsell
7. Aviral Kumar
8. George Tucker

In Artificial Intelligence

1. Sergey Levine
2. Ruslan Salakhutdinov
3. Pieter Abbeel
4. Chelsea Finn
5. Yoshua Bengio
6. Trevor Darrell
7. Ofir Nachum
8. Abhinav Kumar Gupta

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Andrew Zisserman
3. Trevor Darrell
4. Jitendra Malik
5. Dacheng Tao
6. Abhinav Kumar Gupta
7. Xiaogang Wang
8. Yi Yang

In Reinforcement Learning

1. Sergey Levine
2. Ofir Nachum
3. Aviral Kumar
4. George Tucker
5. Pieter Abbeel
6. Shixiang Shane Gu
7. Chelsea Finn
8. Justin Fu

In Robotics & Control

1. Aviral Kumar
2. Justin Fu
3. Karl Pertsch
4. George Tucker
5. Ofir Nachum
6. Sergey Levine
7. Joseph J. Lim
8. Raia Hadsell

Citations in the corpus

• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “In the domain of robotic navigation, Mo et al. (2018) propose a dataset of visual indoor scenes for reinforcement learning, collected via a camera-mounted-robot.”
• Accelerating Reinforcement Learning with Learned Skill Priors (2020, background): “On the other hand, there is an abundance of collected agent experience available in domains like autonomous driving [3], indoor navigation [24], or robotic manipulation [4, 2].”
• Cross-View Policy Learning for Street Navigation (2019, background): “There has been a growing interest in building and benchmarking visual navigation using complex simulators [21, 38] or photorealistic indoor environments [31].”
• Datasets for Data-Driven Reinforcement Learning (2020, background): “Moreover, significant efforts have been made to incorporate large-scale datasets into off-policy RL [Kalashnikov et al., 2018, Mo et al., 2018, Gu et al., 2017], but these works generally use large numbers of robots to collect online interaction during training.”
• Learning to Navigate in Cities Without a Map (2018, methodology): “Photographic environments have been used to train agents on short navigation problem in indoor scenes with limited scale [9, 1, 7, 35].”