Benchmarking Classic and Learned Navigation in Complex 3D Environments

By Dmytro Mishkin, Alexey Dosovitskiy, Vladlen Koltun. 2019

78 citations recorded by the source corpus.

Navigation research is attracting renewed interest with the advent of learning-based methods. However, this new line of work is largely disconnected from well-established classic navigation approaches. In this paper, we take a step towards coordinating these two directions of research. We set up classic and learning-based navigation systems in common simulated environments and thoroughly evaluate them in indoor spaces of varying complexity, with access to different sensory modalities. Additionally, we measure human performance in the same environments. We find that a classic pipeline, when properly tuned, can perform very well in complex cluttered environments. On the other hand, learned systems can operate more robustly with a limited sensor suite. Overall, both approaches are still far from human-level performance.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “In the latter case, the agent builds the map as it navigates an unseen test environment [57, 58, 44], which”
Habitat: A Platform for Embodied AI Research (2019, methodology): “However, the performance of the Habitat stack relative to MINOS [22] used in [19] – thousands vs.”
Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance? (2019, unknown): “For a comparison between simulation-trained and non-learning-based navigation methods, we refer the reader to previous studies [1], [31], [32].”
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames (2019, methodology): “Visual navigation in indoor environments has been the subject of many recent works (Gupta et al., 2017; Das et al., 2018; Anderson et al., 2018b; Savva et al., 2019; Mishkin et al., 2019).”
Audio-Visual Embodied Navigation (2019, background): “As a result, to an agent standing in the bottom right room, the audio reveals the door as a good intermediate goal. environments [117, 42, 41, 81, 5, 66, 61, 98].”
An Exploration of Embodied Visual Exploration (2020, unknown): “In the spirit of recent inﬂuential studies in other domains [17, 33, 20, 36, 35], we aim ﬁrst and foremost to provide a reliable, unbiased, and thorough view of the state of the art that can be a reference to the ﬁeld moving forward.”
Bayesian Relational Memory for Semantic Visual Navigation (2019, background): “There are also nice summaries of recent progresses [2, 37].”
Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation (2021, methodology): “However, in the context of downstream tasks, such as indoor robot navigation, a number of difficulties remain [8; 43; 12].”
Learning Object-conditioned Exploration using Distributed Soft Actor Critic (2020, background): “Learned Navigation System In the last few years a large body of work on learned navigation policies has emerged [5, 6, 7, 8].”
SoundSpaces: Audio-Visual Navigation in 3D Environments (2019, background): “Building on recent embodied visual navigation work [111,41,40,4,64,59], we take a deep reinforcement learning approach, and we introduce audio to the observation.”
Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation (2019, background): “3 [30] Dmytro Mishkin, Alexey Dosovitskiy, and Vladlen Koltun.”