Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal

By Jake Bruce, Niko Sünderhauf, Piotr Wojciech Mirowski, Raia Hadsell, Michael Milford. 2018

37 citations recorded by the source corpus.

Model-free reinforcement learning has recently been shown to be effective at learning navigation policies from complex image input. However, these algorithms tend to require large amounts of interaction with the environment, which can be prohibitively costly to obtain on robots in the real world. We present an approach for efficiently learning goal-directed navigation policies on a mobile robot, from only a single coverage traversal of recorded data. The navigation agent learns an effective policy over a diverse action space in a large heterogeneous environment consisting of more than 2km of travel, through buildings and outdoor regions that collectively exhibit large variations in visual appearance, self-similarity, and connectivity. We compare pretrained visual encoders that enable precomputation of visual embeddings to achieve a throughput of tens of thousands of transitions per second at training time on a commodity desktop computer, allowing agents to learn from millions of trajectories of experience in a matter of hours. We propose multiple forms of computationally efficient stochastic augmentation to enable the learned policy to generalise beyond these precomputed embeddings, and demonstrate successful deployment of the learned policy on the real robot without fine tuning, despite environmental appearance differences at test time. The dataset and code required to reproduce these results and apply the technique to other datasets and robots is made publicly available at rl-navigation.github.io/deployable.

Subfields: Aerospace Engineering, Computer Networks and Communications, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

Citations in the corpus

BADGR: An Autonomous Self-Supervised Learning-Based Navigation System (2020, background): “However, these methods have typically focused on pure geometric reasoning, require on-policy data, and often utilize simulation due to constraints such as sample efﬁciency [4], [18]–[20]; bridging the sim-to-real gap remains a challenging open research problem.”
Cognitive Mapping and Planning for Visual Navigation (2017, methodology): “A related and parallel stream of research studies whether or not models trained in simulators can be effectively transferred to the real world (Sadeghi and Levine 2017; Bruce et al. 2018), and how the domain gap between simulation and the real world may be reduced (Xia et al. 2018).”
Visual Representations for Semantic Target Driven Navigation (2018, background): “[20] learn navigation policies from a single traversal of the environment.”
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action (2022, methodology): “The topological graph G is an important abstraction that allows a simple interface for planning over past experience in the environment and has been successfully used in prior work to perform long-horizon navigation [52, 60, 61].”
Benchmarking Classic and Learned Navigation in Complex 3D Environments (2019, background): “Navigation can be interpreted within the framework of reinforcement learning (RL) and addressed with general-purpose RL algorithms [31, 30, 21, 12, 47, 37, 14, 43, 7].”
Neural Topological SLAM for Visual Navigation (2020, background): “This could be in the form of a simply wandering around without colliding [15, 33], following an object [22], getting to a goal coordinate [1,17]: using sequence of images along the path [5, 21], or language-based instructions [2].”