Semi-parametric Topological Memory for Navigation

By Nikolay Savinov, Alexey Dosovitskiy, Vladlen Koltun. 2018

437 citations recorded by the source corpus.

We introduce a new memory architecture for navigation in previously unseen environments, inspired by landmark-based navigation in animals. The proposed semi-parametric topological memory (SPTM) consists of a (non-parametric) graph with nodes corresponding to locations in the environment and a (parametric) deep network capable of retrieving nodes from the graph based on observations. The graph stores no metric information, only connectivity of locations corresponding to the nodes. We use SPTM as a planning module in a navigation system. Given only 5 minutes of footage of a previously unseen maze, an SPTM-based navigation agent can build a topological map of the environment and use it to confidently navigate towards goals. The average success rate of the SPTM agent in goal-directed navigation across test environments is higher than the best-performing baseline by a factor of three. A video of the agent is available at this https URL

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, methodology): “Machine learning has recently been applied to enable abstract task specification, allow general primitives, or relax constraints [67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78].”
Embodied Question Answering (2017, background): “Visual navigation typically specifies agent goals either implicitly via the reward function [18, 19] (thus training a separate policy for each goal/reward), or explicitly by conditioning on goal state representations [23] like images of target objects [17, 24].”
Continuous Scene Representations for Embodied AI (2022, background): “Neural graphbased [9, 10, 43, 58] and dense representations [8, 25] have been proposed to construct the map of the environment.”
A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “17], which might be useful for downstream tasks like visual navigation [49, 50, 47].”
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System (2020, background): “However, these methods have typically focused on pure geometric reasoning, require on-policy data, and often utilize simulation due to constraints such as sample efficiency [28, 8, 29, 30, 31, 32].”
Cognitive Mapping and Planning for Visual Navigation (2017, methodology): “[65] develop topological representations.”
Playing hard exploration games by watching YouTube (2018, background): “Another approach [33] performs temporal classiﬁcation but limits its categories to frames close or far away in time.”
On Evaluation of Embodied Navigation Agents (2018, background): “Along the spectrum are protocols that give the agent brief exposure (on the order of minutes of subjective experience) to the test environment prior to evaluation; during this pre-liminary exposure the agent can build an internal representation that is then used to support navigation [27].”
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning (2019, methodology): “On this task, we evaluate four state-of-the-art prior methods: hindsight experience replay (HER) (Andrychowicz et al., 2017), distributional RL (C51) (Bellemare et al., 2017), semi-parametric topological memory (SPTM) (Savinov et al., 2018a), and value iteration networks (VIN) (Tamar et al., 2016).”
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks (2019, background): “[36] construct a topolgical map of the environment, and uses it for planning.”
An Exploration of Embodied Visual Exploration (2020, neutral): “This method is similar to the “goal agnostic” baseline in [44].”
Learning Affordance Landscapes for Interaction Exploration in 3D Environments (2020, background): “Exploration policies for visual navigation efficiently map the environment in an unsupervised “preview” stage [12, 50, 18, 11, 47, 46].”
Episodic Curiosity through Reachability (2018, methodology): “Multiple works (Fu et al., 2017; Savinov et al., 2018; Aytar et al., 2018) train a binary classifier for predicting if the distance in time between frames is within a certain threshold or not.”
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation (2019, methodology): “[14], we have also implemented an A3C equipped with LSTM memory as a baseline for our task.”
Rapid Task-Solving in Novel Environments (2020, background): “Savinov et al. (2018) develop an alternative to end-to-end learned planning that learns a distance metric over observations for use with a classical planning algorithm.”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, background): “Methods also investigate the utility of topological and latent memory maps [35,78,37,99], graph-”
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning (2018, background): “[37] propose the use of topological maps for the task of navigation.”
Learning to Reach Goals via Iterated Supervised Learning (2019, methodology): “We additionally consider limited-horizon relabeling, in which only states and goals that are at most 3 steps apart are relabeled, similar to proposals in prior work [Pathak et al., 2018, Savinov et al., 2018].”