Unifying Map and Landmark Based Representations for Visual Navigation

By Saurabh Gupta, David F. Fouhey, Sergey Levine, Jitendra Malik. 2017

78 citations recorded by the source corpus.

This works presents a formulation for visual navigation that unifies map based spatial reasoning and path planning, with landmark based robust plan execution in noisy environments. Our proposed formulation is learned from data and is thus able to leverage statistical regularities of the world. This allows it to efficiently navigate in novel environments given only a sparse set of registered images as input for building representations for space. Our formulation is based on three key ideas: a learned path planner that outputs path plans to reach the goal, a feature synthesis engine that predicts features for locations along the planned path, and a learned goal-driven closed loop controller that can follow plans given these synthesized features. We test our approach for goal-driven navigation in simulated real world environments and report performance gains over competitive baseline approaches.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “Also trained with DAGGer , Gupta et al. (2017b) unify map-based spatial reasoning and path planning.”
A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “17], which might be useful for downstream tasks like visual navigation [49, 50, 47].”
On Evaluation of Embodied Navigation Agents (2018, background): “The next level of architectural sophistication is to use more advanced memory mechanisms that support the construction of rich internal representations of the agent’s environment [12, 13, 22, 23, 27].”
Neural Modular Control for Embodied Question Answering (2018, background): “Recent deep reinforcement learning-based approaches to this explore it in 2D gridworlds [16, 20, 21], simple visual [22–27] and textual [28, 29] environments, perceptually-realistic 3D home simulators [1, 30–33], as well as real indoor scenes [34–36].”
Two Body Problem: Collaborative Visual Task Completion (2019, background): “Consequently the task is well suited for an RL formulation, a perspective which has become popular recently [62, 1, 16, 17, 33, 42, 86, 59, 5, 8, 90, 25, 36, 91, 37].”
Cross-View Policy Learning for Street Navigation (2019, methodology): “[13] introduced landmark- and map-based navigation using a spatial representation for path planning and a goal-driven closed-loop controller for executing the plan.”
Bayesian Relational Memory for Semantic Visual Navigation (2019, background): “Two categories exist for navigation-specific memory: the spatial memory [42, 24] and topological memory [25, 46].”
Learning to Visually Navigate in Photorealistic Environments Without any Supervision (2020, unknown): “This map has been further extended with image features [14] or with a dynamic structure [2, 15].”
Learning to plan with uncertain topological maps (2020, unknown): “It has been shown that humans and other animals navigate through the use of waypoints combined with a local locomotion policy [55,22].”
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks (2019, background): “They are applied to visual navigation [15, 16, 32, 47], interactive question answering [13], and localization [7, 18, 20].”
Generative Temporal Models with Spatial Memory for Partially Observed Environments (2018, background): “In particular, in (Gupta et al., 2017a;b; Zhang et al., 2017; Parisotto & Salakhutdinov, 2018) the memory architectures have a ﬁxed number of slots that are spatially structured as a 2D grid, and can therefore store information on the moving agent.”
Deep Visual MPC-Policy Learning for Navigation (2019, background): “…don’t rely on traditional analytic Image Jacobians nor SLAM-like systems[20, 21, 22, 2] to map and localize and control motion, but rather utilize recent advances in machine learning, perception and reasoning to learn control policies that link images to navigation commands [23, 24, 25, 26, 4].”
An Exploration of Embodied Visual Exploration (2020, unknown): “Embodied active perception [1, 8, 7, 54] tackles these problems by learning task-speciﬁc motion policies for navigation [58, 22, 21, 44, 4] and recognition [34, 27, 57] often via deep reinforcement learning (RL), where the agent is rewarded for reaching a speciﬁc target or inferring the right label.”
Occupancy Anticipation for Efficient Exploration and Navigation (2020, background): “Explicit map-based navigation models [21,46,19,11] usually outperform their implicit counterparts by being more sample-efficient, generalizing well to unseen environments, and even transferring from simulation to real robots [20,10].”
Learning to Navigate in Cities Without a Map (2018, methodology): “The architecture in [21] uses an explicit neural mapper and planner for navigation tasks as well as registered pairs of landmark images and poses.”
Audio-Visual Embodied Navigation (2019, background): “As a result, to an agent standing in the bottom right room, the audio reveals the door as a good intermediate goal. environments [117, 42, 41, 81, 5, 66, 61, 98].”
Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal (2018, background): “…mapless navigation, in which the agent navigates solely on the basis of its egocentric sensor modalities: mapping these inputs to actions with appearance-based matching and path recall [3], or by a process of implicit localisation and mapping in some form of latent representation [4, 5, 6].”