Bayesian Relational Memory for Semantic Visual Navigation

By Yi Wu, Yuxin Wu, Aviv Tamar, Stuart J. Russell, Georgia Gkioxari, Yuandong Tian. 2019

117 citations recorded by the source corpus.

We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Neural Topological SLAM for Visual Navigation (2020, background): “Others build a topological representation with explicit semantics [42, 46], which limits tasks and environments that can be tackled.”
SoundSpaces: Audio-Visual Navigation in 3D Environments (2019, background): “Recent AI agents also aggregate egocentric visual inputs [112,111,63,86,46], often with a spatio-temporal memory [40,79,44,101].”
Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation (2020, background): “[43,46] represent priors in the form of knowledge graphs or probabilistic relation graphs which capture room location priors.”
Audio-Visual Embodied Navigation (2019, background): “Following these ﬁndings, recent visual AI agents aggregate egocentric visual inputs [118, 117, 65, 90], often with a spatio-temporal memory [41, 81, 46, 105, 98].”
RoboTHOR: An Open Simulation-to-Real Embodied AI Platform (2020, background): “Semantic navigation approaches [22, 75, 69, 71, 39, 52, 41] attempt to learn the semantics of the target in conjunction with navigation.”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, background): “Methods also investigate the utility of topological and latent memory maps [35,78,37,99], graph-”
From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN) (2020, methodology): “In [Wu et al., 2019b], the authors adopted the visual observations that contain objects as the goals to guide the agent to approach the image-indicated object, while in [Ye et al., 2018], the authors provided the target objects’ images without any contextual information included.”
Learning Object Relation Graph and Tentative Policy for Visual Navigation (2020, background): “[28] propose a Bayesian relational memory to build room correlations.”