Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation

By Medhini Narasimhan, Erik Wijmans, Xinlei Chen, Trevor Darrell, Dhruv Batra, Devi Parikh. 2020

66 citations recorded by the source corpus.

We introduce a learning-based approach for room navigation using semantic maps. Our proposed architecture learns to predict top-down belief maps of regions that lie beyond the agent's field of view while modeling architectural and stylistic regularities in houses. First, we train a model to generate amodal semantic top-down maps indicating beliefs of location, size, and shape of rooms by learning the underlying architectural patterns in houses. Next, we use these maps to predict a point that lies in the target room and train a policy to navigate to the point. We empirically demonstrate that by predicting semantic maps, the model learns common correlations found in houses and generalizes to novel environments. We also demonstrate that reducing the task of room navigation to point navigation improves the performance further.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Cheston Tan
2. Dhruv Batra
3. Jiafei Duan
4. Samson Yu
5. Irfan Essa
6. Tangyao Li
7. Huaiyu Zhu
8. Erik Wijmans

In Computer Vision and Pattern Recognition

1. Dhruv Batra
2. Erik Wijmans
3. Irfan Essa
4. Cheston Tan
5. Jiafei Duan
6. Samson Yu
7. Luc Van Gool
8. Kristen Grauman

Citations in the corpus

• A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “The internal model can be in forms like a topological graph map [51], semantic map [52], occupancy map [53] or spatial memory [54, 55].”
• How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget (2020, background): “Moreover still, findings and improvements in PointNav with GPS+Compass translate to improved performance on other tasks (PointNav without GPS+Compass [9, 19], ObjectGoal navigation [3, 6], RoomGoal navigation [17]) and to navigation by real robots [14].”
• An Exploration of Embodied Visual Exploration (2020, unknown): “Concurrent to this work, new navigation tasks for room navigation (Narasimhan et al. 2020) and object navigation (Batra et al.”