Object Goal Navigation using Goal-Oriented Semantic Exploration

By Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Kumar Gupta, Ruslan Salakhutdinov. 2020

786 citations recorded by the source corpus.

This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map-based methods and led to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge. Ablation analysis indicates that the proposed model learns semantic priors of the relative arrangement of objects in a scene, and uses them to explore efficiently. Domain-agnostic module design allow us to transfer our model to a mobile robot platform and achieve similar performance for object goal navigation in the real-world.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “Object Navigation is one of the most straightforward tasks, yet one of the most challenging tasks in embodied AI.”
Visual Room Rearrangement (2021, background): “Various tasks have been proposed in this domain: navigation towards objects [4, 51, 48, 7] or towards a speciﬁc point [1, 38, 47, 36], scene exploration [9, 8], embodied question answering [18, 13], task completion [55], instruction following [2, 41], object manipulation [16, 52], multi-agent…”
Shaping embodied agent behavior with activity-context priors from egocentric video (2021, background): “Our video model relates objects based on goal-oriented actions (e.g., knives used to cut potatoes) rather than spatial co-occurrences (e.g., knives are found near spoons) [65, 11, 47, 64] or semantic similarity (e.g., potatoes are like tomatoes) [65].”
VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation (2023, methodology): “Learning approaches to train robots with semantic navigation abilities have typically leveraged reinforcement learning [11]–[14], learning from demonstration [15], or prediction of semantic top-down maps [3], [16]– [20] on which waypoint planners can be used.”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “Other works have investigated modular designs, e.g., using RL as a global exploration policy together with explicit mapping and local planning (81, 82).”
Semantic Exploration from Language Abstractions and Pretrained Representations (2022, methodology): “[10] tackle navigation in 3D by constructing a semantic map of the environment from pretrained SLAM modules, language-defined object categories, and agent location.”
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget (2020, methodology): “provements on ObjectNav [6], ImageNav [7], and PointNav without GPS+Compass [19].”
Open-vocabulary Queryable Scene Representations for Real World Planning (2022, background): “Recently, methods that leverage pre-trained image-text models can do zero-shot Object Goal Navigation [31], [32].”
FILM: Following Instructions in Language with Modular Methods (2021, methodology): “Embodied instruction following (EIF) presents a more complex and human-like setting than VLN or Object Goal Navigation (Gupta et al., 2017; Chaplot et al., 2020b; Du et al., 2021); beyond just navigation, agents are required to execute sequences of sub-tasks that entail both navigation and…”
From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN) (2020, methodology): “Map-like environment representations are always used for navigation tasks, such as non-parametric landmark graph [Savinov et al., 2018], top-down egocentric obstacle map [Gupta et al., 2017] and semantic map [Chaplot et al., 2020a] built from visual observations.”
Diffusion-based Generation, Optimization, and Planning in 3D Scenes (2023, methodology): “However, existing work relies heavily on model-based planners with the single-step dynamic model [5,11,48,67,74,80], lacking a trajectory-level optimization for long-horizon planning.”
Pushing it out of the Way: Interactive Visual Navigation (2021, background): “A popular task that has received a considerable amount of attention is visual navigation [3, 5, 8, 29, 39, 48], where the goal is to navigate towards a specific coordinate or object within an unseen environment.”
CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration (2022, background): “…exploration, many navigation tasks are “goal-conditioned,” where the agent needs to navigate to a given position (i.e., point goal [58, 72, 74, 67, 11, 30]), a view of the environment (i.e., image goal [78, 43, 56]), or a specified object category (i.e., object goal [69, 10, 9, 75, 39, 1, 64]).”
Exploiting Scene-specific Features for Object Goal Navigation (2020, background): “In particular, [7] leveraged depth images to construct in an online fashion semantic maps of the environment.”
Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigation (2021, background): “Several works have also explored the use of DRL in training visual navigation policies using photorealistic 3D simulators of indoor environments, such as [3], [6], [12], [16], [18], [20].”
ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings (2022, background): “The last few years have 21 witnessed the development of new environments [2, 3, 4, 5, 6], annotated 3D scans [7, 8, 9], datasets 22 of human demonstrations [10], and approaches for ObjectNav [11, 12, 13, 14, 15, 16], cumulatively 23 leading to strong progress.”
The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark Towards Physically Realistic Embodied AI (2021, positive): “• Semantic Exploration: We adapted a baseline from [8].”
Multi-Scale Gaussian-Language Map for Zero-shot Embodied Navigation and Reasoning (2026, unknown): “The first type (DD-PPO [37], Se-mExp [3], SGM [51], T-Diff [45] and GOAL [17]) requires task-specific training and operates on a closed set of object goals.”
Knowledge Enhanced Efficient Robotic Decision Making with Intuitive-Deliberate LLMs (2025, unknown): “Chaplot et al. [6] decomposed the navigation task into three modules: a mapping module that converts visual inputs into a semantic map, a policy module that selects long-term goals, and a path planning module that generates action sequences.”