The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation

By Chih-Yao Ma, Zuxuan Wu, Ghassan Al-Regib, Caiming Xiong, Zsolt Kira. 2019

198 citations recorded by the source corpus.

As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making. Specifically, the Vision and Language Navigation (VLN) task involves navigating to a goal purely from language instructions and visual information without explicit knowledge of the goal. Recent successful approaches have made in-roads in achieving good success rates for this task but rely on beam search, which thoroughly explores a large number of trajectories and is unrealistic for applications such as robotics. In this paper, inspired by the intuition of viewing the problem as search on a navigation graph, we propose to use a progress monitor developed in prior work as a learnable heuristic for search. We then propose two modules incorporated into an end-to-end architecture: 1) A learned mechanism to perform backtracking, which decides whether to continue moving forward or roll back to a previous state (Regret Module) and 2) A mechanism to help the agent decide which direction to go next by showing directions that are visited and their associated progress estimate (Progress Marker). Combined, the proposed approach significantly outperforms current state-of-the-art methods using greedy action selection, with 5% absolute improvement on the test server in success rates, and more importantly 8% on success rates normalized by the path length.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

How Much Can CLIP Benefit Vision-and-Language Tasks? (2021, neutral): “On R2R (Wang et al., 2018) 25 23 S-Follower (Fried et al., 2018) 35 28 RCM (Wang et al., 2019) 43 38 SMNA (Ma et al., 2019a) 48 35 Regretful (Ma et al., 2019b) 48 40 FAST-Short (Ke et al., 2019) 54 41 EnvDrop (Tan et al., 2019) 51 47 PRESS (Li et al., 2019b) 49 45 ALTR (Huang et al., 2019a) 48 45 CG”
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks (2019, methodology): “Existing beam-search [17, 47, 52] and backtracking solutions [24, 28] are infeasible due to the larger action and state spaces, long horizon, and inability to undo certain actions.”
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training (2020, methodology): “To improve the alignment of the instruction and visual scenes, a visualtextual co-grounding attention mechanism was proposed in [18], which is further improved with a progress monitor [19].”
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation (2018, background): “Concurrent to our work, [42, 24, 25, 26] studies the VLN tasks from various aspects, and [30] introduces a variant of the VLN task to find objects by requesting language assistance when needed.”
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web (2020, background): “In this setting, a high-probability path is typically generated by performing beam search through the environment and ranking paths according to either: (1) their probability under a ‘follower’ model [3,8,13,20,21,30,31], as in Figure 1 (left), or (2) by how well they explain the instruction accordi”
Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning (2019, methodology): “Our approach takes into account the agent’s past and future behavior to determine actions that are most and least beneﬁcial to them, combining the advantages of both model-based and progress-estimating meth-ods (Wang et al., 2018; Ma et al., 2019a,b).”
Robust Navigation with Language Pretraining and Stochastic Sampling (2019, methodology): “Recent work, e.g., using search strategies (Ke et al., 2019) or progress monitors (Ma et al., 2019b), has focused on mitigating the cost of computing top-k rollouts.”
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation (2019, neutral): “NE Random Student-forcing [1] (CVPR 2018) RPA [2] (ECCV 2018) Speaker-Follower [3] (NeurIPS 2018) RCM [4] (CVPR 2019) Self-Monitoring (ICLR 2019) Regretful [5] (CVPR 2019) Evaluation metrics: • Navigation Error (NE): mean of the shortest path distance between the agent’s final position and the goal ”
Episodic Transformer for Vision-and-Language Navigation (2021, background): “In addition, auxiliary tasks, such as progress estimation [39, 40], back-tracking [30], speaker-driven route selection [20], cross-modal matching [29, 68], back translation [60], pretraining on subtasks [74], and text-based pretraining [14,57] are proposed to improve the performance and…”
Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters (2019, background): “Recent works on VLN [11, 17, 18, 21] integrate the agent with a simplified action space in which it “only needs to make high-level decisions as to which navigable direction to go next” [11].”
FILM: Following Instructions in Language with Modular Methods (2021, methodology): “For Visual Language Navigation, which is the most comparable to the setting of our work, methods with impressive performances (Ke et al., 2019; Wang et al., 2019; Ma et al., 2019) have been proposed since the introduction of R2R (Anderson et al., 2018b).”
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampling (2019, background): “Vision-and-Language Navigation Navigation in 3D environments based on natural language instruction has recently been investigated by many studies [3, 8, 21, 22, 26, 20, 33, 25, 11, 32, 31, 16].”
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments (2019, methodology): “A range of VLN methods [9, 15, 19, 20, 27, 28] have been proposed to address this VLN task.”
Language and Visual Entity Relationship Graph for Agent Navigation (2020, methodology): “Later, the Self-Monitoring agent [22] applies a visual-textual co-grounding network and a progress monitor to guide the transition of language attention, this work is further improved by path-scoring and backtracking methods, such as the Regretful agent [23] and the Tactical Rewind [18].”
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA (2018, unknown): “, Matterport: (Ma et al., 2019a; Ke et al., 2019; Wang et al., 2019; Ma et al., 2019b; Tan et al., 2019; Fried et al., 2018), and EQA: (Das et al.”
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring (2021, methodology): “, 2019), the Regretful Agent (Ma et al., 2019b), and the environment drop-out model (Tan et al.”
Chasing Ghosts: Instruction Following as Bayesian State Tracking (2019, background): “Other work has focused on developing modules for estimating progress towards the goal [4] and learning when to backtrack [5, 8].”
Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information (2021, background): “…Navigation task with better interactions between natural language instructions and visual scenes (Fried et al., 2018; Wang et al., 2019; Landi et al., 2019; Wang et al., 2020a; Huang et al., 2019a; Hu et al., 2019; Majumdar et al., 2020; Ma et al., 2019a; Qi et al., 2020b; Zhu et al., 2020a,c).”