Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout

By Hao Hao Tan, Licheng Yu, Mohit Bansal. 2019

409 citations recorded by the source corpus.

A grand goal in AI is to build a robot that can accurately navigate based on natural language instructions, which requires the agent to perceive the scene, understand and ground language, and act in the real-world environment. One key challenge here is to learn to navigate in new environments that are unseen during training. Most of the existing approaches perform dramatically worse in unseen environments as compared to seen ones. In this paper, we present a generalizable navigational agent. Our agent is trained in two stages. The first stage is training via mixed imitation and reinforcement learning, combining the benefits from both off-policy and on-policy optimization. The second stage is fine-tuning via newly-introduced ‘unseen’ triplets (environment, path, instruction). To generate these unseen triplets, we propose a simple but effective ‘environmental dropout’ method to mimic unseen environments, which overcomes the problem of limited seen environment variability. Next, we apply semi-supervised learning (via back-translation) on these dropout environments to generate new paths and instructions. Empirically, we show that our agent is substantially better at generalizability when fine-tuned with these triplets, outperforming the state-of-art approaches by a large margin on the private unseen test set of the Room-to-Room task, and achieving the top rank on the leaderboard.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Robotics & Control

Citations in the corpus

Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “Following the back-translation strategy that has been shown to be effective in Machine Translation (Sennrich et al., 2016) and Natural Language Navigation (Fried et al., 2018; Tan et al., 2019), we propose two methods to collect synthetic data.”
On the Evaluation of Vision-and-Language Navigation Instructions (2021, methodology): “To establish benchmarks for navigation instruction generation, we evaluate existing English models (Fried et al., 2018; Tan et al., 2019) using human wayfinders.”
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks (2019, background): “Existing beam-search [17, 47, 52] and backtracking solutions [24, 28] are infeasible due to the larger action and state spaces, long horizon, and inability to undo certain actions.”
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training (2020, methodology): “An environment dropout was proposed [28] to generate more environments based on the limited data, so that it can generalize well to unseen environments.”
Vision-and-Dialog Navigation (2019, methodology): “Moving forward, by formulating NDH as a sequential decision process we can use RL to shape the agent’s policy, as in recent VLN work [37].”
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web (2020, methodology): “Many papers report results trained on augmented data including instructions synthesized by a speaker model [8, 20, 21, 28].”
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding (2020, methodology): “7) is trained without data augmentation from model-generated instructions (Fried et al., 2018; Tan et al., 2019) and with hyperparameters tuned for RxR.”
Robust Navigation with Language Pretraining and Stochastic Sampling (2019, background): “• ENVDROP (Tan et al., 2019): proposed an environment dropout method, which can generate more environments based on the limited seen environments.”
Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information (2021, methodology): “…of research in VLN utilizes soft attention over individual words for cross- modal grounding in both the natural language instruction and the visual scene (Wang et al., 2018, 2019; Tan et al., 2019; Landi et al., 2019; Xia et al., 2020; Wang et al., 2020b,a; Xiang et al., 2020; Zhu et al., 2020b).”
Chasing Ghosts: Instruction Following as Bayesian State Tracking (2019, background): “Key developments include the use of instruction-generation (‘speaker’) models for trajectory re-ranking and data augmentation [6, 7], which have been widely adopted.”
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring (2021, methodology): “Many models have been developed, such as the Speaker-Follower model (Fried et al., 2018), the Self-Monitoring Navigation Agent(Ma et al., 2019a; Ke et al., 2019), the Regretful Agent (Ma et al., 2019b), and the environment drop-out model (Tan et al., 2019).”
Vision-Language Navigation With Self-Supervised Auxiliary Reasoning Tasks (2019, methodology): “Besides, it has been proven that finetuning the agent in an unseen environment effectively reducing the domain gap [41, 37].”
Visual Landmark Selection for Generating Grounded and Interpretable Navigation Instructions (2019, background): “Signiﬁcant efforts have been devoted to the problem of following natural language navigation instructions in perceptually realistic virtual environments [1, 5, 18, 11, 10, 8, 15].”
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA (2018, neutral): “, Matterport: (Ma et al., 2019a; Ke et al., 2019; Wang et al., 2019; Ma et al., 2019b; Tan et al., 2019; Fried et al., 2018), and EQA: (Das et al.”
Interactive Learning from Activity Description (2021, background): “Several papers (Fried et al., 2018b; Tan et al., 2019) apply back-translation to vision-language navigation (Anderson et al.”
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution (2021, background): “Modelling improvements have enabled these approaches to achieve good performance on complex navigation tasks [7, 9, 11, 13, 14, 37], a success that has not yet translated to mobile manipulation [12, 46, 47].”
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments (2020, methodology): “38 Back-Translation [26] 10.”
Room-and-Object Aware Knowledge Reasoning for Remote Embodied Referring Expression (2021, background): “EnvDrop [44] is yet another data augmentation technique, breaking the limitation of variability of seen environments.”