Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight

By Valts Blukis, Yannick Terme, Eyvind Niklasson, Ross A. Knepper, Yoav Artzi. 2019

82 citations recorded by the source corpus.

We propose a joint simulation and real-world learning framework for mapping navigation instructions and raw first-person observations to continuous control. Our model estimates the need for environment exploration, predicts the likelihood of visiting environment positions during execution, and controls the agent to both explore and visit high-likelihood positions. We introduce Supervised Reinforcement Asynchronous Learning (SuReAL). Learning uses both simulation and real environments without requiring autonomous flight in the physical environment during training, and combines supervised learning for predicting positions to visit and reinforcement learning for continuous control. We evaluate our approach on a natural language instruction-following task with a physical quadcopter, and demonstrate effective execution and exploration behavior.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Experience Grounds Language (2020, background): “, 2020) or map language to physical robot control (Blukis et al., 2019; Chai et al., 2018).”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, background): “Some works utilize language for planning and grounding [39, 40, 41, 42, 43, 44].”
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation (2021, background): “Numerous prior works have also studied learning language conditioned rewards online from demonstrations or examples of successful completion of tasks and language annotations [5, 22, 39, 49].”
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards (2020, background): “Blukis et al. [20] generate a state visitation distribution given a natural language instruction, which is then used to generate rewards for policy training.”
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution (2021, background): “Modelling improvements have enabled these approaches to achieve good performance on complex navigation tasks [7, 9, 11, 13, 14, 37], a success that has not yet translated to mobile manipulation [12, 46, 47].”
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models (2022, background): “a) Language instruction following in robotics: Language-instruction following agents have been extensively explored with engineered symbolic representations [17, 45], with reinforcement learning (RL) [5, 20, 29], and with imitation learning [3, 6, 24, 30].”
Modularity Improves Out-of-Domain Instruction Following (2020, background): “Prior work has found that modular networks can generalize to new environments or domains through module specialization (Hu et al., 2019b; Blukis et al., 2020), and that modular networks often perform well in compositional generalization because of their composable structure (Devin et al., 2017;…”
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring (2021, unknown): “To facilitate task learning from natural language instructions, several benchmarks using simulated physical environment have been made available (Anderson et al., 2018; Misra et al., 2018; Blukis et al., 2019; Shridhar et al., 2020).”
Correcting Robot Plans with Natural Language Feedback (2022, background): “Recently, instruction following has been studied in robotics by mapping instructions and raw visual observations to actions using end-to-end representation learning and sim-toreal transfer [8, 2, 35].”
Improving Intrinsic Exploration with Language Abstractions (2022, background): “LEARN [22]; also [3, 5, 21, 23, 44, 53]) or prior approaches that assume that messages are always helpful (Harrison et al.”