Imitation learning for natural language direction following through unknown environments

By Felix Duvallet, Thomas Kollar, Anthony Stentz. 2013

61 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Natural Language Processing

Citations in the corpus

LILA: Language-Informed Latent Actions (2021, background): “Other work that relaxes the planning/dynamics assumption is Duvallet et al. [53]; this work uses hand-engineered features on top of a popular logical form – Spatial Description Clauses (SDCs) – to learn policies for robot navigation.”
Reinforcement and Imitation Learning via Interactive No-Regret Learning (2014, background): “sets and lists [22], parse trees [10], image par sing [20, 25] and natural language understanding[12].”
Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations (2016, background): “Other successful examples include teaching a robotic wheelchair to navigate to target positions [11], teaching a robot to follow verbal instructions to navigate across an office building [8]”
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA (2018, background): “…instructions in this simple environment, researchers have achieved state of the art performance with deep recurrent models (Mei et al., 2016), in contrast to inference relying on traditional, symbolic representations of language for navigation (e.g., (Duvallet et al., 2013) and many others).”
Executing Instructions in Situated Collaborative Interactions (2019, background): “Executing single instructions has been widely studied (e.g., Tellex et al., 2011; Duvallet et al., 2013; Misra et al., 2017, 2018; Anderson et al., 2018; Blukis et al., 2018a,b; Chen et al., 2019).”
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning (2018, methodology): “Mapping natural language instructions to actions has been studied extensively, both using physical robots [7, 16, 25, 31, 34, 49, 50] and virtual agents [2, 4, 29, 30, 32].”
Learning models for following natural language directions in unknown environments (2015, background): “[12] use imitation learning to train a policy that reasons about uncertainty in the grounding and that is able to backtrack as necessary.”
Grounding English Commands to Reward Functions (2015, methodology): “These approaches have been applied to various tasks, the ones closest to ours being interpreting verbal commands in the context of navigational instructions [23, 6, 10], robot manipulation [9, 22, 11], and puzzle solving and software control [3].”
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight (2019, background): “Natural language instruction following has been extensively studied using hand-engineered symbolic intermediate representations of world state or instruction semantics with physical robots [1, 2, 3, 4, 17, 5, 18, 6, 19] and simulated agents [20, 21, 22, 23, 24, 25].”
SHIV: Reducing supervisor burden in DAgger using support vectors for efficient learning from demonstrations in high dimensional state spaces (2016, background): “DAgger and related algorithms have been applied in a wide range of applications, from quadrotor flight to natural language to Atari games [12], [10], [28].”
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models (2022, background): “a) Language instruction following in robotics: Language-instruction following agents have been extensively explored with engineered symbolic representations [17, 45], with reinforcement learning (RL) [5, 20, 29], and with imitation learning [3, 6, 24, 30].”
Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following (2020, background): “Natural language instruction following on physical robots is most commonly studied using handengineered symbolic representations of world state or instruction semantics [1, 7, 2, 3, 4, 8, 5, 9], which require representation design and state estimation pipelines that are hard to scale to complex envi”
Vision-Based Navigation With Language-Based Assistance via Imitation Learning With Indirect Intervention (2018, background): “Proposals include a variety of grounded parsing models that are trained from data [41, 42, 36] and models that interact with robots via natural language queries against a knowledge base [52] Most relevant to the present work are [44] who ground natural language to robotic manipulator instructions us”