Natural language command of an autonomous micro-air vehicle

By Albert S. Huang, Stefanie Tellex, Abraham Bachrach, Thomas Kollar, Deb K. Roy, Nicholas A. Roy. 2010

52 citations recorded by the source corpus.

Natural language is a flexible and intuitive modality for conveying directions and commands to a robot but presents a number of computational challenges. Diverse words and phrases must be mapped into structures that the robot can understand, and elements in those structures must be grounded in an uncertain environment. In this paper we present a micro-air vehicle (MAV) capable of following natural language directions through a previously mapped and labeled environment. We extend our previous work in understanding 2D natural language directions to three dimensions, accommodating new verb modifiers such as go up and go down, and commands such as turn around and face the windows. We demonstrate the robot following directions created by a human for another human, and interactively executing commands in the context of surveillance and search and rescue in confined spaces. In an informal study, 71% of the paths computed from directions given by one user terminated within 10m of the desired destination.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Natural Language Processing, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Robotics & Control

Citations in the corpus

Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments (2017, background): “This is typically achieved either by assuming that the set of all navigation goals, or objects to be acted upon, has been enumerated, and that each will be identified by label [11, 43], or by operating in severely restricted or simulated environments requiring limited perception [10, 16, 19, 24, 29,”
Robots That Use Language (2020, background): “For example, Chen et al. (84) created a system that learns to generate natural language descriptions it y o of RoboCup soccer games by probabilistically mapping between word meanings and game events.”
Natural Language Acquisition and Grounding for Embodied Robotic Systems (2017, background): “Combining language and vision is also used to learn natural language commands for robotic systems, for example, learning linguistic instructions to navigate autonomous mobile robots (Lauria et al. 2002; Huang et al. 2010; Matuszek et al. 2013), or to navigate autonomous drones (Tellex et al.”