A Deep Compositional Framework for Human-like Language Acquisition in Virtual Environment

By Haonan Yu, Wei Xu. 2017

26 citations recorded by the source corpus.

We tackle a task where an agent learns to navigate in a 2D maze-like environment called XWORLD. In each session, the agent perceives a sequence of raw-pixel frames, a natural language command issued by a teacher, and a set of rewards. The agent learns the teacher's language from scratch in a grounded and compositional manner, such that after training it is able to correctly execute zero-shot commands: 1) the combination of words in the command never appeared before, and/or 2) the command contains new object concepts that are learned from another task but never learned from navigation. Our deep framework for the agent is trained end to end: it learns simultaneously the visual representations of the environment, the syntax and semantics of the language, and the action module that outputs actions. The zero-shot learning capability of our framework results from its compositionality and modularity with parameter tying. We visualize the intermediate outputs of the framework, demonstrating that the agent truly understands how to solve the problem. We believe that our results provide some preliminary insights on how to train an agent with similar abilities in a 3D environment.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Grounded Language Learning in a Simulated 3D World (2017, background): “…unsurprisingly, the few systems that attempt to learn language grounding in artiﬁcial agents do so with respect to environments that are far simpler than the continuous, noisy sensory experience encountered by humans (Steels, 2008; Roy and Pentland, 2002; Krening et al., 2016; Yu et al., 2017).”
Programmable Agents (2017, background): “Compositional neural architectures paired with different forms of attention have led to impressive results in natural language interfaces for database tables, and language acquisition in 2D navigation environments [28, 37].”
Embodied Question Answering (2017, background): “XWORLD [27]), to 3D game-like environments with limited realism ( e.g .”
Gated-Attention Architectures for Task-Oriented Language Grounding (2017, methodology): “We compare our approach to a Concate-nation unit, based on architecture proposed by Yu et al. (2017).”
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent (2017, neutral): “Grounding language in an interactive environment is an active area of research, however a number of recent works employ synthetic, templated language only (Sukhbaatar et al., 2015; Yu et al., 2017; Bordes et al., 2010; Hermann et al., 2017; Mikolov et al., 2015; Chaplot et al., 2017).”
Talk the Walk: Navigating New York City through Grounded Dialogue (2018, background): “…for embodied AI, many of which involve agents following natural language instructions with respect to an environment(Artzi & Zettlemoyer, 2013; Yu et al., 2017; Hermann et al., 2017; Mei et al., 2016; Chaplot et al., 2018b;a), following-up on early work in this area (MacMahon et al., 2006;…”
ELLA: Exploration through Learned Language Abstraction (2021, background): “Even simple instruction following problems are difficult to learn (Lynch & Sermanet, 2020), which has made 2D environments important test-beds for new learning methods (Andreas et al., 2017; Yu et al., 2017; Chevalier-Boisvert et al., 2019).”
Grounding Language in Play (2020, background): “…work often studies restricted environments (MacMahon et al., 2006; Kollar et al., 2010; Andreas et al., 2016; Oh et al., 2017; Misra et al., 2017; Yu et al., 2017; 2018) and simpliﬁed actuators (Wang et al., 2019; Hill et al., 2019; Chaplot et al., 2018; Goyal et al., 2019; Shridhar et al.,…”