Interactive Grounded Language Acquisition and Generalization in a 2D World

By Haonan Yu, Wei Xu. 2018

80 citations recorded by the source corpus.

We build a virtual agent for learning language in a 2D maze-like world. The agent sees images of the surrounding environment, listens to a virtual teacher, and takes actions to receive rewards. It interactively learns the teacher's language from scratch based on two language use cases: sentence-directed navigation and question answering. It learns simultaneously the visual representations of the world, the language, and the action control. By disentangling language grounding from other computational routines and sharing a concept detection function between language grounding and prediction, the agent reliably interpolates and extrapolates to interpret sentences that contain new word combinations or new words missing from training sentences. The new words are transferred from the answers of language prediction. Such a language ability is trained and evaluated on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words. The proposed model significantly outperforms five comparison methods for interpreting zero-shot sentences. In addition, we demonstrate human-interpretable intermediate outputs of the model in the appendix.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Environmental drivers of systematicity and generalization in a situated agent (2019, background): “Relative to more recent experiments on color-shape generalisation (Higgins et al., 2017; Chaplot et al., 2018; Yu et al., 2018), we study a wider range of phenomena, involving abstract modifying predicates (negation) and verbs (‘to lift’, ‘to put’) corresponding to complex behaviours requiring…”
A Survey of Reinforcement Learning Informed by Natural Language (2019, methodology): “Without transfer from a language model (or another language grounding task as in [Yu et al. , 2018]), instruction-following systems cannot generalize to instructions outside of the training distribution containing unseen synonyms or paraphrases ( e.g. “fetch a stick", “return with a stick", “grab a…”
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text (2020, methodology): “Much recent work applies deep learning and policy optimization in end-to-end approaches to learning instruction following [Chaplot et al., 2018, Oh et al., 2017, Bahdanau et al., 2018, Chevalier-Boisvert et al., 2018, Yu et al., 2018b,a, Jiang et al., 2019].”
Language Conditioned Imitation Learning Over Unstructured Data (2020, background): “games [24, 32, 27, 12] and gridworlds [52]), simplified actuators, (e.”
Neural Modular Control for Embodied Question Answering (2018, background): “Recent deep reinforcement learning-based approaches to this explore it in 2D gridworlds [16, 20, 21], simple visual [22–27] and textual [28, 29] environments, perceptuallyrealistic 3D home simulators [1, 30–33], as well as real indoor scenes [34–36].”
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks (2021, background): “Prior work on mapping language and vision to actions has been studied mostly in restricted environments [20], [21], [22] and simplified actuators with discrete motion primitives [23], [24], [25], [26].”
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning (2018, background): “Navigation using language instructions has been explored by various works [3, 6, 17, 47, 29].”
Learning to Follow Language Instructions with Adversarial Reward Induction (2018, background): “[14, 20, 11, 6, 9, 31] consider instruction-following in 2D or 3D environments and reward the agent for arriving at the correct location or object.”
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation (2019, background): “There is also considerable work on vision-language based embodied navigation [15], [16], [17], [18], which map directly from language instructions and visual observations to actions.”
Why Build an Assistant in Minecraft? (2019, background): “[99] Haonan Yu, Haichao Zhang, and Wei Xu.”
Emergent Systematic Generalization in a Situated Agent (2019, background): “Relative to more recent experiments on color-shape generalization (Higgins et al., 2017; Chaplot et al., 2018; Yu et al., 2018), we study a wider range of phenomena, involving abstract modifying predicates (negation) and verbs (‘to lift’, ‘to put’) corresponding to complex behaviours requiring…”
Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game (2018, methodology): “We also exploit experience replay (Wang et al., 2017; Yu et al., 2018).”
Grounding Language in Play (2020, background): “Prior work typically studies restricted observation spaces (e.g. games (Goyal et al., 2019), 2D gridworlds (Yu et al., 2018)), simpliﬁed actuators, (e.g. binary pick and place primitives (Hill et al., 2019)), and synthetic language data (Hermann et al., 2017).”
Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling (2023, background): “, 2020) prompting many improvements to decision making conditioned on language instructions (Yu et al., 2018; Lynch & Sermanet, 2020; Nottingham et al., 2021; Suglia et al., 2021; Kuo et al., 2021; Zellers et al., 2021; Song et al., 2022a; Blukis et al., 2022).”
Visual Semantic Navigation using Scene Priors (2018, methodology): “Yu et al. (2018) address the problem of learning language in a 2D maze, where they can handle unseen word combinations or new sentences that contain unseen words.”
Fast and Slow Learning of Recurrent Independent Mechanisms (2021, methodology): “We conduct experiments in the domain of grounded language learning, in which poor data efﬁciency is one of the major limitations for agents to learn efﬁciently and generalize well (Hermann et al., 2017; Chaplot et al., 2017; Wu et al., 2018; Yu et al., 2018; Chevalier-Boisvert et al., 2018).”