Understanding Grounded Language Learning Agents

By Felix Hill, Karl Moritz Hermann, Phil Blunsom, Stephen Clark. 2017

32 citations recorded by the source corpus.

Neural network-based systems can now learn to locate the referents of words and phrases in images, answer questions about visual scenes, and even execute symbolic instructions as first-person actors in partially-observable worlds. To achieve this so-called grounded language learning, models must overcome certain well-studied learning challenges that are also fundamental to infants learning their first words. While it is notable that models with no meaningful prior knowledge overcome these learning obstacles, AI researchers and practitioners currently lack a clear understanding of exactly how they do so. Here we address this question as a way of achieving a clearer general understanding of grounded language learning, both to inform future research and to improve confidence in model predictions. For maximum control and generality, we focus on a simple neural network-based language learning agent trained via policy-gradient methods to interpret synthetic linguistic instructions in a simulated 3D world. We apply experimental paradigms from developmental psychology to this agent, exploring the conditions under which established human biases and learning effects emerge. We further propose a novel way to visualise and analyse semantic representation in grounded language learning agents that yields a plausible computational account of the observed effects.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In LLM Agents & Tool Use

Citations in the corpus

Environmental drivers of systematicity and generalization in a situated agent (2019, methodology): “We replicate the demonstration of Hill et al. (2017) in the 3D DeepMind-Lab environment (Beattie et al., 2016 As in the original experiments, we split the available colors and shapes in the environment into sets S = s ∪ ˆ s and C = c ∪ ˆ c .”
IQA: Visual Question Answering in Interactive Environments (2017, methodology): “The learned baseline that we compare to (A3C), is based on a common architecture for reinforcement learning used in past works including for visual semantic planning [73] and task oriented language grounding [8, 19].”
Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View (2020, background): “Until recently, the most commonly studied domains were map-based (Thompson et al., 1993) or game-like (Macmahon et al., 2006; Misra et al., 2017, 2018; Hermann et al., 2017; Hill et al., 2017).”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, background): “Embodied navigation also aids AI agents to develop behavior such as instruction following [38,4,82,95,3], city navigation [18,64,63,94], question answering [21,22,34,97,24], and active visual recognition [105,104].”
A Benchmark for Systematic Generalization in Grounded Language Understanding (2020, background): “Recent work has recognized the advantages of compositional generalization for robustness and sample efficiency, and responded by building synthetic environments to evaluate models on aspects of this skill [7, 21, 25, 30, 4, 16, 17, 8, 19, 3].”
Emergent Systematic Generalization in a Situated Agent (2019, background): “…have shown that neural-network-based agents can correctly interpret instructions referring to both the color and shape of objects ( ﬁnd a red ball ) zero-shot when they have never encountered that particular combination during training (Chaplot et al., 2018; Hill et al., 2017; Yu et al., 2018).”
HIGhER: Improving instruction following with Hindsight Generation for Experience Replay (2020, background): “To the best of our knowledge, the number of different objects and its diversity is greater than concurrent works ([10] used 55 train instructions and 15 test instructions and [22] has a total of 40 different objects).”
Two Body Problem: Collaborative Visual Task Completion (2019, background): “[39] propose to guide a virtual agent via language commands.”
Learning to Navigate in Cities Without a Map (2018, methodology): “Others have incorporated goal instructions for the agent, using simple text descriptions (Hill et al., 2017; Hermann et al., 2017; Chaplot et al., 2017); in our case, the goal is given to the agent in terms of distances to local landmarks.”
Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents (2018, background): “Deep RL agents have even been demonstrated to learn to respond correctly to natural language commands like "go find the blue balloon" (Hermann et al., 2017; Hill et al., 2017; Chaplot et al., 2017).”