Representation Learning for Grounded Spatial Reasoning

By Michael Janner, Karthik Narasimhan, Regina Barzilay. 2017

74 citations recorded by the source corpus.

The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards. The proposed model learns a representation of the world steered by instruction text. This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions. We train our model with reinforcement learning using a variant of generalized value iteration. The model outperforms state-of-the-art approaches on several metrics, yielding a 45% reduction in goal localization error.

Subfields: Artificial Intelligence, Computer Networks and Communications, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Learning with Latent Language (2017, methodology): “The environment implementation and linguistic annotations are in this case adapted from a natural language navigation dataset originally introduced by Janner et al. (2017).”
A Survey of Reinforcement Learning Informed by Natural Language (2019, methodology): “Human-generated natu-ral language instructions are used in [ MacMahon et al. , 2006; Bisk et al. , 2016; Misra et al. , 2017; Janner et al. , 2018; Chen et al. , 2018; Anderson et al. , 2018; Goyal et al. , 2019; Wang et al. , 2019 ] .”
CHALET: Cornell House Agent Learning Environment (2018, unknown): “Simulators have also been used to evaluate natural language instruction following [14, 3, 13, 9, 8] and question answering [7, 6, 10, 19].”
Vision-Based Navigation With Language-Based Assistance via Imitation Learning With Indirect Intervention (2018, background): “Simple simulators as Puddle World Navigation [31] and Rich 3D Blocks World [5, 4] have facilitated understanding of fundamental representational and grounding issues by allowing for fast experimentation in easily-managed environments.”
An Overview of Natural Language State Representation for Reinforcement Learning (2020, methodology): “Narasimhan et al. (2018) used a factorized state representation, concate-nating an object embedding with its textual speciﬁcation embedding (LSTM or bag-of-words).”
Guiding Policies with Language via Meta-Learning (2018, background): “Considerable research has also focused on building autonomous agents that can follow instructions provided via language (Janner et al. (2018); Andreas & Klein (2015); Fried et al. (2018); Tellex et al. (2011)).”
Vygotskian Autotelic Artificial Intelligence: Language and Culture Internalization for Human-Like AI (2022, background): “Linguistic autotelic agents can reach and make sense of abstract goals like “ go to any circle ,” 87 relational goals “ sort objects by size ,” 88 “ put the cylinder in the drawer ,” 89 sequential goals “ open the yellow door after opening a red door ,” 90 or even learning goals “ is the ghargh…”
Mapping natural language commands to web elements (2018, background): “…2017), database queries (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2007; Berant et al., 2013; Yih et al., 2015), navigation (Artzi and Zettlemoyer, 2013; Janner et al., 2018), and object manipulation (Tellex et al., 2011; Andreas and Klein, 2015; Guu et al., 2017; Fried et al., 2018).”
Learning to Parse Natural Language to Grounded Reward Functions with Weak Supervision (2018, background): “Another series of approaches map natural language directly to policies in an MDP, learned from demonstrations provided at training time [19, 14].”
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation (2021, background): “While these approaches have been largely studied in simulated spatial games [19, 21, 22, 23] or in object-directed visual navigation in simulated robots [24, 25, 26, 27, 28, 29, 23] some of which include high-level object interaction [30], in this work we focus on the domain of learning control for ”
Learning to Follow Language Instructions with Adversarial Reward Induction (2018, background): “[14, 20, 11, 6, 9, 31] consider instruction-following in 2D or 3D environments and reward the agent for arriving at the correct location or object.”
InfoBot: Transfer and Exploration via the Information Bottleneck (2019, methodology): “For reproducibility purposes of our experiments, we will further release the code on github that will be available on Hyperparameter Search for Grid World Experiments : For the proposed method, we only varied the weight of KL loss.”
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction (2018, methodology): “Finally, on goal prediction (Table 5), our approach outperforms the method of Janner et al. (2018).”
Entity Abstraction in Visual Model-Based Reinforcement Learning (2019, background): “Unsupervised grounding of abstract entity variables in concrete objects: Prior works that model entities and their interactions often pre-specify the identity of the entities [24–30], provide additional supervision [31–34], or provide additional specification such as segmentations [35], crops [36], ”