Reading between the Lines: Learning to Map High-Level Instructions to Commands

By S. Branavan, Luke S. Zettlemoyer, Regina Barzilay. 2010

134 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Semantic Parsing on Freebase from Question-Answer Pairs (2013, background): “, 2013), playing computer games (Branavan et al., 2010; Branavan et al., 2011), following navigational instructions (Chen, 2012; Artzi and Zettlemoyer, 2013), and interacting in the real world via perception (Matuszek et al.”
Language Understanding for Text-based Games using Deep Reinforcement Learning (2015, methodology): “Example applications include interpreting help documentation for software (Branavan et al., 2010), navigating with directions (Vogel and Jurafsky, 2010; Kollar et al.”
Learning to Automatically Solve Algebra Word Problems (2014, background): “Reinforcement learning can be used to learn to read instructions and perform actions in an external world (Brana-van et al., 2009; Branavan et al., 2010; Vogel and Jurafsky, 2010).”
Learning Dependency-Based Compositional Semantics (2011, background): “as Montague’s (1973) quantifying in, Cooper storage (Cooper 1975), and Carpenter’s (1998) scoping constructor handle scope divergence during semantic interpretation.”
Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions (2013, methodology): “This style of algorithm, especially when using the weaker end state validation, is closely related to reinforcement learning approaches (Branavan et al., 2009, 2010).”
Learning Language Games through Interaction (2016, background): “Examples include playing games (Branavan et al., 2009, 2010; Reckman et al., 2010) interacting with robotics (Tellex et al., 2011, 2014), and following instructions (Vogel and Jurafsky, 2010; Chen and Mooney, 2011; Artzi and Zettlemoyer, 2013) Semantic parsing utterances to logical forms, which we…”
A Survey of Reinforcement Learning Informed by Natural Language (2019, background): “The line of work involving sequences of instructions has strong ties to Hierarchical RL [Barto and Mahadevan, 2003], with individual sentences or clauses from instructions corresponding to subtasks [ Branavan et al. , 2010 ] .”
Learning to Win by Reading Manuals in a Monte-Carlo Framework (2011, methodology): “Within this line of work, we are most closely related to reinforcement learning approaches that learn language by proactively interacting with an external environment (Branavan et al., 2009; Branavan et al., 2010; Vogel and Jurafsky, 2010).”
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning (2017, background): “…and Mooney, 2011; Kim and Mooney, 2012, 2013; Artzi and Zettlemoyer, 2013; Artzi et al., 2014a,b; Misra et al., 2015, 2016), alignment models (Andreas and Klein, 2015), reinforcement learning (Branavan et al., 2009, 2010; Vogel and Jurafsky, 2010), and neural network models (Mei et al., 2016).”
An Overview of Natural Language State Representation for Reinforcement Learning (2020, methodology): “In this task, states were ﬁrst designed as tuples of world and linguistic features: words and documents (Branavan et al., 2009; 2010) or cardinal directions and utterances (Vogel & Jurafsky, 2010).”
Interactively shaping robot behaviour with unlabeled human instructions (2019, methodology): “One method for interpreting instructions consists in providing the robot with a description of the task; then making it interact with its environment in order to interpret the instructions using either demonstrations [27] or a predeﬁned reward function [4, 5, 49].”
Grounding English Commands to Reward Functions (2015, methodology): “These approaches have been applied to various tasks, the ones closest to ours being interpreting verbal commands in the context of navigational instructions [23, 6, 10], robot manipulation [9, 22, 11], and puzzle solving and software control [3].”
Lexical Generalization in CCG Grammar Induction for Semantic Parsing (2011, background): “Other work has learnt semantic analyses from text in the context of interactions in computational environments (Branavan et al. (2010), Vogel and Juraf-sky (2010)); text grounded in partial observations of a world state (Liang et al., 2009); and from raw text alone (Poon and Domingos, 2009, 2010).”
LanCon-Learn: Learning with Language to Enable Generalization in Multi-Task Manipulation (2021, methodology): “Rather than use a conventional neural network with language goals concatenated to state-data as input to the network [56], we leverage a soft-modular architecture based upon empirical experimentation discussed in Sec.”
Bringing Machine Learning and Compositional Semantics Together (2015, background): “…et al. 2013), interpreting natural language for performing programming tasks (Kushman & Barzilay 2013; Lei et al. 2013), playing computer games (Branavan et al. 2010, 2011) and interacting in the real world via perception (Matuszek et al. 2012a; Tellex et al. 2011; Krishnamurthy & Kollar 2013).”
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning (2018, background): “Mapping natural language instructions to actions has been studied extensively, both using physical robots [7, 16, 25, 31, 34, 49, 50] and virtual agents [2, 4, 29, 30, 32].”
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight (2019, background): “Natural language instruction following has been extensively studied using hand-engineered symbolic intermediate representations of world state or instruction semantics with physical robots [1, 2, 3, 4, 17, 5, 18, 6, 19] and simulated agents [20, 21, 22, 23, 24, 25].”