Learning Language Games through Interaction

By Sida I. Wang, Percy Liang, Christopher D. Manning. 2016

197 citations recorded by the source corpus.

We introduce a new language learning setting relevant to building adaptive natural language interfaces. It is inspired by Wittgenstein's language games: a human wishes to accomplish some task (e.g., achieving a certain configuration of blocks), but can only communicate with a computer, who performs the actual actions (e.g., removing all red blocks). The computer initially knows nothing about language and therefore must learn it from scratch through interaction, while the human adapts to the computer's capabilities. We created a game in a blocks world and collected interactions from 100 people playing it. First, we analyze the humans' strategies, showing that using compositionality and avoiding synonyms correlates positively with task performance. Second, we compare computer strategies, showing how to quickly learn a semantic parsing model from scratch, and that modeling pragmatics further accelerates learning for successful players.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Deep Reinforcement Learning from Human Preferences (2017, background): “…reinforcement learning from human ratings or rankings, including Akrour et al. (2011), Pilarski et al. (2011), Akrour et al. (2012), Wilson et al. (2012), Sugiyama et al. (2012), Wirth and Fürnkranz (2013), Daniel et al. (2015), El Asri et al. (2016), Wang et al. (2016), and Wirth et al. (2016).”
Grounded Language Learning in a Simulated 3D World (2017, background): “Other related work focuses on language grounding from the perspective of human-machine communication (Thomason et al., 2015; Wang et al., 2016; Arumugam et al., 2017).”
Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “While system debugging through dialogue has been explored in many contexts (Hixon et al., 2015; Wang et al., 2016; Davis, 1977), our novel contribution is a dialogue about the model’s understanding of the user’s intent.”
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, background): “Ideas such as combining robot planning and language [85], using language models as a pre-training mechanism for policies [44] and many other ways of combining language and interaction [21, 22, 38, 86] are exciting avenues for future research.”
Embodied Question Answering (2017, background): “The form and structure of these goal speciﬁca-tions range from declarative programs [26], to simple tem-plated commands [27,28], to free-form natural language instructions [29,30].”
Multi-Agent Cooperation and the Emergence of (Natural) Language (2016, background): “Others (e.g., the SHRLDU program of Winograd 1971 or the game in Wang et al. 2016) propose building a communicating AI by putting humans in the loop from the very beginning.”
Learning to Speak and Act in a Fantasy Text Adventure Game (2019, methodology): “…are instruction-following in the Neverwinter Nights game (Fleischman and Roy, 2005), dialogue about soccer videogames (Pa-sunuru and Bansal, 2018), placing blocks appropriately given a ﬁnal plan (Wang et al., 2016) and a more open ended building task using a grid of voxels (Wang et al., 2017).”
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent (2017, neutral): “Some works that do utilize real natural language and interaction include Wang et al. (2016), where language is learnt to solve block puzzles, and Wang et al. (2017) where language is learnt to draw voxel images, which are both quite different to our case of studying text adventure games.”
A Survey of Reinforcement Learning Informed by Natural Language (2019, background): “Some work in this area focuses on simple object manipulation tasks [Wang et al. , 2016; Bahdanau et al. , 2019], while other work focuses on 2D or 3D navigation tasks where the goal is to reach a speciﬁc entity.”
BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning (2018, background): “Finally, Wang et al. (2016) presented a system that interactively learned language from a human.”
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text (2020, methodology): “…pipeline-based approaches use learning algorithms to map language to a program that can then interface with a planner [Chen and Mooney, 2011, Matuszek et al., 2013, Wang et al., 2016], and/or a controller, both of which may have priviledged information about how the world connects to the program.”
Learning a Neural Semantic Parser from User Feedback (2017, background): “…paraphrases (often gathered through crowdsourcing (Wang et al., 2015)), data augmentation (e.g. based on manually engineered semantic grammars (Jia and Liang, 2016)) and learning through direct interaction with users (e.g., where a single user teaches the model new concepts (Wang et al., 2016)).”
Do explanations make VQA models more predictable to a human? (2018, background): “Wang et al. (2016) and Pelikan and Broth (2016) observe humans’ strategies while adapting to the limited capabilities of an AI in interactive language games.”
Naturalizing a Programming Language via Interactive Learning (2017, methodology): “Compared to interactive language learning with weak denotational supervision (Wang et al., 2016), definitions are critical for learning complex ac-”
Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game (2018, background): “This makes our game distinct from other seemingly relevant games, in which the agent cannot speak (Wang et al., 2016) or “speaks” by selecting a candidate from a provided set (He et al., 2016; Weston, 2016; Li et al., 2017) rather than generating sentences by itself, or games mainly focus on slow…”
Learning Interpretable Spatial Operations in a Rich 3D Blocks World (2017, methodology): “Simultaneously, work within NLP (Bisk, Marcu, and Wong 2016; Wang, Liang, and Manning 2016) and Robotics (Li et al. 2016) returned to the question of action taking and scene understanding in SHRDLU style worlds.”