Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent

By Zhilin Yang, Saizheng Zhang, Jack Urbanek, Will Feng, Alexander H. Miller, Arthur Szlam, Douwe Kiela, Jason Aaron Edward Weston. 2017

25 citations recorded by the source corpus.

Contrary to most natural language processing research, which makes use of static datasets, humans learn language interactively, grounded in an environment. In this work we propose an interactive learning procedure called Mechanical Turker Descent (MTD) and use it to train agents to execute natural language commands grounded in a fantasy text adventure game. In MTD, Turkers compete to train better agents in the short term, and collaborate by sharing their agents' skills in the long term. This results in a gamified, engaging experience for the Turkers and a better quality teaching signal for the agents compared to static datasets, as the Turkers naturally adapt the training data to the agent's abilities.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, methodology): “Mechanical Turker Descent proposes a gamiﬁed environment for the collaborative training of grounded language learning agents over multiple rounds (Yang et al., 2017).”
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, background): “There has also been some work on predicting conversation quality from signals such as sentiment or questions (Ghandeharioun et al., 2019) or using other forms of feedback from the conversation (Li et al., 2017a,b; Weston, 2016; Yang et al., 2017), but results are still preliminary.”
Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack (2019, methodology): “Our work is also closely connected to the “Mechanical Turker Descent” algorithm detailed in (Yang et al., 2018) where language to action pairs were collected from crowdworkers by incen-tivizing them with a game-with-a-purpose technique: a crowdworker receives a bonus if their contribution results…”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, neutral): “Examples for this include 368 [39, 40, 41, 12, 42, 34, 43, 38, 44], with recent works featuring re-training during dataset collection 369 (Beat-the-AI [22], StrategyQA [23] and Dynabench [21]).”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, background): “There is a rich body of recent work showing the beneﬁts of dynamic adversarial data collection in model evaluation (Yang et al., 2017; Dua et al., 2019; Dinan et al., 2019; Nie et al., 2020; Bar-tolo et al., 2020; Kiela et al., 2021; Wallace et al., 2021), although the approach has been challenged…”
Learning to Speak and Act in a Fantasy Text Adventure Game (2019, background): “Yang et al. (2017) and Bordes et al. (2010) proposed small world setups for instruction following or labeling, but these are much more restricted than the large multi-player text adventure game environment with rich dialogue that we propose here.”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, methodology): “We also experimented with BERTScore (Zhang et al., 2019) and found similar trends as BLEU score. its length in words, its Flesch-Kincaid readabil- 479 ity (Flesch, 1948), and its syntactic complexity 480 using Yngve scores (Yngve, 1960; Roark et al., 481 2007).”
Deploying Lifelong Open-Domain Dialogue Learning (2020, background): “cases or the ones with the most learning signal, such as in the work on adversarial collection (Yang et al., 2017; Nie et al., 2019).”
Toward Automated Quest Generation in Text-Adventure Games (2019, background): “…there has been much work recently on text-adventure gameplay (Bordes et al., 2010; He et al., 2016; Narasimhan et al., 2015; Fulda et al., 2017; Yang et al., 2018; Haroush et al., 2018; Cˆot´e et al., 2018; Tao et al., 2018; Ammanabrolu and Riedl, 2019a; Hausknecht et al., 2019a; Am-manabrolu…”
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study (2021, background): “Yang et al. (2018b) ask workers to generate challenging datasets working ﬁrst as adversaries and later as collaborators.”
Why Build an Assistant in Minecraft? (2019, background): “Starcraft [80, 86], Atari games [11], Go [74, 84], TORCS [95], Doom [41] and Text adventure games [20, 96].”