How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds

By Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktaschel, Jason Aaron Edward Weston. 2020

45 citations recorded by the source corpus.

We seek to create agents that both act and communicate with other agents in pursuit of a goal. Towards this end, we extend LIGHT (Urbanek et al. 2019)—a large-scale crowd-sourced fantasy text-game—with a dataset of quests. These contain natural language motivations paired with in-game goals and human demonstrations; completing a quest might require dialogue or actions (or both). We introduce a reinforcement learning system that (1) incorporates large-scale language modeling-based and commonsense reasoning-based pre-training to imbue the agent with relevant priors; and (2) leverages a factorized action space of action commands and dialogue, balancing between the two. We conduct zero-shot evaluations using held-out human expert demonstrations, showing that our agents are able to act consistently and talk naturally with respect to their motivations.

Subfields: LLM Agents & Tool Use, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In LLM Agents & Tool Use

1. Yejin Choi
2. Prithviraj Ammanabrolu
3. Timothy P. Lillicrap
4. Hannaneh Hajishirzi
5. Mark O. Riedl

Citations in the corpus

• Quark: Controllable Text Generation with Reinforced Unlearning (2022, methodology): “Finally, prior work has used RL techniques to generate language grounded in text-based narrative games [23, 4, 3].”
• A data-driven approach for learning to control computers (2022, background): “If we can develop agents that can assist with even a tiny fraction of these tasks, we can hope to enter a virtuous cycle of agent assistance, followed by human feedback on failures, and hence to agent improvement and new capabilities.”
• Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering (2022, background): “…et al., 2021), summarization (Stiennon et al., 2020; Paulus et al., 2018), machine translation (Shen et al., 2016; Wu et al., 2016), grounded text generation (Ammanabrolu et al., 2021, 2022), controlled text generation (Lu et al., 2022), and prompt generation (Guo et al., 2021; Deng et al., 2022).”
• Aligning to Social Norms and Values in Interactive Narratives (2022, background): “, 2020); (2) scaling RL algorithms to handle combinatorial natural language state-action spaces (Zahavy et al., 2018; Yao et al., 2020; Ammanabrolu et al., 2021; Jang et al., 2021); and (3) giving agents commonsense priors to better reason about the world (Murugesan et al.”
• Situated Dialogue Learning through Procedural Environment Generation (2021, methodology): “The study ablates across how the curriculums are generated, for the best two model types found in Ammanabrolu et al. (2021).”