TextWorld: A Learning Environment for Text-based Games

By Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, B. Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew J. Hausknecht, Layla El Asri, Mahmoud Adada and 2 more. 2018

482 citations recorded by the source corpus.

We introduce TextWorld, a sandbox learning environment for the training and evaluation of RL agents on text-based games. TextWorld is a Python library that handles interactive play-through of text games, as well as backend functions like state tracking and reward assignment. It comes with a curated list of games whose features and challenges we have analyzed. More significantly, it enables users to handcraft or automatically generate new games. Its generative mechanisms give precise control over the difficulty, scope, and language of constructed games, and can be used to relax challenges inherent to commercial text games like partial observability and sparse rewards. By generating sets of varied but similar games, TextWorld can also be used to study generalization and transfer learning. We cast text-based games in the Reinforcement Learning formalism, use our framework to develop a set of benchmark games, and evaluate several baseline agents on this set and the curated list.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (2021, background): “A prewritten text-based game [4] can be used to test various dimensions of world-modelling and agency, such as problem solving, information gathering, and social intelligence (including deception).”
Pre-Trained Language Models for Interactive Decision-Making (2022, neutral): “A large body of recent work has used LM pre-training to scaffold interactive decision-making in text-based games (Yuan et al., 2018; Narasimhan et al., 2015; Ammanabrolu & Riedl, 2018; Côté et al., 2018; Yao et al., 2020; 2021) in which both observations and actions are encoded as text.”
Language Models are Few-Shot Butlers (2021, methodology): “However, the application of modern language models to action generation in text-based environments (Côté et al., 2018; Hausknecht et al., 2019; Shridhar et al., 2021) remains largely unexplored (Yao et al., 2020).”
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning (2020, methodology): “We build ALFWorld by extending two prior works: TextWorld (Côté et al., 2018) - an engine for interactive text-based games, and ALFRED (Shridhar et al., 2020) - a large scale dataset for vision-language instruction following in embodied environments.”
Neural Approaches to Conversational AI (2018, positive): “Recent advances for solving text games, such as handling natural-language actions (Narasimhan et al., 2015; He et al., 2016; Côté et al., 2018) and interpretable policies (Chen et al.”
Learning to Speak and Act in a Fantasy Text Adventure Game (2019, background): “(2015) and TextWorld (Côté et al., 2018), but these do not have human dialogue within the game.”
Interactive Fiction Games: A Colossal Adventure (2019, methodology): “The TextWorld framework (Côté et al. 2018) supports procedural generation of parser-based IF games, allowing complexity and content of the generated games to be scaled to the difficulty needed for research.”
Implicit Representations of Meaning in Neural Language Models (2021, background): “Data: TextWorld TextWorld (Côté et al., 2018) is a platform for generating synthetic worlds for text-based games, used to test RL agents.”
Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning (2019, background): “There are efforts in building complex interactive text-based worlds (Côté et al., 2018; Urbanek et al., 2019) but the lack of a graphical component makes them not suitable for visually grounded learning.”
Graph Constrained Reinforcement Learning for Natural Language Action Spaces (2020, background): “Cˆot´e et al. (2018) introduce TextWorld, a framework for procedurally generating parser-based games, allowing a user to control the difﬁculty of a generated game.”
Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning (2018, background): “TBG provide a testbed for research at the intersection of RL and NLP, presenting a broad spectrum of challenges for learning algorithms (Côté et al., 2018) 1 .”
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds (2020, background): “(Côté et al., 2018) introduce TextWorld, a framework for procedurally generating text-based games via grammars, and (Yuan et al., 2018; Yin and May, 2019; Adolphs and Hofmann, 2019; Adhikari et al., 2020) build agents that operate in this environment—focusing on aspects such as efficient…”
Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions (2023, background): “Meanwhile, TextWorld serves as a dynamic learning playground for training and evaluating reinforcement learning agents on text-based games.”
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task (2022, background): “Grammar engineering tools (Weston et al., 2015; Hermann et al., 2017; Cˆot´e et al., 2018) could help define a synthetic data generation process that maps world representations onto natural language sentences, providing a similarly controllable setting like Othello while closing the distance to…”
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning (2018, methodology): “…of S, T, A, Ω , O, R, γ (cid:105) representing the set of environment states, conditional transition probabilities between states, words used to compose text commands, observations, observation conditional probabilities, reward function, and the discount factor respectively (Cˆot´e et al., 2018).”
Reflexion: an autonomous agent with dynamic memory and self-reflection (2023, background): “AlfWorld is a suite of text-based environments that challenge an agent to solve multi-step tasks in a variety of interactive environments based on TextWorld (Côté et al., 2019).”