ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

By Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, Matthew J. Hausknecht. 2020

1,059 citations recorded by the source corpus.

Given a simple request (e.g., Put a washed apple in the kitchen fridge), humans can reason in purely abstract terms by imagining action sequences and scoring their likelihood of success, prototypicality, and efficiency, all without moving a muscle. Once we see the kitchen in question, we can update our abstract plans to fit the scene. Embodied agents require the same abilities, but existing work does not yet provide the infrastructure necessary for both reasoning abstractly and executing concretely. We address this limitation by introducing ALFWorld, a simulator that enables agents to learn abstract, text-based policies in TextWorld (Cote et al., 2018) and then execute goals from the ALFRED benchmark (Shridhar et al., 2020) in a rich visual environment. ALFWorld enables the creation of a new BUTLER agent whose abstract knowledge, learned in TextWorld, corresponds directly to concrete, visually grounded actions. In turn, as we demonstrate empirically, this fosters better agent generalization than training only in the visually grounded environment. BUTLER's simple, modular design factors the problem to allow researchers to focus on models for improving every piece of the pipeline (language understanding, planning, navigation, visual scene understanding, and so forth).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (2022, background): “Recently, many efforts have been put into creating more realistic environments with the goal to further advances in this area [38, 47, 48, 22, 44, 1].”
Language Models are Few-Shot Butlers (2021, methodology): “We demonstrate the effectiveness of our approach in the recently introduced ALFWorld environment (Shridhar et al., 2021) 1 , which was designed with an extensive set of tasks and expert demonstrations.”
Skill Induction and Planning with Latent Language (2021, background): “As in our work, representations of language learned from large text corpora facilitate grounded language learning (Shridhar et al., 2021), and interaction with the environment can in turn improve the accuracy of language generation (Zellers et al., 2021); future work might extend our framework for…”
A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, neutral): “Essentially, parametric memory boasts a higher information density, expressing semantics through continuous real-number vectors in a latent space, whereas textual memory employs a combination of tokens in a discrete space for semantic expression.”
Executable Code Actions Elicit Better LLM Agents (2024, background): “Your solution should be enclosed using "<solution>" tag, for example: The answer is < solution> A </solution>.”
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring (2021, methodology): “For example, on the ALFRED benchmark for task learning (Shridhar et al., 2020), the state-of-the-art system only achieves less than 10% task success rate in an unseen environment (Singh et al., 2020), compared to the human performance of over 90%.”
Semantic Exploration from Language Abstractions and Pretrained Representations (2022, methodology): “Language for exploration Some recent works have used language to guide agent learning, by either using language subgoals for exploration/planning or providing task-specific reward shaping [47, 33, 13, 19].”
WebArena: A Realistic Web Environment for Building Autonomous Agents (2023, background): “As a result, the functionality of many environments is a limited version of their real-world counterparts, leading to a lack of task diversity within the environment [27, 1, 10, 23, 29, 30, 37].”
Few-shot Language Coordination by Modeling Theory of Mind (2021, methodology): “We use the Alfworld (Shridhar et al., 2021) platform, which creates a natural language command action space upon the Alfred environment for vision-language navigation (Shridhar et al., 2020).”
FILM: Following Instructions in Language with Modular Methods (2021, methodology): “Similarly, separate instance segmentation models for small and large objects were ﬁne-tuned, starting from their respective parameters released by Shridhar et al. (2021), with learning rate 1e-3 and the SGD optimizer (momentum 0.9, weight decay 5e-4).”
ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (2024, result): “It also demonstrates comparable performance to GPT-4 and Claude 3 Opus in Chinese math and logic reasoning capabilities though it lags behind GPT-4 Turbo.”
Reflexion: language agents with verbal reinforcement learning (2023, positive): “Specifically, we challenge an agent to perform search-based question answering on HotPotQA [30], multi-step tasks in common household environments in AlfWorld [24], and code writing tasks in competition-like environments with interpreters and compilers in HumanEval [5], MBPP [2], and LeetcodeHard, a”
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models (2023, background): “…(despite being not universally applicable in all possible environments), including programming (HumanEval (Chen et al., 2021)), web search (WebShop (Yao et al., 2022)), text-based manipulation tasks (Alfworld (Shridhar et al., 2020)), and LMs with tool use (ToolBench (Qin et al., 2024)).”
Tool Learning with Foundation Models (2023, positive): “ALFWorld (Shridhar et al., 2021) is an aligned text and embodied environment game, where agents need to interact with objects (e.g., fridge, microwave, drawer, etc.) in a house to complete a task 9https://www.alphavantage.co/documentation 10https://pypi.org/project/python-pptx…”
Episodic Transformer for Vision-and-Language Navigation (2021, background): “…[39, 40], back-tracking [30], speaker-driven route selection [20], cross-modal matching [29, 68], back translation [60], pretraining on subtasks [74], and text-based pretraining [14,57] are proposed to improve the performance and generalization of neural agents in seen and unseen environments.”
Improving Intrinsic Exploration with Language Abstractions (2022, background): “Many modern RL environments are pre-equipped with language, including NetHack/MiniHack [28, 41], text-based games [15, 45, 52], and in fact most video games in general.”
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences (2020, background): “…category, (Frazier et al., 2020) is notable for proposing the use of linguistic priors to guide the behaviour of intelligent agents as a viable alternative to imitation and preference learning, which has been recently attempted for procedural, objectoriented reasoning by (Shridhar et al., 2020).”