Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents
By Shunyu Yao, Karthik Narasimhan, Matthew J. Hausknecht. 2021
28 citations recorded by the source corpus.
Text-based games simulate worlds and interact with players using natural language. Recent work has used them as a testbed for autonomous language-understanding agents, with the motivation being that understanding the meanings of words or semantics is a key component of how humans understand, reason, and act in these worlds. However, it remains unclear to what extent artificial agents utilize semantic understanding of the text. To this end, we perform experiments to systematically reduce the amount of semantic information available to a learning agent. Surprisingly, we find that an agent is capable of achieving high scores even in the complete absence of language semantics, indicating that the currently popular experimental setup and models may be poorly designed to understand and leverage game texts. To remedy this deficiency, we propose an inverse dynamics decoder to regularize the representation space and encourage exploration, which shows improved performance on several games including Zork I. We discuss the implications of our findings for designing future agents with stronger semantic understanding.
Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shunyu Yao
- Karthik Narasimhan
- John Yang
- Howard Chen
- Sergey Levine
- Jacob Andreas
- Mohit Bansal
- Matthew J. Hausknecht
In Natural Language Processing
In Reinforcement Learning
- Shunyu Yao
- John Yang
- Karthik Narasimhan
- Howard Chen
- Matthew J. Hausknecht
- Prithviraj Ammanabrolu
- Sergey Levine
Citations in the corpus
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, background): “These include: 1) generation of good search queries [22, 59] and reformulation [37, 51], 2) strategic exploration for navigating through the website [55, 56, 29], 3) robust language understanding for textual state and action spaces [3, 7, 17, 44], and 4) long-term”
- Multi-Stage Episodic Control for Strategic Exploration in Text Games (2022, unknown): “INV-DY is a Q-based policy π inv − dy similar to DRRN (He et al., 2016a), optimized with the standard TD loss (see Background of Section 3).”