Language is Power: Representing States Using Natural Language in Reinforcement Learning.
By Erez Schwartz, Guy Tennenholtz, Chen Tessler, Shie Mannor. 2019
13 citations recorded by the source corpus.
Recent advances in reinforcement learning have shown its potential to tackle complex real-life tasks. However, as the dimensionality of the task increases, reinforcement learning methods tend to struggle. To overcome this, we explore methods for representing the semantic information embedded in the state. While previous methods focused on information in its raw form (e.g., raw visual input), we propose to represent the state using natural language. Language can represent complex scenarios and concepts, making it a favorable candidate for representation. Empirical evidence, within the domain of ViZDoom, suggests that natural language based agents are more robust, converge faster and perform better than vision based agents, showing the benefit of using natural language representations for reinforcement learning.
Subfields: Artificial Intelligence, Control and Systems Engineering, Natural Language Processing, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Luke S. Zettlemoyer
- Jianfeng Gao
- Percy Liang
- Hannaneh Hajishirzi
- Caiming Xiong
- Sergey Levine
In Control and Systems Engineering
- Mohit Shridhar
- Nicholas A. Roy
- Edward Grefenstette
- Tim Rocktaschel
- Matthew J. Hausknecht
- Yonatan Bisk
- Neil C. Rabinowitz
- Roberta Raileanu
In Natural Language Processing
- Yejin Choi
- Mohit Bansal
- Tim Rocktaschel
- Yonatan Bisk
- Percy Liang
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Richard Socher
In Reinforcement Learning
- Marc-Alexandre Côté
- Xingdi Yuan
- Mohit Shridhar
- Yonatan Bisk
- Tim Rocktaschel
- Matthew J. Hausknecht
- Adam Trischler
- Felix Hill
Citations in the corpus
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning (2020, background): “te instructions, goals, or queries to the agent, but not as a fully interactive modality. Language for State and Action Representation: Others have used language for more than just goal-specification. Schwartz et al. (2019) use language as a state representation for VizDoom. Hu et al. (2019) use a na”
- Semantic Exploration from Language Abstractions and Pretrained Representations (2022, methodology): “[45] use a custom semantic parser for VizDoom and show that representing states with language, rather than vision, leads to faster learning by simplifying policy inputs.”
- Improving Intrinsic Exploration with Language Abstractions (2022, methodology): “While language-guided exploration methods exist in the literature [3, 5, 12, 13, 21–24, 31, 44, 51, 53], we make two key contributions over prior work.”