Guiding Reinforcement Learning Exploration Using Natural Language
By Brent Harrison, Upol Ehsan, Mark O. Riedl. 2017
29 citations recorded by the source corpus.
In this work we present a technique for using natural language to help reinforcement learning generalize to unseen environments using neural machine translation techniques. These techniques are then integrated into policy shaping to make it more effective at learning in unseen environments. We evaluate this technique using the popular arcade game, Frogger, and show that our modified policy shaping algorithm improves over a Q-learning agent as well as a baseline version of policy shaping.
Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Percy Liang
- Tim Rocktaschel
- Tom Michael Mitchell
- Hannaneh Hajishirzi
- Richard Socher
- Yejin Choi
In Natural Language Processing
- Percy Liang
- Mohit Bansal
- Luke S. Zettlemoyer
- Tom Michael Mitchell
- Tim Rocktaschel
- Mirella Lapata
- Edward Grefenstette
- Igor Labutov
In Reinforcement Learning
- Tim Rocktaschel
- Edward Grefenstette
- Scott Niekum
- Raymond J. Mooney
- Prasoon Goyal
- Victor Zhong
- Noah D. Goodman
- Igor Labutov
Citations in the corpus
- Learning with Latent Language (2017, neutral): “…instructions available at concept-learning time (rather than language-learning time) have been used to provide side information to reinforcement learners about high-level strategy (Branavan et al., 2011), environment dynamics (Narasimhan et al., 2017) and exploration (Harrison et al., 2017).”
- Improving Intrinsic Exploration with Language Abstractions (2022, methodology): “Meanwhile, reward shaping and inverse RL methods [3, 5, 21–24, 31, 44, 51, 53] reward an agent for actions associated with linguistic descriptions, but again are primarily designed for instruction following, where an extrinsic goal is available to help shape intermediate rewards.”
- Learning to Learn Semantic Parsers from Natural Language Supervision (2019, background): “, 2015) and guiding policy learning using high-level human advice (Kuhlmann et al., 2004; Squire et al., 2015; Harrison et al., 2017).”
- PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards (2020, background): “[22] learn a distribution of states and actions given a natural language command, which is used for policy shaping.”