Deep Transfer in Reinforcement Learning by Language Grounding

By Karthik Narasimhan, Regina Barzilay, Tommi S. Jaakkola. 2017

25 citations recorded by the source corpus.

In this paper, we explore the utilization of natural language to drive transfer for reinforcement learning (RL). Despite the wide-spread application of deep RL techniques, learning generalized policy representations that work across domains remains a challenging problem. We demonstrate that textual descriptions of environments provide a compact intermediate channel to facilitate effective policy transfer. We employ a model-based RL approach consisting of a differentiable planning module, a model-free component and a factorized representation to effectively utilize entity descriptions. Our model outperforms prior work on both transfer and multi-task scenarios in a variety of different environments.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Learning with Latent Language (2017, neutral): “…instructions available at concept-learning time (rather than language-learning time) have been used to provide side information to reinforcement learners about high-level strategy (Branavan et al., 2011), environment dynamics (Narasimhan et al., 2017) and exploration (Harrison et al., 2017).”
Boosting Search Engines with Interactive Agents (2021, neutral): “This line of research shows that RL for text reading can help transfer (Narasimhan et al., 2018) and generalization (Zhong et al., 2020) in synthetic tasks but skirts the challenges of more realistic language-based problems.”
Investigating Human Priors for Playing Video Games (2018, background): “…have looked into incorporating priors in RL agents via object representations (Diuk et al., 2008; Kansky et al., 2017) or language grounding (Narasimhan et al., 2017), but progress will be constrained until the ﬁeld develops a better understanding of the kinds of prior knowledge humans…”
A Deep Reinforcement Learning Perspective on Internet Congestion Control (2019, background): “While these empirical results are promising, the question of generalization, or transfer, in RL is an active field of research (Tamar et al., 2016; Barreto et al., 2017; Narasimhan et al., 2017; Higgins et al., 2017), and providing guarantees about the performance of a congestion control policy when”
Direct Policy Transfer via Hidden Parameter Markov Decision Processes (2018, background): “There exists a broad literature on transfer learning within reinforcement learning via latent representations (e.g. (Taylor and Stone, 2009; Hausman et al., 2018; Ha and Schmidhuber, 2018; Narasimhan et al., 2017; Morton and Kochenderfer, 2017)).”
Cognitive Architectures for Language Agents (2023, background): “In RL, “reading to learn” approaches (Branavan et al., 2012; Narasimhan et al., 2018; Hanjie et al., 2021; Zhong et al., 2021) leverage game manuals and facts as a semantic memory to affect the policy.”