Adaptive Transformers in RL
By Shakti Kumar, Jerrod Parker, Panteha Naderian. 2020
18 citations recorded by the source corpus.
Recent developments in Transformers have opened new interesting areas of research in partially observable reinforcement learning tasks. Results from late 2019 showed that Transformers are able to outperform LSTMs on both memory intense and reactive tasks. In this work we first partially replicate the results shown in Stabilizing Transformers in RL on both reactive and memory based environments. We then show performance improvement coupled with reduced computation when adding adaptive attention span to this Stable Transformer on a challenging DMLab30 environment. The code for all our experiments and models is available at this https URL.
Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Reinforcement Learning, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Michael Janner
- Qiyang Li
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Josh Tenenbaum
- Igor Mordatch
- Ofir Nachum
In Electrical and Electronic Engineering
- Michael Janner
- Qiyang Li
- Sergey Levine
- Chelsea Finn
- Aviral Kumar
- Ofir Nachum
- Shixiang Shane Gu
- George Tucker
In Reinforcement Learning
- Michael Janner
- Qiyang Li
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Ofir Nachum
- Igor Mordatch
- Shixiang Shane Gu
In Model Architectures
Citations in the corpus
- Offline Reinforcement Learning as One Big Sequence Modeling Problem (2021, methodology): “…works have studied applying sequence models of various types to represent components in standard RL algorithms, such as policies, value functions, and models (Bakker, 2002; Heess et al., 2015a; Chiappa et al., 2017; Parisotto et al., 2020; Parisotto & Salakhutdinov, 2021; Kumar et al., 2020b).”
- Reinforcement Learning as One Big Sequence Modeling Problem (2021, background): “Indeed, a number of prior works have studied applying sequence models of various types to represent components in standard RL algorithms, such as policies, value functions, and models [4, 21, 9, 45, 44, 34].”