Reinforcement Learning with Attention that Works: A Self-Supervised Approach
By Anthony Manchin, Ehsan Abbasnejad, Anton van den Hengel. 2019
67 citations recorded by the source corpus.
Attention models have had a significant positive impact on deep learning across a range of tasks. However previous attempts at integrating attention with reinforcement learning have failed to produce significant improvements. We propose the first combination of self attention and reinforcement learning that is capable of producing significant improvements, including new state of the art results in the Arcade Learning Environment. Unlike the selective attention models used in previous attempts, which constrain the attention via preconceived notions of importance, our implementation utilises the Markovian properties inherent in the state input. Our method produces a faithful visualisation of the policy, focusing on the behaviour of the agent. Our experiments demonstrate that the trained policies use multiple simultaneous foci of attention, and are able to modulate attention over time to deal with situations of partial observability.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Li Fei-Fei
- Silvio Savarese
- Sergey Levine
- Yoshua Bengio
- J. Urgen Schmidhuber
- Ruslan Salakhutdinov
- Trevor Darrell
- Chelsea Finn
Citations in the corpus
- Towards Interpretable Reinforcement Learning Using Attention Augmented Agents (2019, background): “image classification and captioning [12, 13, 14, 15, 16, 17, 18, 19, 20], text classification [21, 22], generative models [6, 23, 24], object tracking [25], and reinforcement learning [26, 27, 28].”
- Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control (2021, background): “Recent works have attempted to equip interactive agents with attention [17, 24, 35, 43].”
- Free-Lunch Saliency via Attention in Atari Agents (2019, methodology): “[22] PPO 7 10 [24] 3 Soft self-attention * 7 Grad-CAM + upscale Reward”