The Natural Language of Actions
By Guy Tennenholtz, Shie Mannor. 2019
63 citations recorded by the source corpus.
We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We show how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then use these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shimon Whiteson
- Gregory Farquhar
- Gabriel Synnaeve
- Sergey Levine
- Nicolas Usunier
- Joseph J. Lim
- Chen Tessler
- Pieter Abbeel
Citations in the corpus
- Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, background): “…enabled learning in large action spaces (Dulac-Arnold et al., 2015; Chandak et al., 2019), transfer learning to a different action space (Chen et al., 2019b), and efficient exploration by exploiting the shared structure among actions (He et al., 2015; Tennenholtz & Mannor, 2019; Kim et al., 2019).”
- Language is Power: Representing States Using Natural Language in Reinforcement Learning. (2019, methodology): “This use of semantic structure has also been leveraged by Tennenholtz and Mannor [2019], where abstract actions (not necessarily words) were recognized as symbols of a natural and expressive language, improving performance and transfer of RL agents.”
- Generalization to New Actions in Reinforcement Learning (2020, background): “Tennenholtz & Mannor (2019) use expert demonstration data to extract contextual action representations.”
- Growing Action Spaces (2019, background): “If action-embeddings in a latent space (Tennenholtz & Mannor, 2019) can be learned from random behaviour or demonstrations then we can cluster actions in the latent space, and choose representative actions to form a coarse space.”