Growing Action Spaces
By Gregory Farquhar, Laura Gustafson, Zeming Lin, Shimon Whiteson, Nicolas Usunier, Gabriel Synnaeve. 2019
50 citations recorded by the source corpus.
In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.
Subfields: Artificial Intelligence, Sociology and Political Science
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Chongjie Zhang
- Tonghan Wang
- Bei Peng
- Shimon Whiteson
- Anuj Mahajan
- Tarun Gupta
- Christian Schröder de Witt
- J. Urgen Schmidhuber
In Sociology and Political Science
Citations in the corpus
- RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, methodology): “ss different tasks (Chen et al., 2019). By contrast, we use action representations to factor multi-agent tasks. Besides learning action representations, Czarnecki et al. (2018); Murali et al. (2018); Farquhar et al. (2020) propose to use curriculum learning over action spaces of increasing sizes to ”