BeBold: Exploration Beyond the Boundary of Explored Regions
By Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian. 2020
45 citations recorded by the source corpus.
Efficient exploration under sparse rewards remains a key challenge in deep reinforcement learning. To guide exploration, previous work makes extensive use of intrinsic reward (IR). There are many heuristics for IR, including visitation counts, curiosity, and state-difference. In this paper, we analyze the pros and cons of each method and propose the regulated difference of inverse visitation counts as a simple but effective criterion for IR. The criterion helps the agent explore Beyond the Boundary of explored regions and mitigates common issues in count-based methods, such as short-sightedness and detachment. The resulting method, BeBold, solves the 12 most challenging procedurally-generated tasks in MiniGrid with just 120M environment steps, without any curriculum learning. In comparison, the previous SoTA only solves 50% of the tasks. BeBold also achieves SoTA on multiple tasks in NetHack, a popular rogue-like game that contains more challenging procedurally-generated environments.
Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Networks and Communications
In Management Science and Operations Research
Citations in the corpus
- TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations (2021, background): “However, current DRL systems still suffer from challenges of multi-agent coordination (Rashid et al., 2018; Mahajan et al., 2019; Yu et al., 2021), sparse rewards (Taiga et al., 2019; Zhang et al., 2020), stochastic environments (Kurach et al., 2019; Team et al., 2021), etc.”