Diversity-Driven Exploration Strategy for Deep Reinforcement Learning

By Zhang-Wei Hong, Tzu-Yun Shann, Shih-Yang Su, Yi-Hsiang Chang, Chun-Yi Lee. 2018

147 citations recorded by the source corpus.

Efficient exploration remains a challenging research problem in reinforcement learning, especially when an environment contains large state spaces, deceptive local optima, or sparse rewards. To tackle this problem, we present a diversity-driven approach for exploration, which can be easily combined with both off- and on-policy reinforcement learning algorithms. We show that by simply adding a distance measure to the loss function, the proposed methodology significantly enhances an agent's exploratory behaviors, and thus preventing the policy from being trapped in local optima. We further propose an adaptive scaling method for stabilizing the learning process. Our experimental results in Atari 2600 show that our method outperforms baseline approaches in several tasks in terms of mean scores and exploration efficiency.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “The design of exploration strategies for RL agents is an area of active research (see e.g. [18]).”
Exploration in Deep Reinforcement Learning: A Survey (2022, background): “The diversity among policies was measured in [80].”
Effective Diversity in Population-Based Reinforcement Learning (2020, background): “This formulation has been shown to boost exploration, and as such, there have been a variety of attempts to incorporate novelty-based metrics into RL algorithms (Jung et al., 2020; Doan et al., 2019; Hong et al., 2018).”