強化学習におけるVariational Information Maximizing Explorationに基づく状態探索の効率化

By 土井一輝, 大川慧悟, 志賀元紀. 2019

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Count-Based Exploration with Neural Density Models (2017, methodology): “Houthooft et al. (2016) used a second-order Taylor approximation of the prediction gain to drive exploration in continuous control.”
Variational Intrinsic Control (2016, methodology): “Let us compare this to the commonly used intrinsic motivation objective of maximizing the amount of model-learning progress, measured as the difference in compression of its experience before and after learning (Schmidhuber, 1991; 2010; Bellemare et al., 2016; Houthooft et al., 2016).”
Stochastic Neural Networks for Hierarchical Reinforcement Learning (2016, methodology): “…these success stories, these deep RL algorithms typically employ naive exploration strategies such as (cid:15) -greedy or uniform Gaussian exploration noise, which have been shown to perform poorly in tasks with sparse rewards (Duan et al., 2016; Houthooft et al., 2016; Bellemare et al., 2016).”
Loss is its own Reward: Self-Supervision for Reinforcement Learning (2016, background): “Rewards that formalize novelty, curiosity, and competence focus on learning progress and predictive error (Schmidhuber, 1991; Oudeyer et al., 2005; Houthooft et al., 2016).”
Supervision via competition: Robot adversaries for learning tasks (2016, background): “Much of the recent work in improving exploration for deep learned models has focused reducing uncertainty [22], [23] or”
Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning (2017, methodology): “In particular, we compare to Variational Information Maximizing Exploration (VIME) [7], a method which approximately maximizes Bayesian surprise and currently achieves state-of-the-art performance on continuous control with sparse rewards.”
CASSL: Curriculum Accelerated Self-Supervised Learning (2017, background): “Therefore, recent papers have investigated other exploration strategies, such as curiosity-driven exploration [6].”
Automatic Goal Generation for Reinforcement Learning Agents (2017, background): “Recently there have been other formulations of intrinsic motivation, relating to optimizing surprise (Houthooft et al., 2016; Achiam & Sastry, 2016) or surrogates of statevisitation counts (Bellemare et al.”
Cooperative Multi-agent Control Using Deep Reinforcement Learning (2017, background): “Recently, curiosity based information gain maximizing exploration strategy was explored by [16].”
Deep reinforcement learning: a survey (2020, unknown): “Compared to the aforementioned exploration approaches, variational information maximizing exploration (VIME) (Houthooft et al., 2017) approximates the probability of the dynamic model and greatly reduces the reliance on rewards using variational inference in Bayesian neural networks.”
A Separation Principle for Control in the Age of Deep Learning (2017, methodology): “This quantity is considered in [17], which uses a variational approximation similar to ours to measure the information content in the data using a neural network, and then learns a control policy for exploration that maximizes this information quantity.”