Control of exploitation-exploration meta-parameter in reinforcement learning

By Shin Ishii, Wako Yoshida, Junichiro Yoshimoto. 2002

225 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Molecular Biology, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Molecular Biology

In Reinforcement Learning

Citations in the corpus

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning (2016, background): “Earlier studies have investigated automatic tuning of hyper-parameters, such as learning rate and temperature (Ishii et al., 2002; Schweighofer & Doya, 2003), as a form of meta-learning.”
Meta-learning in Reinforcement Learning (2003, background): “Ishii et al. (2002) proposed to directly tune b as a function of the inverse of the variance of the action value function.”
Free Energy, Value, and Attractors (2011, background): “There are clearly many different ways in which we could formulate generalised policies and constrain them with cost-functions: We will concentrate on the use of cost to specify itinerant policies: itinerancy is important because it provides a principled explanation for exploration and foraging in et”
The free-energy principle: a unified brain theory? (2010, background): “Priors of this sort may provide a principled way to understand the exploration–exploitation trade-off”
Online meta-learning by parallel algorithm competition (2017, methodology): “Proposed approaches for adapting individual meta-parameters include the incremental delta bar delta method [29] to tune α , a method based on variance of the action value function [15] to tune the inverse temperature β in softmax action selection, and a Bayesian model averaging approach [5] and the ”
Metalearning and neuromodulation (2002, methodology): “In order to keep variability of actions at different states, Ishii and colleagues proposed a method of state-dependent control of the inverse temperature (Ishii et al., 2002).”
Modulators of decision making (2008, background): “Another common solution is 'Boltzmann selection' in which selection probabilities p are proportional to the exponentials of the estimated values: By an analogy with thermodynamic”
Medial prefrontal cortex and the adaptive regulation of reinforcement learning parameters. (2013, background): “The adjustment of behavioral parameters based on such outcome history follows meta-learning principles (Doya, 2002; Ishii et al., 2002) and is here restricted to the tuning of the β parameter which regulates the exploration rate of the agent.”