Exploration Bonuses and Dual Control

By Peter S. Dayan, Terrence J. Sejnowski. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

An empirical evaluation of interval estimation for Markov decision processes (2004, methodology): “A greedy policy with respect to the learned " certainty equivalent " model (Dayan & Sejnowski, 1996) can be computed by solving Equation 1 with T = ˆ T and R = ˆ R.”
Decision theory, reinforcement learning, and the brain (2008, background): “Some approaches evaluate actions according to their expected exploitative value plus an uncertainty bonus intended to capture the additional value of exploration (Sutton, 1990; Kaelbling, 1993; Dayan and Sejnowski, 1996; Dearden et al., 1998).”
From hardware and software to kernels and envelopes: a concept shift for robotics, developmental psychology, and brain sciences (2011, background): “…framework including for instance “novelty bonuses” (Kakade and Dayan, 2002) that distort the structure of the reward to include novelty effects (in a similar manner that ”exploration bonuses” permit to ensure continued exploration in theoretical machine learning models (Dayan and Sejnowski, 1996)).”
Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning (2017, background): “…rewards to value visited actions and states through measuring their novelty or the improvement of predictions that they provide, e.g. Sutton (1990); Dayan and Sejnowski (1996); Schmidhuber (1991); Oudeyer et al. (2007) or more recently Bellemare et al. (2016); Houthooft et al. (2016); Pathak et…”
A Agent Environment StatesActions Rewards Critic B Agent Internal Environment Rewards Critic External Environment Sensations StatesDecisions Actions " Organism " Figure 1 (2004, background): “Theoretical treatments [14, 15] have directly related dopamine activity with mechanisms for controlling exploration in RL such as exploration and shaping bonuses [27, 4, 19].”
In Search of the Neural Circuits of Intrinsic Motivation (2007, background): “…framework including for instance ‘‘novlty bonuses’’ (Kakade and Dayan, 2002) that distort the structure of the eward to include novelty effects (in a similar manner that ‘èxploration onuses’’ permit to ensure continued exploration in theoretical machine earning models (Dayan and Sejnowski, 1996)).”
Exploration from Generalization Mediated by Multiple Controllers (2013, background): “Under these circumstances, the index quantifies precisely an exploration bonus (Dayan and Sejnowski 1996; Kakade and Dayan 2002; Ng et al. 1999; Sutton 1990) for choosing (i.”
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms (2000, background): “All such algorithms face a tradeoff between exploitation and exploration (Thrun, 1992; Kumar & Varaiya, 1986; Dayan & Sejnowski, 1996), i.e., between choosing actions that are best according to the current state of knowledge, and actions that are not the current best but improve the state of…”