Domain-Independent Optimistic Initialization for Reinforcement Learning

By Marlos C. Machado, Sriram Srinivasan, Michael H. Bowling. 2014

27 citations recorded by the source corpus.

In Reinforcement Learning (RL), it is common to use optimistic initialization of value functions to encourage exploration. However, such an approach generally depends on the domain, viz., the scale of the rewards must be known, and the feature representation must have a constant norm. We present a simple approach that performs optimistic initialization with less dependence on the domain.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Marc G. Bellemare
2. Sergey Levine
3. Pieter Abbeel
4. Yuxi Li
5. R´emi Munos
6. Yoshua Bengio
7. Joelle Pineau
8. Marlos C. Machado

In Electrical and Electronic Engineering

1. Marc G. Bellemare
2. R´emi Munos
3. Matthew J. Hausknecht
4. Georg Ostrovski
5. David Saxton
6. Joel Veness
7. Michael H. Bowling
8. Tom Schaul

In Reinforcement Learning

1. Marc G. Bellemare
2. Sergey Levine
3. Pieter Abbeel
4. R´emi Munos
5. Marlos C. Machado
6. Yuxi Li
7. Michael H. Bowling
8. Matthew J. Hausknecht

In Optimization & Training

1. R´emi Munos
2. David Saxton
3. Joel Veness
4. Michael H. Bowling
5. Erik Talvitie
6. Tom Schaul
7. Marc G. Bellemare
8. Georg Ostrovski

Citations in the corpus

• Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (2017, background): “Machado, Srinivasan, and Bowling (2015) extended optimistic initialization to function approximation.”
• Unifying Count-Based Exploration and Intrinsic Motivation (2016, background): “This modification led to improved results both with and without exploration bonuses (details in the appendix).”
• State of the Art Control of Atari Games Using Shallow Reinforcement Learning (2015, methodology): “These feature sets were, for some time, the standard representations for Atari 2600 games, being directly used in other work [7, 16, 17] or serving as the base for more elaborate feature sets [4].”