Efficient model-based exploration

By Marco A. Wiering, J. Urgen Schmidhuber. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Meta-Learning in Neural Networks: A Survey (2020, neutral): “lowed by various extensions [54], [55].”
The many faces of optimism: a unifying approach (2008, methodology): “In this paper, we used the improved prioritized sweeping algorithm of Wiering and Schmidhuber (1998).”
A theoretical analysis of Model-Based Interval Estimation (2005, background): “The core idea of MBIE was first introduced by Wiering and Schmidhuber (1998); however the form of the confidence intervals were ad hoc and problematic for analysis.”
Using MDP Characteristics to Guide Exploration in Reinforcement Learning (2003, background): “Interval Estimation (IE) [3, 16], as well as its global equivalent, IEQL+ [7], bias exploration toward actions that have the highest variance in the action value samples.”
Exploring parameter space in reinforcement learning (2010, background): “Many exploration techniques have been developed for the case of discrete actions [28, 32], commonly divided into undirected and directed exploration.”
An analysis of model-based Interval Estimation for Markov Decision Processes (2008, methodology): “We introduce a version of Model-based Interval Estimation (MBIE), which not only combines Interval Estimation with model-based reinforcement learning, but also comes with a formal PAC-like 2 learning-time guarantee in MDPs.”
The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach (2018, methodology): “…: Model-based RL research dates back to the 90s, and includes well-known algorithms such as Dyna, R-max and E 3 (Sutton, 1990; Moore & Atkeson, 1993; Peng & Williams, 1993; Kuvayev & Sutton, 1997; Brafman & Tennenholtz, 2002; Kearns & Singh, 2002; Wiering & Schmidhuber, 1998; Wiering et al., 1999).”
Learning and adaptivity in interactive recommender systems (2007, methodology): “However, the success of an optimal policy depends heavily on exploration while gaining experience [20].”
Sample Complexity Bounds of Exploration (2012, methodology): “Advanced techniques exist that guide exploration using interval estimation (Kaelbling, 1993; Meuleau and Bourgine, 1999; Wiering and Schmidhuber, 1998), information gain (Dearden et al, 1999), MDP characteristics (Ratitch and Precup, 2002), or expert demonstration (Abbeel and Ng, 2005), to name a…”
Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress (2012, background): “An interesting aspect about this work is its reliance on empirical measures of learning progress to drive exploration in reinforcement learning [17, 6].”
An empirical evaluation of interval estimation for Markov decision processes (2004, methodology): “Many approaches to exploration in MDPs have been proposed (Thrun, 1992; Wiering & Schmidhuber, 1998; Dearden et al., 1999; Meuleau & Bourgine, 1999; Wyatt, 2001).”
Learning and Using Models (2012, background): “Model Based Interval Estimation (MBIE) (Wiering and Schmidhuber, 1998; Strehl and Littman, 2005) is a similar approach that looks at the distribution over transition probabilities rather than action-value distributions.”