A theoretical analysis of Model-Based Interval Estimation

By Alexander L. Strehl, Michael L. Littman. 2005

196 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Informing sequential clinical decision-making through reinforcement learning: an empirical study (2010, background): “For example, in (Kaelbling et al. 1996), (Strehl and Littman 2004), and (Strehl and Littman 2005), measures of confidence are used to guide exploration.”
Variational Bayesian Reinforcement Learning with Regret Bounds (2018, methodology): “ptimisminthefaceofuncertainty ,wherebythe agent searches for a value function that is a high-probability upper bound on the true value function, then follows the greedy policy for that value function [57, 103, 100]. This process may involve, for example, constructing an upper-bound on the reward fun”
Hindsight Experience Replay (2017, methodology): “Moreover, we compare against DDPG with count-based exploration5 (Strehl and Littman, 2005; Kolter and Ng, 2009; Tang et al., 2016; Bellemare et al., 2016; Ostrovski et al., 2017).”
PAC Bounds for Discounted MDPs (2012, methodology): “The lower bound, which comes without assumptions, improves on the work of Strehl et al. (2009) by being both larger and more general.”
Safe Exploration in Markov Decision Processes (2012, background): “We are comparing against our own adapted R-max exploration objective, the original R-max objective (Brafman & Tennenholtz, 2001) and the Near-Bayesian exploration objective (Kolter & Ng, 2009).”
Minimax Regret Bounds for Reinforcement Learning (2017, background): “…of concentration inequalities to the optimal value function as a whole, rather than to the transitions probabilities (to improve scaling in S), and we define Bernstein-based "exploration bonuses" that use the empirical variance of the estimated values at the next states (to improve scaling in H).”
On Oracle-Efficient PAC RL with Rich Observations (2018, background): “[5] Alexander L.”
The many faces of optimism: a unifying approach (2008, methodology): “The first two benchmark problems, RiverSwim and SixArms, were taken from Strehl and Littman (2006).”
Near-Optimal BRL using Optimistic Local Transitions (2012, unknown): “For example, r-max [Brafman and Tennenholtz, 2003], mbie-eb [Strehl and Littman, 2005] and Delayed Q-Learning [Strehl et al.”
Quantifying Uncertainty in Batch Personalized Sequential Decision Making (2014, methodology): “In the online model-based RL literature, Strehl and Littman (2005) estimate model-based intervals that guide learning but do not evaluate policies.”
Meta reinforcement learning as task inference (2019, background): “…with shaping the agent’s memory which is an essential component of any learning algorithm–for example, many efﬁcient reinforcement learning algorithms keep track of how many times the agent visited each state (e.g. Strehl & Littman (2005)), a quantity which depends on the agent’s full history.”
Percentile Optimization for Markov Decision Processes with Parameter Uncertainty (2010, background): “Exploration methods range from slowly converging ones such as †-greedy exploration to better behaved ones, such as regret minimization (see Hazan et al. (2006)) and model based interval estimation (see Strehl and Littman (2005) ) which is for instance known to lead with high-probability to near-opt”
Model-Based Active Exploration (2018, methodology): “We used a randomized version of the Chain Environment (Figure 1), designed to be hard to explore proposed by (Osband et al., 2016) which is a simpliﬁed generalization of RiverSwim (Strehl and Littman, 2005).”
Bandit Algorithms (2020, background): “The RiverSwim environment is originally due to Strehl and Littman [2008]. The environment has two actions A = {left,right} and S = [S] with S ≥ 2.”
Distributional Reinforcement Learning for Efficient Exploration (2019, background): “Exploration is a long standing problem in Reinforcement Learning (RL), where optimism in the face of uncertainty is one fundamental principle (Lai & Robbins, 1985; Strehl & Littman, 2005).”