Off-policy Learning with Options and Recognizers

By Yasin Abbasi-Yadkori, Csaba Szepesvari. 2005

301 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Model-based reinforcement learning with parametrized physical models and optimism-driven exploration (2015, methodology): “In the case of linear dynamics or discrete systems, this optimistic exploration scheme has a number of desirable theoretical properties that make it a good choice [1], [3], [4], [7].”
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees (2018, background): “…2018; Lakshmanan et al., 2015; Hinderer, 2005; Pirotta et al., 2015; 2013), or the linear parametrization of the dynamics, policy, or value function (Abbasi-Yadkori & Szepesvári, 2011; Simchowitz et al., 2018; Dean et al., 2017; Sutton et al., 2012; Tamar et al., 2012), but not much on non-linear…”
Generalization and Exploration via Randomized Value Functions (2014, methodology): “Reinforcement learning in linear systems with quadratic cost is treated in [Abbasi-Yadkori and Szepesvári, 2011].”
Efficient Exploration Through Bayesian Deep Q-Networks (2018, methodology): “For linear quadratic models OFU is deployed to provide an optimal regret bound [2].”
Deep Exploration via Randomized Value Functions (2017, background): “There is work on model learning algorithms [20, 21, 22, 8, 23, 24, 25, 26], which apply to specific model classes and become statistically or computationally intractable for problems of practical scale.”
Near-optimal Reinforcement Learning in Factored MDPs (2014, background): “There are regret bounds for continuous state spaces where the underlying reward and transition functions are known to belong to some function class, such as Höder continuous [23] or linear quadratic control [24].”
Bayesian Optimal Control of Smoothly Parameterized Systems (2015, methodology): “The nonlinear dynamics shown in (3) shares similarities to, but allows signiﬁcantly greater generality than the Linear Quadratic (LQ) problem considered by Abbasi-Yadkori and Szepesv´ari (2011).”
Improved Regret Bounds for Undiscounted Continuous Reinforcement Learning (2015, methodology): “…results for RL with deterministic transitions in the discounted setting (Bernstein and Shimkin, 2010), as well as for RL with transition functions that are linear in state and action (Strehl and Littman, 2008; Brunskill et al., 2009; Abbasi-Yadkori and Szepesvári, 2011; Ibrahmi et al., 2012).”
Incremental Off-policy Reinforcement Learning Algorithms (2017, background): “To reduce variance, Precup et al. (2005) proposed a class of policies for which off-policy estimation produces minimum variance.”
Bootstrapped Thompson Sampling and Deep Exploration (2015, background): “Of those algorithms which do combine generalization with exploration, many require an intractable model-based planning step, or are restricted to unrealistic parametric domains [10, 11, 12].”
Data center cooling using model-predictive control (2018, background): “More recent approaches [8, 2, 17] use optimism in the face of uncertainty, where at each iteration the algorithm selects the dynamics with lowest attainable cost from some confidence set.”
Spectral Filtering for General Linear Dynamical Systems (2018, background): “For a linear dynamical system with no hidden state, the system is identiﬁable by a convex program and thus well understood (see [DMM + 17, AYS11], who address sample complexity issues and regret for system identiﬁcation and linear-quadratic control in this setting).”
Adaptive Execution: Exploration and Learning of Price Impact (2012, methodology): “It is also worth comparing CTRACE to a reinforcement learning algorithm recently proposed in Abbasi-Yadkori and Szepesvàri [2010] which appears well-suited for our problem.”