Provably Efficient Q-learning with Function Approximation via Distribution Shift Error Checking Oracle

By Simon Shaolei Du, Yuping Luo, Ruosong Wang, Hanrui Zhang. 2019

88 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction (2020, background): “Recent work [36] has attempted to use a distribution-checking oracle to control the amount of exploration performed.”
Provably Efficient Reinforcement Learning with Linear Function Approximation (2019, background): “…[48] which alleviates the difﬁculty of exploration, or assume the transition dynamics to be deterministic [46, 47], to have a low variance [18], or are parametrizable by a relatively small matrix [49], which alleviates the difﬁculty in estimating the transition dynamics (see Section 1.1…”
Is Long Horizon Reinforcement Learning More Difficult Than Short Horizon Reinforcement Learning? (2020, background): “…sample complexity does not depend on |S| [Li et al., 2011, Wen and Van Roy, 2017, Krishnamurthy et al., 2016, Jiang et al., 2017, Dann et al., 2018, Du et al., 2019b, Feng et al., 2020, Du et al., 2019c, Zhong et al., 2019, Yang and Wang, 2019, Jin et al., 2019, Du et al., 2019a, Roy and Dong,…”
Provably Efficient Exploration in Policy Optimization (2019, background): “See also (Du et al., 2019a; Van Roy & Dong, 2019; Lattimore & Szepesvari, 2019) for a related discussion on the necessity of the linear representation.”
Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction (2020, methodology): “Additionally, finite-time performance of Q-learning and its variants have been analyzed by [5, 9, 20, 28, 35, 44, 48] in the tabular setting, by [6, 8, 10, 17, 18, 21, 49, 52] in the context of function approximations, and by [36] with nonparametric regression.”
On Reward-Free Reinforcement Learning with Linear Function Approximation (2020, background): “…for the tabular case, while the reward-free setting is ﬁrst formalized in Jin et al. [2020], algorithms in earlier works also guarantee to collect a polynomial-size dataset with coverage guarantees [Brafman and Tennenholtz, 2002, Hazan et al., 2018, Du et al., 2019a, Misra et al., 2019].”
Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium (2020, methodology): “, 2019), as well as nonlinear ones (Wen and Van Roy, 2017; Jiang et al., 2017; Dann et al., 2018; Du et al., 2019b; Dong et al., 2019a; Du et al., 2019a).”
Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning? (2019, methodology): “Later work gave computationally efﬁcient algorithms for certain special cases (Dann et al., 2018; Du et al., 2019a; Yang & Wang, 2019b; Jin et al., 2019).”