Regularized Policy Iteration with Nonparametric Function Spaces

By Amir-massoud Farahmand, Mohammad Ghavamzadeh, Csaba Szepesvari, Shie Mannor. 2016

115 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

AlgaeDICE: Policy Gradient from Arbitrary Experience (2019, methodology): “There are several alternatives to handle this issue (e.g., Antos et al., 2008; Farahmand et al., 2016; Feng et al., 2019).”
Learning When-to-Treat Policies (2019, background): “There is a long history of theoretical work on batch offline Qlearning in the reinforcement learning community (e.g., Munos 2003; Munos and Szepesvári 2008; Farahmand et al. 2016) focused on bounding the error of the returned policy during batch learning relative to the optimal policy.”
Information-Theoretic Considerations in Batch Reinforcement Learning (2019, neutral): “While results similar to Theorems 2 and 11 exist (Farahmand 2011, Chapter 5; see also Lazaric et al. (2012); Pires & Szepesv´ari (2012); Farahmand et al. (2016)), according to our knowledge, fast rate for the minimax al-gorithm has not been established before: for example, Antos et al. (2008);…”
A Kernel Loss for Solving the Bellman Equation (2019, background): “Our work is also related to a line of interesting work on Bellman residual minimization (BRM) based on nested optimization (Antos et al., 2008; Farahmand et al., 2008, 2016; Hoffman et al., 2011; Chen & Jiang, 2019).”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, methodology): “To embrace general function classes, the framework of batch RL (Munos, 2007; Antos et al., 2008; Munos and Szepesvári, 2008; Antos et al., 2008; Farahmand et al., 2016) can be adapted to the multi-agent settings, as in the recent works Yang et al.”
Is Pessimism Provably Efficient for Offline RL? (2020, result): “In comparison with existing literature (Antos et al., 2007; 2008; Munos and Szepesv´ari, 2008; Farahmand et al., 2010; 2016; Scherrer et al., 2015; Liu et al., 2018; Nachum et al., 2019a;b; Chen and Jiang, 2019; Tang et al., 2019; Kallus and Uehara, 2019; 2020; Fan et al., 2020; Xie and Jiang,…”
A Geometric Perspective on Optimal Representations for Reinforcement Learning (2019, methodology): “tile coding (Sutton, 1996) and the domain-independent features used in some Atari 2600 game-playing agents (Bellemare et al., 2013; Liang et al., 2016), and nonparametric methods (Ernst et al., 2005; Farahmand et al., 2016; Tosatto et al., 2017). Today, the method of choice is deep learning. Deep le”
Iterative Value-Aware Model Learning (2018, background): “…et al., 2009, Farahmand and Precup, 2012, Mnih et al., 2015, Tosatto et al., 2017, Farahmand et al., 2017b] or (Approximate) Policy Iteration (API) algorithms [Lagoudakis and Parr, 2003, Antos et al., 2008, Bertsekas, 2011, Lazaric et al., 2012, Scherrer et al., 2012, Farahmand et al., 2016b].”
A Theoretical Analysis of Deep Q-Learning (2019, methodology): “See, e.g., Munos and Szepesvári (2008); Lazaric et al. (2016); Farahmand et al. (2010, 2016).”
Reinforcement learning for control: Performance, stability, and deep approximators (2018, methodology): “Research on batch RL methods is ongoing, with recent developments in e.g. nonparametric approximation (Farahmand et al., 2016), exploitation of low-level controllers called “op-tions” (Mann et al., 2015), and combinations with deep learning (Lee et al., 2017).”