Analysis of Classification-based Policy Iteration Algorithms

By Alessandro Lazaric, Mohammad Ghavamzadeh, R´emi Munos. 2016

105 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Dual Policy Iteration (2018, background): “5 and classic API policy improvement procedure is that we use η n ’s disadvantage function A η n , i.e., we are performing imitation learning by treating η n as an expert in this iteration [24, 33].”
Approximate Policy Iteration Schemes: A Comparison (2014, methodology): “Implementing this algorithm in practice can trivially be done through cost-sensitive classification in a way similar to Lazaric et al. (2010). It could also be done with a straight-forward extension of LSTD(λ) to non-stationary policies.”
Policy Distillation (2015, methodology): “Often, the teacher signal comes from a model-based algorithm, for example in regret-based approximate policy iteration (Lazaric et al., 2010) or by using Monte Carlo tree search as an oracle Guo et al. (2014).”
Finite-Sample Analysis of Proximal Gradient TD Algorithms (2015, methodology): “…focused on batch RL (or approximate dynamic programming) algorithms (e.g., Kakade and Langford 2002; Munos and Szepesv´ari 2008; Antos et al. 2008; Lazaric et al. 2010a), especially those that are least squares TD (LSTD)-based (e.g., Lazaric et al. 2010b; Ghavamzadeh et al. 2010, 2011; Lazaric…”
Approximate Dynamic Programming for Two-Player Zero-Sum Markov Games (2015, methodology): “Zero-sum two-player discounted SG is a subclass of SSPG. Similar algorithms exist to solve those games including PI (Patek, 1997), VI (Shapley, 1953) and MPI (called also Generalized Policy Iteration (GPI) (Patek, 1997) in the context of SSPG).”
Direct Policy Iteration with Demonstrations (2015, methodology): “We first introduce two technical lemma whose proofs follow exactly the same steps as in Lemma 1 of [Lazaric et al., 2010].”
Preference-based Evolutionary Direct Policy Search (2013, background): “[7] generalize classification-based approximate policy iteration as proposed in [19, 20].”
A Theoretical Analysis of Deep Q-Learning (2019, background): “1, also known as the error propagation (Munos and Szepesvári, 2008; Lazaric et al., 2016; Scherrer et al., 2015; Farahmand et al., 2010, 2016) in the batch reinforcement”
Is the Bellman residual a bad proxy? (2016, background): “The bound of Thm. 1 matches the bounds of DPI or CPI.”
Approximate modified policy iteration and its application to the game of Tetris (2015, methodology): “The idea is similar to the classification-based PI algorithms (Lagoudakis and Parr, 2003b; Fern et al., 2006; Lazaric et al., 2010c; Gabillon et al., 2011) in which we search for the greedy policy in a policy space Π (defined by a classifier) instead of computing it from the estimated value or…”
Safe Policy Iteration (2013, background): “Most API studies and algorithms focus on reducing the approximation error in the policy evaluation step (Lagoudakis & Parr, 2003; Munos, 2005; Lazaric et al., 2010; Gabillon et al., 2011), and then perform policy improvement by taking the related greedy policy.”