Reinforcement Learning as Classification: Leveraging Modern Classifiers

By Michail G. Lagoudakis, Ronald E. Parr. 2003

184 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Trust Region Policy Optimization (2015, methodology): “Pirotta et al. (2013) also build on and generalize Kakade and Langford’s results, and they derive different algorithms from the ones here.”
Approximate Policy Iteration Schemes: A Comparison (2014, methodology): “It is natural to also consider the algorithm API( α ) (mentioned by Lagoudakis & Parr (2003a)), a variation of API that is conservative like CPI( α ) in the sense that it mixes the new policy with the previous ones with weights α and 1 − α , but that directly uses the distribution ν in the…”
Preference-Based Policy Iteration: Leveraging Preference Learning for Reinforcement Learning (2011, methodology): “Most notably, the training set here is formed as a multi-class training set, whereas in [11] it was formed as a binary training set, learning a binary policy predicate π̂ : S × A → {0, 1}.”
Tree-Based Batch Mode Reinforcement Learning (2005, background): “…than the complete generative model assumption of Gordon (footnote 9, page 511). standard supervised classification problems has been developed (see Lagoudakis and Parr, 2003b; Bagnell et al., 2003), taking its roots from the policy iteration algorithm, another classical dynamic programming…”
Rollout Sampling Policy Iteration for Decentralized POMDPs (2010, unknown): “The rollout sampling method has been introduced to learn MDP policies without explicitly representing the value function [9, 10, 12].”
Approximate Dynamic Programming Finally Performs Well in the Game of Tetris (2013, unknown): “Fortunately a class of such ADP algorithms, called classification-based policy iteration (CbPI), have been recently developed and analyzed [12, 7, 13, 9, 17].”
Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning (2014, methodology): “Further details on the algo-rithm can be obtained from Lagoudakis and Parr (2003).”
Multi-Bandit Best Arm Identification (2011, methodology): “Moreover, we plan to apply the algorithms in troduced in this paper to the problem of rollout allocation for classification-based policy iterat ion in reinforcement learning [9, 6], where the goal is to identify the greedy action ( arm) in each of the states ( bandit) in a training set.”
Approximate Policy Iteration with a Policy Language Bias (2003, methodology): “(2)In concurrent work, [18] pursued a similar approach to API in attribute -value domains.”
Deep Reinforcement Learning in Large Discrete Action Spaces (2015, methodology): “In the ECOC-based approach case, a policy is learned through Rollouts Classification Policy Iteration (Lagoudakis & Parr, 2003), and the policy is defined as a multi-class ECOC classifier.”
Approximate modified policy iteration and its application to the game of Tetris (2015, methodology): “The idea is similar to the classification-based PI algorithms (Lagoudakis and Parr, 2003b; Fern et al., 2006; Lazaric et al., 2010c; Gabillon et al., 2011) in which we search for the greedy policy in a policy space Π (defined by a classifier) instead of computing it from the estimated value or…”
Analysis of Classification-based Policy Iteration Algorithms (2016, methodology): “While in Lagoudakis and Parr (2003b) the goal is to minimize the number of misclassifications , i.e., they use a 0/1 loss function, DPI learns a policy which aims at minimizing the error L π k .”
Conservative and Greedy Approaches to Classification-Based Policy Iteration (2012, methodology): “The CBPI algorithms by Lagoudakis and Parr (2003b) and Fern et al. (2006) and the algorithm presented and analyzed in Lazaric et al. (2010a) belong to this category.”
Kernelized value function approximation for reinforcement learning (2009, methodology): “This decomposition can be useful for the model-based view because it can guide the selection of potentially different regularization parameters for each component of the solution.”
Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes (2011, background): “Lagoudakis and Parr (2003) proposed a classification-based API approach that is closely related to ours.”
Multistage Stochastic Programming: A Scenario Tree Based Approach to Planning under Uncertainty (2011, background): “The literature on this aspect is large: see, for instance, Lagoudakis and Parr (2003); Ernst, Geurts, and Wehenkel (2005); Langford and Zadrozny (2005); Munos and Szepesvári (2008).”