Approximate Policy Iteration with a Policy Language Bias

By Alan Fern, Sung Wook Yoon, Robert Givan. 2003

172 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning First-Order Symbolic Representations for Planning from the Structure of the State Space (2019, methodology): “Inductive logic programming methods [31] have been used for learning general policies but from symbolic encodings too [23, 28, 14].”
Integrating Guidance into Relational Reinforcement Learning (2004, background): “Fern, Yoon, and Givan (2003) use an approximate variant of policy iteration to handle large state spaces ( Fern, Yoon, and Givan, 2003 ).”
Transfer Learning for Reinforcement Learning Domains: A Survey (2009, background): “The goals of this survey are to introduce the reader to the transfer learning problem in RL domains, to organize and discuss current transfer methods, and to enumerate important open questions in RL transfer.”
Preference-Based Policy Iteration: Leveraging Preference Learning for Reinforcement Learning (2011, background): “For example, it was suggested that a policy-based generation of states may be preferable to a random selection [4].”
Practical solution techniques for first-order MDPs (2009, methodology): “This can be done with pure reinforcement learning using relational decision or regression trees to learn a value or Q-function ( Dzeroski et al., 2001 ), combining this with supervised guidance (Driessens and Dzeroski, 2002), or using Gaussian processes and graph kernels over relational structures t”
Value Functions for RL-Based Behavior Transfer: A Comparative Study (2005, background): “Other research ( Fern, Yoon, & Givan 2004 ) has shown that it is possible to learn policies for large-scale planning task s that generalize across different tasks in the same domain.”
Conservative and Greedy Approaches to Classification-Based Policy Iteration (2012, methodology): “2003; Lagoudakis and Parr 2003b; Fern, Yoon, and Givan 2006; Lazaric, Ghavamzadeh, and Munos 2010a).”
Approximate modified policy iteration and its application to the game of Tetris (2015, methodology): “…three implementations of AMPI (Section 3) that generalize the AVI implementations of Ernst et al. (2005); Antos et al. (2007); Munos and Szepesvári (2008) and the classification-based API algorithms of Lagoudakis and Parr (2003b); Fern et al. (2006); Lazaric et al. (2010c); Gabillon et al. (2011).”
FF-Replan: A Baseline for Probabilistic Planning (2007, methodology): “In the deterministic planning community, probabilistic planning techniques have been applied to find planning domain “solvers” that find control information for planning domains, for example using approximate policy iteration ( Fern, Yoon, & Givan 2006 ).”
Analysis of Classification-based Policy Iteration Algorithms (2016, methodology): “To address the above issues, mainly 3 and 4, 1 variants of API have been proposed that replace the usual value function learning step (approximating the action-value function over the entire state-action space) with a learning step in a policy space (Lagoudakis and Parr, 2003b; Fern et al., 2004).”
Robust Supervised Learning (2005, unknown): “(See for instance, (Bagnell et al. 2003) and (Fern et al. 2003)).”
Approximate Modified Policy Iteration (2012, methodology): “The idea is similar to the classification-based PI algorithms (Lagoudakis & Parr, 2003; Fern et al., 2006; Lazaric et al., 2010; Gabillon et al., 2011) in which we search for the greedy policy in a policy space Π (defined by a classifier) instead of computing it from the estimated value or action-va”
A new representation and associated algorithms for generalized planning (2011, background): “[12] present an approach for developing general policies which can be used over a wide variety of problem instances.”
Rollout Sampling Policy Iteration for Decentralized POMDPs (2010, methodology): “The rollout sampling method has been introduced to learn MDP policies without explicitly representing the value function [9, 10, 12].”
Object focused q-learning for autonomous agents (2013, unknown): “Fern proposes a relational policy iteration algorithm [5], but it relies on a resettable simulator to perform Monte-Carlo rollouts and also needs either an initial policy that performs well or a good cost heuristic of the domain.”