Model Selection in Reinforcement Learning

By Amir-massoud Farahmand, Csaba Szepesvari. 2011

67 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Doubly Robust Off-policy Value Evaluation for Reinforcement Learning (2015, background): “In some discounted problems the true horizon is inﬁnite, but for the purpose of policy value evaluation we can still use a ﬁnite H (usually set to be on the order of O (1 / (1 − γ )) ) so that v π,H approximates v π, ∞ with a bounded error that diminishes as H increases.”
Hyperparameter Selection for Offline Reinforcement Learning (2020, background): “Farahmand and Szepesvári (2011) have proposed BerMin for oﬄine model selection, and they investigated the convergence and other theoretical properties of it.”
Off-Policy Evaluation via Off-Policy Classification (2019, methodology): “The temporal-difference error (TD Error) is the standard Q-learning training loss, and Farahmand & Szepesvári [9] proposed a model selection algorithm based on minimizing TD error.”
Regularized Policy Iteration with Nonparametric Function Spaces (2016, methodology): “…(Samson, 2000) can be used to carry on the analysis—as have been done by Antos et al. (2008b) in the API context, by Farahmand and Szepesvári (2012) for analyzing the regularized regression problem, and by Farahmand and Szepesvári (2011) in the context of model selection for RL problems.”
Offline Policy Selection under Uncertainty (2020, background): “…notion of policy selection are a number of previous works which use model selection to refer to the problem of choosing a near-optimal Q -function from a set of candidate approximation functions (Fard and Pineau, 2010; Farahmand and Szepesvári, 2011; Irpan et al., 2019; Xie and Jiang, 2020).”
Model selection in markovian processes (2013, background): “More recent works investigating model selection in Markov processes have largely focused on a single state space (see, for example, Farahmand and Szepesvári 6, Fard and Pineau 7), selecting state representations in RL focusing on the regret; see [13], or minimizing the errors of the Bellman operator”
Learning from Limited Demonstrations (2013, methodology): “The choice of these parameters might be automated by model selection [15].”
Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error (2022, result): “These empirically minded observations caution against traditional results which suggest Bellman error as a metric for model selection (Farahmand & Szepesvári, 2011).”
Learning to Manipulate Unknown Objects in Clutter by Reinforcement (2015, background): “Bandwidth learning is a typical model selection problem (Farahmand and Szepesvári 2011).”
Regularization in reinforcement learning (2011, background): “This leads to a tractable optimization problem and enables a connection to regression function estimation [Györfi et al., 2002]. More importantly, results such as Munos [2003]; Antos et al.”
Approximate MaxEnt Inverse Optimal Control and Its Application for Mental Simulation of Human Interactions (2015, background): “We also define the following concentrability coefficients, similar to (Kakade and Langford 2002; Munos 2007; Farahmand, Munos, and Szepesvári 2010).”
Policy evaluation with temporal differences: a survey and comparison (2015, methodology): “The performance of all regularization schemes strongly depends on the regularization strength β, which has to be specified by hand, found by cross-validation or set with the method of Farahmand and Szepesvári (2011).”