Policy Evaluation Using the Ω-Return

By Philip S. Thomas, Scott Niekum, Georgios Theocharous, George Dimitri Konidaris. 2015

18 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning (2016, background): “Recent work has suggested that the λ-return could be replaced by more statistically principled complex returns like the γ-return (Konidaris et al., 2011) or Ωreturn (Thomas et al., 2015).”
Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion (2018, background): “[30] define the Ω-return, an alternative to the λ-return that accounts for the variance of, and correlations between, predicted returns at multiple timesteps.”
Using Options and Covariance Testing for Long Horizon Off-Policy Policy Evaluation (2017, background): “Other recent efforts that estimate a value function using off-policy data rather than just the performance of a policy [6, 11, 19] also suffer from bias if the input state description is not Markovian (such as if the domain description induces partial observability).”
Safe Reinforcement Learning (2015, methodology): “However, the analyses and methods in this dissertation extend to the settings where these sets are countably or uncountably infinite and where the probability distributions thereover may or may not have probability density functions (Thomas et al., 2015b).”
True Online Temporal-Difference Learning (2015, methodology): “Furthermore, Thomas et al. (2015) proposed a method based on a multi-step update target, which they call the Ω-return.”
On the Analysis of Complex Backup Strategies in Monte Carlo Tree Search (2016, background): “Such complex backup strategies include the λ-return (Sutton, 1988), γ-return (Konidaris et al., 2011), and the Ω-return (Thomas et al., 2015), where the last two approaches are parameter free.”