Learning the Variance of the Reward-To-Go

By Aviv Tamar, Dotan Di Castro, Shie Mannor. 2016

82 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

RUDDER: Return Decomposition for Delayed Rewards (2018, background): “ors of the expectations (arithmetic mean and exponentially weighted arithmetic mean) is governed by the variance of the samples, we compute mean and variance of the return estimate qˇ(s;a). We follow [121,129,130] for deriving the mean and variance. We consider an MDP with ﬁnite horizon T, that is, ”
The Uncertainty Bellman Equation and Exploration (2017, methodology): “For application to rich environments with complex generalization we will use a deep learning architecture to learn a solution to the UBE, in the style of (Tamar et al., 2016).”
Variational Bayesian Reinforcement Learning with Regret Bounds (2018, background): “Epistemic uncertainty is distinct from the so-called aleatoric uncertainty or risk, which is the natural randomness of the process we are observing (Sobel, 1982; Tamar et al., 2016; Bellemare et al., 2017).”
A Distributional Perspective on Reinforcement Learning (2017, methodology): “Although the Gaussian case has previously been considered (Morimura et al., 2010a; Tamar et al., 2016), to the best of our knowledge we are the ﬁrst to use a rich class of parametric distributions.”
Universal Off-Policy Evaluation (2021, background): “In order to improve user experiences, applications involving direct human-machine interaction, such as robotics and autonomous driving, focus on minimizing uncertainty in their outcomes and thus use metrics like variance and entropy [52, 84].”
Efficient exploration with Double Uncertain Value Networks (2017, background): “Tamar et al. (2016) studied the variance of the return with linear function approximation.”