Comparing Value-Function Estimation Algorithms in Undiscounted Problems

8 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

RUDDER: Return Decomposition for Delayed Rewards (2018, background): “ially small, fading exponentially with the number of delay steps. Q-learning with learning rates 1=iat the ith update leads to an arithmetic mean as estimate, which was shown to be exponentially slow [9]. If for a ﬁxed learning rate the agent always travels along the same sequence of states, then TD”
Algorithms for Reinforcement Learning (2010, methodology): “In this example, the rate of convergence of the Monte-Carlo method is not impacted by the value of N , while TD(0) would get slower with N increasing (for an informal argument, see Sutton, 1988; for a formal one with exact rates, see Beleznay et al., 1999).”