Biasing Approximate Dynamic Programming with a Lower Discount Factor

68 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Addressing Function Approximation Error in Actor-Critic Methods (2018, background): “Another approach is a reduction in the discount factor (Petrik & Scherrer, 2009), reducing the contribution of each error.”
An Introduction to Deep Reinforcement Learning (2018, unknown): “policy found using a shorter planning horizon can actually be better than a policy learned with the true horizon (Petrik and Scherrer, 2009; Jiang et al., 2015b).”
The Dependence of Effective Planning Horizon on Model Accuracy (2015, background): “To our knowledge, [20] is the first work to show how a short horizon can reduce loss when the model is inaccurate due to approximation errors.”
Beyond the One Step Greedy Approach in Reinforcement Learning (2018, methodology): “As was previously analyzed in (Petrik & Scherrer, 2009; Strehl et al., 2009; Jiang et al., 2015) and reported (François-Lavet et al., 2015), solving a MDP with a smaller discount factor is in general easier.”
On the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes (2012, background): “Interestingly, this very constant 2γ (1−γ)2 appears in many works analyzing AVI algorithms [25, 11, 27, 12, 13, 23, 7, 6, 20, 21, 22, 9], API algorithms [15, 19, 16, 1, 8, 18, 5, 17, 10, 3, 9, 2] and in one of their generalization [26], suggesting that it cannot be improved.”