Temporal Difference Bayesian Model Averaging: A Bayesian Perspective on Adapting Lambda

By Carlton Downey, Scott Sanner. 2010

28 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning (2016, methodology): “…which includes the original linear-time algorithm (Sutton, 1988), least-squares formulations (Bradtke & Barto, 1996; Mahmood et al., 2014), methods for adapting λ (Downey & Sanner, 2010), true-online methods (van Hasselt et al., 2014), and the recent emphatic methods (Mahmood et al., 2015).”
TD γ : Re-evaluating Complex Backups in Temporal Difference Learning (2011, methodology): “While several different algorithms exist within the TD(λ) family—the original incremental and online algorithm [2], replacing traces [3], least-squares algorithms [4], algorithms for learning state-action value functions [1, 5], and algorithms for adapting λ [6], among others—the λ-return formulatio”
Policy Evaluation Using the Ω-Return (2015, background): “While several different algorithms exist within the TD(λ) family—the original linear-time algorithm [1], least-squares formulations [2], and methods for adapting λ [3], among others—the λ-return formulation has remained unchanged since its introduction in 1988 [1].”
Reinforcement Learning with Multiple Experts: A Bayesian Model Combination Approach (2018, methodology): “In transfer learning, for example, the decision maker uses prior knowledge obtained from training on task(s) to improve performance on future tasks (Konidaris and Barto [2006]).”
A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning (2016, methodology): “However, it has also been described as a credit-assignment parameter [19], as a method to encode probability of transitions [25], a way to incorporate the agent’s confidence in its value func- tion estimates [23, 29], and as an averaging of n-step returns [23].”
TD_gamma: Re-evaluating Complex Backups in Temporal Difference Learning (2011, methodology): “While several different algorithms exist within the TD(λ) family—the original incremental and online algorithm [2], replacing traces [3], least-squares algorithms [4], algorithms for learning state-action value functions [1, 5], and algorithms for adapting λ [6], among others—the λ-return formulatio”
Online meta-learning by parallel algorithm competition (2017, methodology): “Proposed approaches for adapting individual meta-parameters include the incremental delta bar delta method [29] to tune α , a method based on variance of the action value function [15] to tune the inverse temperature β in softmax action selection, and a Bayesian model averaging approach [5] and the ”