Combining importance sampling and temporal difference control variates to simulate Markov Chains

By Ramandeep S. Randhawa, Sandeep Juneja. 2004

26 citations recorded by the source corpus.

Subfields: Management Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Information Systems

1. Matthieu Geist
2. Huizhen Yu

In Management Science and Operations Research

1. Bruno Scherrer
2. Matthieu Geist
3. Huizhen Yu
4. Dimitri P. Bertsekas
5. Richard S. Sutton
6. Martha White
7. Shie Mannor
8. Aviv Tamar

Citations in the corpus

• On Convergence of Emphatic Temporal-Difference Learning (2015, methodology): “7) has a unique solution, and bounds on the approximation error of ETD(λ) can be derived using the approach of Scherrer (2010). (For details of this discussion, see Remark C.”
• Off-policy learning with eligibility traces: a survey (2013, background): “See also Randhawa and Juneja (2004).”
• Least Squares Temporal Difference Methods: An Analysis under General Conditions (2012, methodology): “It is also known in this setting that the estimator of the total cost up to time τ , L τ 0 τ − 1 =0 g ( i (cid:5) , i (cid:5) +1 ), can have infinite variance; this is shown by Randhawa and Juneja [27].”
• Weak Convergence Properties of Constrained Emphatic Temporal-difference Learning with Constant and Slowly Diminishing Stepsize (2015, methodology): “However, beyond simply constraining the iterates, more variance reduction techniques are needed, such as control variates [38, 1] and weighted importance sampling [36, 35, 27, 26].”