Importance sampling for stochastic simulations

By Peter W. Glynn, D. Iglehart. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

On Convergence of Emphatic Temporal-Difference Learning (2015, background): “Related examples can be found in (Glynn and Iglehart, 1989; Randhawa and Juneja, 2004; Sutton et al., 2015). the conditions of a convergence theorem from stochastic approximation theory (Kushner and Yin, 2003) and yield convergence results for TD(λ).”
Least Squares Temporal Difference Methods: An Analysis under General Conditions (2012, methodology): “In the simulation context, P need not correspond to an actual policy; instead, any sampling mechanism may be used to induce dynamics P (which may not be realizable by any policy) for the purpose of eﬃcient computation (Glynn and Iglehart [16]).”
Policy evaluation with temporal differences: a survey and comparison (2015, background): “Leveraging temporal-difference methods for off-policy estimation is based on the idea of importance sampling (cf. Glynn and Iglehart, 1989).”
Monte Carlo Gradient Estimation in Machine Learning (2019, background): “If we follow an argument due to Glynn and Iglehart (1989) and Glasserman (2013, p. 259), and assume that the expectation of the log ratio is is bounded, i.e., Ep(x;θ) [logωd(θ, h)] < ∞, then the using the strong law of large numbers we can find that this expectation converges to a constant, Ep(x;θ)…”
On importance sampling Monte Carlo approach to uncertainty analysis for flow and transport in porous media (2003, background): “A detailed mathematical framework for importance sampling is given in [8] and a thorough review on this topic can be found in [14].”
Survey of sampling-based methods for uncertainty and sensitivity analysis (2006, background): “If large sample sizes are required to provide appropriate coverage of low probability/high consequence subsets of values for x, then importance sampling may be a more effective sampling procedure than either random or Latin hypercube sampling [126–134].”
Convergence of Least Squares Temporal Difference Methods Under General Conditions (2010, methodology): “The technique is based on importance sampling , which is widely used in dynamic programming and reinforcement learning contexts [see e.g., (Glynn & Iglehart, 1989; Sutton & Barto, 1998; Precup et al., 2001; Ahamed et al., 2006)].”
Weak Convergence Properties of Constrained Emphatic Temporal-difference Learning with Constant and Slowly Diminishing Stepsize (2015, background): “e behavior of the algorithms in practice. (The problem of growing variances when applying importance sampling to simulate Markov systems was also known earlier and discussed in prior works; see e.g., [14, 38].) The two biased constrained algorithms discussed in this paper were motivated by the need ”