LIKELIHOOD RATIO GRADIENT ESTIMATION : AN OVERVIEW by

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Counterfactual reasoning and learning systems: the example of computational advertising (2012, neutral): “The resulting expression is then identical to the celebrated policy gradient (Aleksandrov et al., 1968; Glynn, 1987; Williams, 1992) which expresses how the accumulated rewards in a reinforcement learning problem are affected by small changes of the parameters of the policy function.”
Reinforcement learning in robotics: A survey (2013, methodology): “A common technique in the statistics and simulation community (Glynn, 1987) to address this problem is to re-use the”
Approximate Gradient Methods in Policy-Space Optimization of Markov Reward Processes (2003, methodology): “In order to construct such an estimate Fy, we start with the gradient formula Hly i [ S p i y Hg i y j [ S HP ij yv j y 2 3 (see Cao and Chen, 1997, Glynn, 1987, or Marbach and Tsitsiklis, 2001, for a derivation) which we rewrite as Hly i [ S p i y Hg i y j [ S P…”
Model learning for robot control: a survey (2011, methodology): “, by resetting the random seed, a trick well-known in simulation optimization (Glynn 1987) which is the most common way to implement PEGASUS for complex systems (Ng and Jordan 2000)].”
Reinforcement learning of motor skills with policy gradients (2008, methodology): “Aleksandrov, Sysoyev, and Shemeneva (1968) and Glynn (1987), or REINFORCE trick (Williams, 1992).”
Policy Gradient in Continuous Time (2006, methodology): “…convergence result contrasts with the usual likelihood ratio method (also called score method) in discrete time (see e.g. (Reiman and Weiss, 1986; Glynn, 1987) or more recently in the reinforcement learning literature (Williams, 1992; Sutton et al., 2000; Baxter and Bartlett, 2001; Marbach and…”
Policy Gradient Methods for Robotics (2006, methodology): “This type of method is guaranteed to converge to the true gradient at the fastest theoretically possible pace of O ( I−1/2 ) where I denotes the number of roll-outs [22] even if the data is generated from a highly stochastic system.”
Monte Carlo and Quasi-Monte Carlo Sampling (2009, methodology): “The two main alternatives are infinitesimal perturbation analysis (IPA – also called pathwise differentiation) [124, 129, 136, 177] and the likelihood ratio method (LR – also called score function method) [124, 137].”
Unsupervised Recurrent Neural Network Grammars (2019, methodology): “9 An estimator for the gradient with respect to Eqφ(z |x)[log pθ(x, z)] is obtained via the score function gradient estimator (Glynn, 1987; Williams, 1992),”
Adversarially Regularized Autoencoders (2017, background): “Policy gradient methods are a natural way to deal with the resulting non-differentiable generator objective when training directly in discrete space (Glynn, 1987; Williams, 1992).”
On a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient (2010, methodology): “REINFORCE has been shown to be moderately efficient in terms of number of samples used [6, 7].”
Online Model Learning Algorithms for Actor-Critic Control (2015, methodology): “by using infinitesimal perturbation analysis (IPA) or likelihood-ratio methods (Aleksandrov et al., 1968; Glynn, 1987).”