Stochastic Optimization

By Anton J. Kleywegt, A. August. 2003

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reward-estimation variance elimination in sequential decision processes (2018, methodology): “Policy gradient (PG) methods [1, 5, 10, 12, 16], a class of RL algorithms, do oﬀer convergence guaranties in principle, as they directly optimize the expected reward via gradient ascent.”
A Simple, Fast Diverse Decoding Algorithm for Neural Generation (2016, methodology): “…method, to ﬁnd the optimal diversity rate policy by maximizing the expectation of the ﬁnal reward, denoted as follows: The expectation is approximated by sampling from π and the gradient is computed based on the likelihood ratio (Glynn, 1987; VM et al., 1968): ) where b denotes the baseline value.”
Understanding Neural Networks through Representation Erasure (2016, methodology): “The gradient of (5) is approximated using the likelihood ratio trick (Williams, 1992; Glynn, 1990; Aleksandrov et al., 1968), in which for a given e, we sample a sequence of decisions based on π, compute the associated reward and backward propagate gradients to update π, which can be summarized as f”
GPU accelerated Monte Carlo simulation of the 2D and 3D Ising model (2009, methodology): “, molecular dynamics simulations [12–15] or stochastic optimization [16], need a large amount of high performance computing resources.”
Online Model Learning Algorithms for Actor-Critic Control (2015, methodology): “by using infinitesimal perturbation analysis (IPA) or likelihood-ratio methods (Aleksandrov et al., 1968; Glynn, 1987).”
Simulation optimization: a review of algorithms and applications (2014, background): “A recent review of stochastic approximation methods is available in [188], and an excellent tutorial and review of results in stochastic approximation is presented in [154].”
Policy Gradient Methods for Robotics (2006, unknown): “[22], [23]) or REINFORCE [24] trick, i.”
Adversarial Learning for Neural Dialogue Generation (2017, methodology): “The gradient of (1) is approximated using the likelihood ratio trick (Williams, 1992; Glynn, 1990; Aleksandrov et al., 1968): where π denotes the probability of the generated responses. b ( { x, y } ) denotes the baseline value to reduce the variance of the estimate while keeping it unbiased.”
Deep Reinforcement Learning for Dialogue Generation (2016, methodology): “We use the likelihood ratio trick (Williams, 1992; Glynn, 1990; Aleksandrov et al., 1968) for gradient updates: ∇JRL(θ) ≈ ∑ i ∇ log p(ai|pi, qi) i=T∑ i=1 R(ai, [pi, qi]) (9) We refer readers to Williams (1992) and Glynn (1990) for more details.”