Stochastic search using the natural gradient

By Yi Sun, Daan Wierstra, Tom Schaul, J. Urgen Schmidhuber. 2009

95 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

Revisiting Natural Gradient for Deep Networks (2013, methodology): “The algorithm has also been successfully applied in the reinforcement learning community (Kakade, 2001; Peters and Schaal, 2008) and for stochastic search (Sun et al., 2009).”
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) (2015, methodology): “al gradient was also applied to deep Boltzmann machines Desjardins et al. (2014), reinforcement learning Kakade (2002); Peters et al. (2005); Bhatnagar et al. (2008), and natural evolution strategies Sun et al. (2009); Wierstra et al. (2014). Since for neural networks and deep learning the computati”
Variational Bayesian Inference with Stochastic Search (2012, methodology): “Stochastic search algorithms have also been developed for models of Evolution Strategies (see, e.g., Yi et al. (2009)).”
Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017, methodology): “The version of ES we use in this work belongs to the class of natural evolution strategies (NES) [Wierstra et al., 2008, 2014, Yi et al., 2009, Sun et al., 2009, Glasmachers et al., 2010a,b, Schaul et al., 2011] and is closely related to the work of Sehnke et al. [2010].”
Self-Delimiting Neural Networks (2012, background): “…through a focus on reducing search spaces by co-evolving the comparatively small weight vectors of individual neurons and synapses [19], by Natural Gradient-based Stochastic Search Strategies [80, 73, 74, 48, 17, 79], and by reducing search spaces through weight matrix compression [55, 35].”
A linear time natural evolution strategy for non-separable functions (2011, methodology): “For full Gaussian distribution, the Fisher can be derived analytically [4,16].”
Intrinsically motivated neuroevolution for vision-based reinforcement learning (2011, methodology): “The particular instantiation of UL-ERL in this paper uses a computationally efficient variant of Natural Evolution Strate gies (NES; [5,23,24,26]), called Separable NES or SNES [17] as the evolutionary search method, and a competitive clustering algorithm based on Vector Quantization (VQ; [7]) for ”
Exploring parameter space in reinforcement learning (2010, methodology): “Additional techniques were developed to enhance NES’ performance and viability, including importance mixing to reduce the number of required samples [24, 25], and exponential parametrization of the search distribution to guarantee invariance while at the same time providing an elegant and efficient ”
Exponential natural evolution strategies (2010, methodology): “BOTTLENECKS AND FIXES: EFFICIENT EXACT NES In this section we briefly outline the current state-of-the-art Exact NES (eNES) algorithm as presented in [12].”
Model-based contextual policy search for data-efficient generalization of robot skills (2017, methodology): “While the early PG algorithms were developed for action-based exploration, the ideas have been adopted to parameter-based exploration algorithms [43, 35, 33] as well.”
Efficient natural evolution strategies (2009, methodology): “=1 F k g k(z i) > F k g k(z i) . Here F k denotes the inverse of the k-th diagonal block of F , while Gkand gkdenote the submatrices corresponding to dierentiation w.r.t. k. In a companion paper [7], we empirically investigated the convergence properties when using the various types of baseline. ”