Natural Evolution Strategies

By Daan Wierstra, Tom Schaul, Jan Peters, J. Urgen Schmidhuber. 2008

1,034 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Optimization & Training

Citations in the corpus

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) (2015, methodology): “o applied to deep Boltzmann machines Desjardins et al. (2014), reinforcement learning Kakade (2002); Peters et al. (2005); Bhatnagar et al. (2008), and natural evolution strategies Sun et al. (2009); Wierstra et al. (2014). Since for neural networks and deep learning the computation of the Fisher in”
SNAS: Stochastic Neural Architecture Search (2018, methodology): “t;j @L @x m ZT m @O m(x j) @x j ; @L @ k i;j = @L @x j Zk i;j @O i;j(x i) @ i;j ; @L @k i;j = @L @x j OT i;j(x i)((k0 k) Z )Zk i;j 1 k i;j : (6) We name @L @ search gradient similartothe one in Wierstra et al. (2008), even though no policy gradient is involved. This renders SNAS a differentiable ver”
Deep learning in neural networks: An overview (2014, background): “…Strategies (NES) (Glasmachers, Schaul, Sun, Wierstra, & Schmidhuber, 2010; Sun, Gomez, Schaul, & Schmidhuber, 2013; Sun, Wierstra, Schaul, & Schmidhuber, 2009; Wierstra et al., 2008) link policy gradient methods and evolutionary approaches through the concept of Natural Gradients (Amari, 1998).”
Deep Networks with Internal Selective Attention through Feedback Connections (2014, methodology): “Therefore, we instead evolve the policy using a variant for Natural Evolution Strategies (NES; [25, 26]), called Separable NES (SNES; [19]).”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, neutral): “, 2010), or according to the principles of evolutionary computation (Rechenberg, 1971; Gomez et al., 2008; Wierstra et al., 2008).”
Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017, methodology): “The version of ES we use in this work belongs to the class of natural evolution strategies (NES) [Wierstra et al., 2008, 2014, Yi et al., 2009, Sun et al., 2009, Glasmachers et al., 2010a,b, Schaul et al., 2011] and is closely related to the work of Sehnke et al. [2010].”
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning (2017, methodology): “A team from OpenAI recently experimented with a simpliﬁed version of Natural Evolution Strategies (Wierstra et al., 2008), speciﬁcally one that learns the mean of a distribution of parameters, but not its variance.”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, methodology): “…networks tend to sample v from Gaussian distributions (Salimans et al., 2017a; Ilyas et al., 2017) as is done in natural evolution strategies (Wierstra et al., 2008), we sample v from a Rademacher distribution (i.e., Bernoulli ±1), as SPSA requires v−1 to have finite first and second moments…”
Parameter Space Noise for Exploration (2017, background): “Our work is also closely related to evolution strategies (ES) [39, 40], and especially neural evolution strategies (NES) [41, 8, 42–45].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1, except that C is now an FNN or RNN trained by evolutionary algorithms [200, 255, 105, 56, 68] applied to NNs [165, 321, 180, 259, 72, 90, 89, 110, 94], or by policy gradient methods [314, 315, 316, 274, 18, 1, 63, 128, 313, 210, 192, 191, 256, 85, 312, 190, 82, 93][245, Sec.”
Backpropagation through the Void: Optimizing control variates for black-box gradient estimation (2017, background): “Wierstra et al. (2014) also introduce a sampling distribution to build a gradient estimator, and consider optimizing the sampling distribution.”
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam (2018, methodology): “…particular, our algorithm is related to recent approaches in deep learning for exploration to avoid local minima, e.g., natural evolution strategy (Wierstra et al., 2014), entropy-SGD (Chaudhari et al., 2016), and noisy networks for reinforcement learning (Fortunato et al., 2018; Plappert et al.,…”
One Big Net For Everything (2018, background): “Without a teacher, reward-maximizing programs of an RNN must be learned through repeated trial and error, e.g., through artiﬁcial evolution [39, 107, 83, 40, 16, 18, 103, 14, 89, 88] [76, Sec.”
Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions (2019, methodology): “REINFORCE with multiparameter distributions), as well as from subsequent population-based optimization methods including Natural Evolution Strategies (NES) [48] and Parameter-Exploring Policy Gradients (PEPG) [49].”
Deep Exploration via Randomized Value Functions (2017, background): “Policy learning algorithms, and the closely-related ‘evolutionary’ algorithms identify high-performers among a set of policies (Kakade, 2003; Wierstra et al., 2008; Deisenroth et al., 2013; Plappert, 2017).”
Security and Privacy Issues in Deep Learning (2018, methodology): “[Ilyas et al. 2018], more speciﬁc information is assumed to be available, namely, query-limited settings, partial-information settings, and label-only settings. Natural evolutionary strategies (NES) [Wierstra et al. 2008] is used to generate adversarial examples in a query-limited setting. In the pa”
A Survey of Black-Box Adversarial Attacks on Computer Vision Models (2019, methodology): “σ is found using Grid Search and NES [42] is used to find μ:”
Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents (2017, methodology): “Recent work from OpenAI outlines a version of NES applied to standard RL benchmark problems (Salimans et al., 2017).”
Stochastic Gradient Descent: Going As Fast As Possible But Not Faster (2017, methodology): “The ALERA scheme takes inspiration from the famed CMA-ES [12] and NES [27] algorithms, today considered among the best-performing derivative-free continuous optimization algorithms.”