Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES)

By Nikolaus Hansen, Sibylle D. Müller, Petros Koumoutsakos. 2003

1,459 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Optimization & Training, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Optimization & Training

In Evolutionary Computation

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “Related methods include probability distributionbased EAs (Baluja, 1994; Larraanaga & Lozano, 2001; Sałustowicz & Schmidhuber, 1997; Saravanan & Fogel, 1995), Covariance Matrix Estimation Evolution Strategies (CMA-ES) (Hansen, Müller, & Koumoutsakos, 2003; Hansen & Ostermeier, 2001; HeidrichMeisner ”
Evolving Robust Neural Architectures to Defend from Adversarial Attacks (2019, methodology): “Specifically, we use the adversarial samples from two types of attacks (L0 and L∞ attacks) with two optimization algorithms (Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [Hansen et al., 2003] and Differential Evolution (DE) [Storn and Price, 1997]).”
One Pixel Attack for Fooling Deep Neural Networks (2017, neutral): “adaptation evolution strategy (CMA-ES) [49]–[51].”
S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning (2018, background): “To do so, our framework integrates 8 algorithms (A2C, ACKTR, ACER, DQN, DDPG, PPO1, PPO2, TRPO) from Stable-Baselines (Hill et al., 2018) (a fork of OpenAI baselines (Dhariwal et al., 2017)), Augmented Random Search (ARS) (Mania et al., 2018), Covariance Matrix Adaptation Evolutionary Strategy (CMA-”
Machine Learning for Fluid Mechanics (2019, methodology): “The CMA-ES algorithm (Ostermeier et al. 1994; Hansen et al. 2003) is a prominent example of evolution strategies using an adaptive estimation of the covariance matrix of a Gaussian probability distribution, to guide the search for optimal parameters.”
Hierarchical Relative Entropy Policy Search (2014, methodology): “Specifically, we compared to the PIBB algorithm (Stulp and Sigaud, 2013), the CMA-ES algorithm (Hansen et al., 2003) as well as the NES algorithm (Wierstra et al.”
An exhaustive review of the metaheuristic algorithms for search and optimization: taxonomy, applications, and open challenges (2023, neutral): “141 Covariance Matrix Adaptation-Evolution Strategy (CMAES) Hansen et al. (2003)”
Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents (2017, background): “inue decreasing wuntil performance increases, at which point we increase w. While many previous works have adapted exploration pressure online by learning the amount of noise to add to the parameters [25, 26, 28, 34], such approaches rest on the assumption that an increased amount of parameter noise”
Adversarial Robustness Assessment: Why both $L_0$ and $L_\infty$ Attacks Are Necessary (2019, methodology): “The performance of both Differential Evolution (DE) [31] and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [32] are evaluated.”
Evaluating the CMA Evolution Strategy on Multimodal Test Functions (2004, methodology): “We thoroughly define the CMA-ES combining weighted recombination [1] and rank-μ-update of the covariance matrix [3, 4].”
Novel composition test functions for numerical global optimization (2005, methodology): “05 Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) [9]: x0: random position in the search space, expected initial distance to optimum per coordinate Coordist=5 G3 model with PCX crossover (G3-PCX) [10]: population size=150, μ =3, λ =2, replace number=1”
Learning to Predict Without Looking Ahead: World Models Without Forward Prediction (2019, background): “In an effort to emulate nature, evolutionary algorithms where proposed [17, 22, 26, 59, 66].”
Equilibrium optimizer: A novel optimization algorithm (2020, neutral): “Differential evolution [11], evolutionary programming [12] and evolution strategy [13] are other examples of this class.”
Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, methodology): “Some of state-of-the-art methods such as CMA-ES [79] are applied to solve kinds of sophisticated optimization problems.”
Towards High Confidence Off-Policy Reinforcement Learning for Clinical Applications (2018, methodology): “In this work, we use CMA-ES (Hansen et al., 2003) to find suitable parameters for the policy function.”
Hippopotamus optimization algorithm: a novel nature-inspired optimization algorithm (2024, methodology): “Regarding competitor algorithms, WOA, GWO, SSA , PSO, SCA, FA, GOA, CMA-ES, SSA, MFO, and IWO have a time complexity equal to O ( N m ( 1 + T )) and TLBO and AOA have a computational complexity equal to O ( N m ( 1 + 2 T )) .”
Irrigation scheduling of a classical gravity network based on the Covariance Matrix Adaptation - Evolutionary Strategy algorithm (2014, methodology): “The Covariance Matrix Adaptation – Evolution Strategy (CMA-ES) algorithm (Hansen et al., 2003) is used to perform the optimization of such problem.”
How to Maximize the Likelihood Function for a DSGE Model (2010, positive): “The third term has rank min(n;�), and this part of the update is thus based on information of higher rank than 1. For further details and interpretations, we refer to Hansen et al. (2003) , Muller, Hansen & Koumoutsakos (2002), and Hansen (2005).”
Bio-inspired computation: Where we stand and what's next (2019, methodology): “Furthermore, other sophisticated bioinspired approaches have shown an ability to perform global and local search simultaneously; this is the case of CMA-ES when configured with a highly adaptive step size [263], or the more recent HCLPSO approach [184], which defines two subpopulations (with paramet”