Deriving and improving CMA-ES with information geometric trust regions
By Abbas Abdolmaleki, Bob Price, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. 2017
38 citations recorded by the source corpus.
CMA-ES is one of the most popular stochastic search algorithms. It performs favourably in many tasks without the need of extensive parameter tuning. The algorithm has many beneficial properties, including automatic step-size adaptation, efficient covariance updates that incorporates the current samples as well as the evolution path and its invariance properties. Its update rules are composed of well established heuristics where the theoretical foundations of some of these rules are also well understood. In this paper we will fully derive all CMA-ES update rules within the framework of expectation-maximisation-based stochastic search algorithms using information-geometric trust regions. We show that the use of the trust region results in similar updates to CMA-ES for the mean and the covariance matrix while it allows for the derivation of an improved update rule for the step-size. Our new algorithm, Trust-Region Co-variance Matrix Adaptation Evolution Strategy (TR-CMA-ES) is fully derived from first order optimization principles and performs favourably in compare to standard CMA-ES algorithm.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Nicolas Manfred Otto Heess
- Chelsea Finn
- Ruslan Salakhutdinov
- Yoshua Bengio
- J. Urgen Schmidhuber
- Jost Tobias Springenberg
In Management Science and Operations Research
- Nicolas Manfred Otto Heess
- Matthew Botvinick
- Martin A. Riedmiller
- Jost Tobias Springenberg
- Arun Ahuja
- Abbas Abdolmaleki
- Dhruva Tirumala
- Yuval Tassa
Citations in the corpus
- Maximum a Posteriori Policy Optimisation (2018, background): “Using KL constraint in M-step has also been shown effective for stochastic search algorithms (Abdolmaleki et al., 2017).”
- One Pixel Attack for Fooling Deep Neural Networks (2017, background): “In fact, there are a couple of recent developments [60]–[62] in evolutionary strategies and related areas that could further improve”
- V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control (2019, methodology): “…used in the Cross Entropy Method (CEM) (Mannor et al., 2003) and Covariance Matrix Adaptation - Evolutionary Strategy (CMA-ES) (Hansen et al., 1997; Abdolmaleki et al., 2017), and is a special case of the more general feature that any rank-preserving transformation is allowed under this formalism.”