A linear time natural evolution strategy for non-separable functions
By Yi Sun, Faustino J. Gomez, Tom Schaul, J. Urgen Schmidhuber. 2011
36 citations recorded by the source corpus.
We present a novel Natural Evolution Strategy (NES) variant, the Rank-One NES (R1-NES), which uses a low-rank approximation of the search distribution covariance matrix. The algorithm allows computation of the natural gradient with cost linear in the dimensionality of the parameter space, and excels in solving high-dimensional non-separable problems.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- J. Urgen Schmidhuber
- David R Ha
- Alex Graves
- Sergey Levine
- Yoshua Bengio
- Quoc V. Le
- Faustino J. Gomez
- Mohit Bansal
In Signal Processing
- J. Urgen Schmidhuber
- Alex Graves
- Yoshua Bengio
- Hasim Sak
- Franccoise Beaufays
- Santiago Fernández
- Zachary Chase Lipton
- Jinyu Li
Citations in the corpus
- On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1 RL through Direct and Indirect Search in RNN Program Space It is possible to train small RNNs with a few 100 or 1000 weights using evolutionary algorithms [200, 255, 105, 56, 68] to search the space of NN weights [165, 307, 44, 321, 180, 259, 320, 164, 173, 69, 71, 187, 121, 313, 66, 270, 269, 305”
- One Big Net For Everything (2018, background): “Without a teacher, reward-maximizing programs of an RNN must be learned through repeated trial and error, e.g., through artificial evolution [39, 107, 83, 40, 16, 18, 103, 14, 89, 88] [76, Sec.”