Accelerated Neural Evolution through Cooperatively Coevolved Synapses

By Faustino J. Gomez, J. Urgen Schmidhuber, Risto Miikkulainen. 2008

354 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Evolutionary Computation

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, methodology): “Before the popularity of Deep RL methods [54], evolution-based algorithms have been shown to be effective at solving RL tasks [18, 21, 22, 88, 92].”
Deep learning in neural networks: An overview (2014, background): “Co-Synaptic Neuro-Evolution (CoSyNE) does something similar on the level of synapses or weights (Gomez et al., 2008); benefits of this were shown on difficult nonlinear POMDP benchmarks.”
World Models (2018, methodology): “Before the popularity of Deep RL methods (Mnih et al., 2013), evolution-based algorithms have been shown to be effective at solving RL tasks (Stanley & Miikkulainen, 2002; Gomez et al., 2008; Gomez & Schmidhuber, 2005; Gauci & Stanley, 2010; Sehnke et al., 2010; Miikkulainen, 2013).”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, neutral): “, 2010), or according to the principles of evolutionary computation (Rechenberg, 1971; Gomez et al., 2008; Wierstra et al., 2008).”
Learning Simple Algorithms from Examples (2015, background): “Genetic programming (Holland, 1992; Goldberg, 1989; Gomez et al., 2008) also can be considered a form of program induction, but the underlying algorithm relies on random mutations rather than gradient information.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1, except that C is now an FNN or RNN trained by evolutionary algorithms [200, 255, 105, 56, 68] applied to NNs [165, 321, 180, 259, 72, 90, 89, 110, 94], or by policy gradient methods [314, 315, 316, 274, 18, 1, 63, 128, 313, 210, 192, 191, 256, 85, 312, 190, 82, 93][245, Sec.”
Multifactorial Evolutionary Algorithm With Online Transfer Parameter Estimation: MFEA-II (2020, methodology): “In addition, we investigate the practicality of our proposition on increasingly challenging variants of the double-pole balancing controller design task [56].”
Online Evolution of Deep Convolutional Network for Vision-Based Reinforcement Learning (2014, methodology): “The fitness of the controllers use by CoSyNEc is computed by: fc = d− 3m 1000 + vmax 5 − 100c , (2) where d is the distance along the track axis measured from the starting line, vmax is maximum speed, m is the cumulative damage, and c is the sum of squares of the control signal differences, divided ”
Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks (2013, background): “A restriction placed on the controller, also used by [6, 8, 45], is to limit the magnitude of the applied force to always be greater than 1 256 × 10N ; this ensures the controller cannot position the poles to be almost at the point of balance and then do nothing until they fall.”
Investigating recurrent neural network memory structures using neuro-evolution (2019, methodology): “le neuro-evolution (applying evolutionary processes to the development of ANNs) been used in searching the space of feed forward and even convolutional architectures for tasks involving static inputs [4]–[13], less effort has been put into exploring the evolution of recurrent memory structures that ”
Deep Learning and Its Applications in Biomedicine (2018, methodology): “A variant of the BP method called back propagation through time (BPTT) is the standard optimization method for RNNs [25,103], and some alternative methods have also been proposed to speed up the optimization or to extend its capacity [63,104–107].”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “…box optimization (BBO) [36, 57, 19, 9] such as neuroevolution [29, 59, 14, 12] which can be used to solve complex RL problems in partially observable environments [15] through iterative discovery of better and better parameters of an adaptive controller, yielding more and more reward per trial.”
Human action recognition using genetic algorithms and convolutional neural networks (2016, methodology): “The controller and CNN are simultaneously optimized using CoSyNE [63] using the images generated due to the turn and speed predictions of the controller.”
Evolving deep unsupervised convolutional networks for vision-based reinforcement learning (2014, methodology): “The 33 network weights were directly encoded in real-valued genomes and evolved using CoSyNE [9] with population size of 100, a mutation rate of 0 .”
Evolving neural networks in compressed weight space (2010, methodology): “As in [5], single-layer feedforward networks were used for the Markov versions (no memory is needed for the task), and FRNNs for the non(a) Pole balancing”
Evolution of neural networks using Cartesian Genetic Programming (2010, background): “The conventional algorithm has advantages over cooperative coevolution (ESP) as it evolves genotypes at the entire network level rather than neuron level, thus allowing potential global solutions to be found for a predefined network topology and size [5].”
Evolutionary Computation for Reinforcement Learning (2012, methodology): “On several versions of the pole balancing problem, CoSyNE has been shown to outperform various temporal-difference methods and other policy-search approaches, as well as SANE, ESP, and NEAT (Gomez et al, 2006, 2008).”
Neuroevolution (2010, methodology): “For instance in the benchmark task of balancing the inverted pendulum without velocity information (making the problem partially observable), the advanced methods have been shown to find solutions two orders of magnitude faster than value-function based reinforcement-learning methods (measured by nu”