Solving Non-Markovian Control Tasks with Neuro-Evolution

By Faustino J. Gomez, Risto Miikkulainen. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Evolving Deep Neural Networks (2017, methodology): “eepNEAT, called Coevolution DeepNEAT (CoDeepNEAT), is proposed. „e algorithm behind CoDeepNEAT is inspired mainly by Hierarchical SANE [37] but is also in…uenced by component-evolution approaches ESP [13] and CoSyNE [14]. In CoDeepNEAT, two populations of modules and blueprints are evolved separatel”
Compositional pattern producing networks: A novel abstraction of development (2007, background): “neuroevolution methods, evolved either fixed-topology networks [63, 64], or arbitrary random-topology networks [21, 56, 65], NEAT is the first to begin evolution with a population of small, simple networks and complexify the network topology over generations, leading to increasingly sophisticated be”
Genetic Representation and Evolvability of Modular Neural Controllers (2010, background): “Comparison of performance in a pole-balancing problem [41], [42], [43] revealed that Cellular Encoding was outperformed by direct encodings.”
Natural Evolution Strategies (2008, background): “In order to illustrate the importance and prevalence of this general problem setup, one could point to a diverse set of tasks such as the classic nozzle shape design problem [2], developing an Aibo robot gait [3] or non-Markovian neurocontrol [4].”
Evolving plastic neural networks with novelty search (2010, methodology): “…many previous methods that evolved neural networks, that is, neuroevolution methods, which traditionally evolve either fixed-topology networks (Gomez & Miikkulainen, 1999; Saravanan & Fogel, 1995) or arbitrary random-topology networks (Angeline, Saunders, & Pollack, 1994; Gruau, Whitley, &…”
Modeling systems with internal state using evolino (2005, background): “Q-learning, SARSA) on several difficult learning benchmarks [15, 8].”
Evolution of neural networks using Cartesian Genetic Programming (2010, background): “Symbiotic, Adaptive Neural Evolution (SANE) has successfully been applied to pole balancing task obtaining the desired behaviour within relatively few evaluations [6].”
Developmental Plasticity in Cartesian Genetic Programming based Neural Networks (2016, background): “Method SinglePole DoublePole CNE 352 22100 SANE 302 12600 ESP 289 3800 NEAT 743 3600 CoSyNE 98 954 FCGPANN 21 77 PCGPANN 104 1169 Evolution (SANE)(Moriarty, 1997), Enforced SubPopulation (ESP) (Gomez and Miikkulainen, 1999), Neuro Evolution of Augmenting Topologies (NEAT) (Stanley and Miikkulainen, ”
Spectrum-Diverse Neuroevolution With Unified Neural Models (2017, methodology): “The dynamics used here are the same mentioned by the following papers [4], [15], [47].”
Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes (2008, background): “The components are coevolvedby combining them into networks, and selecting those for reproduction that participated in the best performing networks [36, 16, 17 , 15, 18, 19, 14].”
Evolutionary neural AutoML for deep learning (2019, methodology): “amics [53], and subroutine optimization [52]. The specific coevolutionary mechanism in CoDeepNEAT is inspired by Hierarchical SANE [38] but is also influenced by component-evolution approaches of ESP [20] and CoSyNE [19]. These methods differ from conventional neuroevolution in that they do not evol”
Modelling brain emergent behaviours through coevolution of neural agents (2006, background): “Even the additional random selection of more collaborators, followed by some approaches (Casillas et al., 2001; Gomez & Miikkulainen, 1999), is not always able to improve the performance, especially in the case of more than two coevolved species.”
Evolutionary Computation for Reinforcement Learning (2012, methodology): “The enforced subpopulations (ESP) method (Gomez and Miikkulainen, 1999) eliminates the blueprint population but segregates neurons into subpopulations, one”
Evolving Neural Networks through Augmenting Topologies (2002, methodology): “Neuroevolution (NE), the artificial evolution of neural networks using genetic algorithms , has shown great promise in complex reinforcement learning tasks (Gomez and Miikkulainen, 1999; Gruau et al., 1996; Moriarty and Miikkulainen, 1997; Potter et al., 1995; Whitley et al., 1993).”
A Taxonomy for Artificial Embryogeny (2003, methodology): “For example, pole balancing has been used as a reinforcement learning benchmark for over 30 years [4, 7, 32, 35, 56, 58, 67, 76, 81–84], making it convenient for comparison with other methods.”
A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks (2006, methodology): “The particular instantiation of Evolino in this paper uses the Enforced SubPopulations algorithm (ESP; [18], [19]) to evolve LSTM networks.”