Evolving neural network controllers for unstable systems

By A. Wieland. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “Many RNN evolvers have been proposed (e.g., Miller et al., 1989; Wieland, 1991; Cliff et al., 1993; Yao, 1993; Nolfi et al., 1994a; Sims, 1994; Yamauchi and Beer, 1994; Miglino et al., 1995; Moriarty, 1997; Pasemann et al., 1999; Juang, 2004).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1 RL through Direct and Indirect Search in RNN Program Space It is possible to train small RNNs with a few 100 or 1000 weights using evolutionary algorithms [200, 255, 105, 56, 68] to search the space of NN weights [165, 307, 44, 321, 180, 259, 320, 164, 173, 69, 71, 187, 121, 313, 66, 270, 269, 305”
Solving Non-Markovian Control Tasks with Neuro-Evolution (1999, methodology): “Other approaches that have applied shaping to the easier double pole task (with velocities) have incremented the short pole by only 1%[Wieland, 1991; Saravanan and Fogel, 1995].”
Policy Gradient Critics (2007, background): “This yields non-Markovian pole balancing [12], a more challenging task.”
Natural Evolution Strategies (2008, methodology): “It provides a perfect testbed for algorithms focusing on learning fine control with memory in continuous state and action spaces (Wieland, 1991).”
Evolutionary Computation for Reinforcement Learning (2012, background): “…context, this internal state can record aspects of the agent's observation history , which can help it cope with partial observability (Wieland, 1991; Gomez and Miikkulainen, 1999; Moriarty et al, 1999; Stanley and Miikkulainen, 2002; Igel, 2003; Gomez and Schmidhuber, 2005a,b).”
Robust non-linear control through neuroevolution (2003, background): “These changes to the environment are significant: other approaches that have applied shaping to the easier double pole task (with velocities) have incremented the short pole by only 1% (Wieland 1991; Saravanan and Fogel 1995).”
Co-evolving recurrent neurons learn deep memory POMDPs (2005, background): “To make it more challenging, the basic task has been extended in various ways such as providing the controller with only the position of the cart, x, and the angle of the pole, θ, and not their respective velocities (ẋ, θ̇), as is normally the case [13].”
Evolving Artiicial Neural Networks (2007, methodology): “and multimodal spaces, which are the typical case in the real world, considerable research and application has been conducted on the evolution of connection weights [24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58”
Recurrent policy gradients (2010, unknown): “This yields non-Markovian double pole balancing [28], a truly challenging task that has not been solved by any other single-agent RL method but RPGs.”
Solving Deep Memory POMDPs with Recurrent Policy Gradients (2007, methodology): “This yields non-Markovian double pole balancing [21], a truly challenging task that has not been solved by any other single agent RL method but RPGs.”
Evolving neural network controllers (1995, methodology): “While not for robot control, Wieland [ 14 ] used a genetic algorithm to create recurrent neural networks to control a series of unstable systems.”
Real-time neuroevolution in the NERO video game (2005, background): “While some evolve only the connection weight values of fixed-topology networks [37], [47]–[49], others evolve both weights and network topology simultaneously [38], [50]–[62].”
Evolving artificial neural networks (1999, methodology): “Some of the early work in evolving ANN connection weights followed this approach [24], [26], [28], [37], [38], [41], [52], [53].”
Incremental Evolution of Complex General Behavior (1997, methodology): “In order to apply this method to evolving complex general behavior, two extensions are necessary: In ESP, as in SANE, the population consists of individual neurons instead of full networks.”
2-D Pole Balancing with Recurrent Evolutionary Networks (1998, background): “The equations of motion for this system are an extension of those found in [7] for the single pole problem with one equation for each principal axis.”
Evolutionary Bilevel Optimization for Complex Control Tasks (2015, background): “Pole balancing, or inverted pendulum, has long been a standard benchmark for control algorithms [37, 38, 3, 7].”
Evolving Neural Networks through Augmenting Topologies (2002, methodology): “A parallel can be drawn between structure evolution in NEAT and incremental evolution (Gomez and Miikkulainen, 1997; Wieland, 1991).”
A Taxonomy for Artificial Embryogeny (2003, methodology): “For example, pole balancing has been used as a reinforcement learning benchmark for over 30 years [4, 7, 32, 35, 56, 58, 67, 76, 81–84], making it convenient for comparison with other methods.”