Co-evolving recurrent neurons learn deep memory POMDPs

By Faustino J. Gomez, J. Urgen Schmidhuber. 2005

77 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

In Model Architectures

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, methodology): “Before the popularity of Deep RL methods [54], evolution-based algorithms have been shown to be effective at solving RL tasks [18, 21, 22, 88, 92].”
Deep learning in neural networks: An overview (2014, background): “In general, however, POMDPs may imply greatly increased CAP depth.”
World Models (2018, methodology): “Before the popularity of Deep RL methods (Mnih et al., 2013), evolution-based algorithms have been shown to be effective at solving RL tasks (Stanley & Miikkulainen, 2002; Gomez et al., 2008; Gomez & Schmidhuber, 2005; Gauci & Stanley, 2010; Sehnke et al., 2010; Miikkulainen, 2013).”
Frame Skip Is a Powerful Parameter for Learning to Play Atari (2015, background): “…is a variant of Enforced Sub-populations (ESP) (Gomez and Miikkulainen 1997), a neu-roevolution approach that has been successfully imple-Copyright mented and extended to train agents for a variety of complex behaviors and control tasks (Gomez and Schmidhuber 2005; Schmidhuber et al. 2007, e.g.).”
Training Recurrent Networks by Evolino (2007, background): “This approach can quickly learn t o solve difficult reinforcement learning control tasks [5,6,22], including ones that require use of deep memory [7].”
Evolino for recurrent support vector machines (2005, methodology): “ed SubPopulations (ESP; [26]) to evolve Long ShortTerm Memory (LSTM; [21]) networks. We combine these two particular methods because both have routinely outperformed previous methods in their domains [27, 28, 21, 29, 30, 31, 32, 33, 34]. ESP differs from standard neuroevolution methods in that, inst”
Evolutionary Computation for Reinforcement Learning (2012, methodology): “…context, this internal state can record aspects of the agent's observation history , which can help it cope with partial observability (Wieland, 1991; Gomez and Miikkulainen, 1999; Moriarty et al, 1999; Stanley and Miikkulainen, 2002; Igel, 2003; Gomez and Schmidhuber, 2005a,b).”
Policy Gradient Critics (2007, background): “[5]), still hold a competitive edge over PGAC.”
Evolving Modular Fast-Weight Networks for Control (2005, methodology): “Figure 2 shows the basic operation of the algorithm (see [6] for further details).”
Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes (2008, methodology): “To improve our previous implementations of these in gredients (Section 3), we will (1) study better adaptive compressors, in particula r, recent, novel RNNs [94] and other general but practically feasible methods for maki ng predictions [75]; (2) investigate under which conditions learning progress ”