How novelty search escapes the deceptive trap of learning to learn

By Sebastian Risi, Sandy D. Vanderbleek, C. Hughes, Kenneth O. Stanley. 2009

95 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “In Risi et al. (2009, 2010), EPANN-controlled simulated robots, evolved in a discrete T-Maze domain, reveal that the evolutionary stepping stones towards adaptive behaviors are often not rewarded by objective-based performance measures.”
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (2019, background): “Another type of learner network includes neuromodulation, wherein the output of one neuron in a network can change the learning rates of other connections in the network [40, 187, 125, 161, 141, 160].”
Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks (2017, background): “First, in a “learning-to-learn” approach (Thrun & Pratt, 1997; Risi et al., 2009; Finn et al., 2017, a.o.), a network can be exposed to a number of different learning environments regulated by similar rules.”
Single-unit pattern generators for quadruped locomotion (2013, background): “Previous work has shown that novelty search [21], which alters the traditional fitness function to search for novelty rather than optimality, is effective for biped locomotion because it avoids deception [27].”
Minimal criterion coevolution: a new approach to open-ended search (2017, background): “It is important to note that this kind of genetics-based speciation does not require characterizing behavior or maintaining an archive [8, 10, 24] or map [17], and therefore sustains the unique nature of MCC as a QD algorithm without a behavior characterization.”
Artificial Evolution of Plastic Neural Networks: A Few Key Concepts (2014, methodology): “Notable setups in which authors evolved plastic neurocontrollers for behavioral changes are the T-maze [15], [17], [18], the bumblebee foraging task [21], the “dangerous forag-”
Evolving Neural Turing Machines for Reward-based Learning (2016, methodology): “Especially tasks involving cognitive behaviors and adaptation have shown to be very deceptive for objective-based search methods [7, 9, 12] and benefit from more explorative search methods such as novelty search [8].”
Robot coverage control by evolved neuromodulation (2013, background): “A number of these studies focus on neuromodulatory subsystems with projections that have diffuse action on the synapses of more classical neurons [36], [31], [34], [29].”
Exploration in Deep Reinforcement Learning: A Survey (2022, methodology): “[75] measure of policies distance NEAT Evolution strategies T-maze: solved after 50,000 evaluations enumarated state id Ac MF D/ D”
Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills (2015, methodology): “This finding is in line with previous experiments that have showed that diversity is especially necessary for problems that involve learning, because learning problems are especially laden with local optima [74].”
Indirectly Encoding Neural Plasticity as a Pattern of Local Rules (2010, background): “Additionally, synaptic plasticity could be controlled by neuromodulation [2, 3, 19], which means that some neurons can enhance or dampen the neural plasticity of their target nodes.”
Novelty-Based Multiobjectivization (2011, unknown): “Two recent papers [14, 21] introduced a new but radical approach to evolve robot controllers: they propose to maximize the novelty of behaviors instead of the eﬃciency (the ﬁtness) of these behaviors.”
Abandoning Objectives: Evolution Through the Search for Novelty Alone (2011, methodology): “In fact, since novelty search was ﬁrst introduced by Lehman and Stanley (2008), several works have built upon it (Lehman and Stanley, 2010a; Mouret, 2009; Risi et al., 2009), some of which do not use NEAT or even neuroevolution.”
Picbreeder: A Case Study in Collaborative Evolutionary Exploration of Design Space (2011, background): “…over the course of evolution, NEAT was able to solve several difﬁcult control problems (Aaltonen et al., 2009; Stanley et al., 2005a,b; Taylor et al., 2006; Risi et al., 2009; Stanley and Miikkulainen, 2002a; Trujillo et al., 2008; Kohl et al., 2006; Stanley and Miikkulainen, 2004b,a, 2002b).”
Autonomous Evolution of Topographic Regularities in Artificial Neural Networks (2010, background): “…(Hubel, Wiesel, & LeVay, 1977), ANNs with plastic synapses have been evolved in the past (Floreano & Urzelai 2000, 2001; Floreano & Mondada, 1996; Risi et al., 2009; Soltoggio et al., 2008), and in principle, HyperNEAT can potentially evolve the geometry of the learning rules , taking it another…”
Quality Diversity: A New Frontier for Evolutionary Computation (2016, background): “…studies were followed by a wave of interest in divergent search algorithms as researchers explored their potential for discovery and open-endedness (Risi et al., 2009, 2011; Soltoggio and Jones, 2009; Doucette and Heywood, 2010; Goldsby and Cheng, 2010; Graening et al., 2010; Krcah, 2010;…”
Using a map-based encoding to evolve plastic neural networks (2011, methodology): “These results are similar to the ones obtained by Risi [29] and Soltoggio [7].”
Spectrum-Diverse Neuroevolution With Unified Neural Models (2017, background): “The results obtained from using such mechanisms are much more encouraging [35], [36], [37], [38], [39].”
Evolvability Search: Directly Selecting for Evolvability in order to Study and Produce It (2016, background): “Similar to other divergent EAs [7,25,26,31], it also increases population diversity and evolves solutions to deceptive problems.”