Abandoning Objectives: Evolution Through the Search for Novelty Alone

By Joel Lehman, Kenneth O. Stanley. 2011

1,147 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evolutionary Computation

Citations in the corpus

World Models (2018, background): “…at ways to incorporate artiﬁcial curiosity and intrinsic motivation (Schmidhu-ber, 2010; 2006; 1991b; Pathak et al., 2017; Oudeyer et al., 2007) and information seeking (Schmidhuber et al., 1994; Gottlieb et al., 2013) abilities in an agent to encourage novel exploration (Lehman & Stanley, 2011).”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Finally, recent advances in the theory of evolutionary computation (Lehman and Stanley, 2011; Stanley and Lehman, 2015) suggest that is not always the fittest, but at times is the novel individual or behavior that can exploit environmental niches, thus leading to creative evolutionary processes as t”
Diversity is All You Need: Learning Skills without a Reward Function (2018, background): “Prior work in neuroevolution and evolutionary algorithms has studied how complex behaviors can be learned by directly maximizing diversity (Lehman & Stanley, 2011a;b; Woolley & Stanley, 2011; Stanley & Miikkulainen, 2002; Such et al., 2017; Pugh et al., 2016; Mouret & Doncieux, 2009).”
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning (2017, methodology): “Overall the results for the Image Hard Maze domain con-ﬁrm that the Deep GA allows algorithms developed for small-scale neural networks to operate at DNN scale, and can thus be harnessed on hard, high-dimensional problems that require DNNs.”
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities (2018, neutral): “The surprise inspired Stanley to conceive the novelty search algorithm [107], which searches without an explicit objective (just as Stanley found the car unintentionally), selecting instead the most different, novel outcomes at each evolution step.”
Curiosity Driven Exploration of Learned Disentangled Goal Spaces (2018, background): “More involved methods that value states or actions in terms of novelty, information gain, or prediction errors are limited the presence of “distractors” that cannot be controlled [9, 10, 11, 12, 13].”
Large-Scale Study of Curiosity-Driven Learning (2018, background): “Learning without extrinsic rewards or ﬁtness functions has also been studied extensively in the evolutionary computing where it is referred to as ‘novelty search’ [17, 18, 43].”
Unsupervised Curricula for Visual Meta-Reinforcement Learning (2019, background): “A large body of prior work has studied exploration and intrinsic motivation objectives [51, 48, 43, 22, 8, 5, 35, 42].”
Episodic Curiosity through Reachability (2018, background): “Another line of work, Novelty Search (Lehman & Stanley, 2011) and its recent follow-up (Conti et al., 2018), proposed maintaining an archive of behaviours and comparing current behaviour to those — however, the comparison is done by euclidean distance and behaviours are encoded using coordinates,…”
Adaptation of Robot Behaviour through Online Evolution and Neuromodulated Learning (2012, positive): “In this domain, open-ended evolutionary techniques such as novelty search have shown promising results [13, 16] and may complement our method.”
Diverse Agents for Ad-Hoc Cooperation in Hanabi (2019, background): “in its own right or as an intermediate step to high-quality solutions, as showcased by novelty search, which can outperform traditional optimization in deceptive environments even though it abandons the notion of an objective function [23].”
Four concepts for resilience and the implications for the future of resilience engineering (2015, background): “Studies of biological systems [17] and evolutionary computational modeling of biological systems [23,24] have shown that properties that will sustain adaptive capacity in the future can be selected for [4].”
Evolvability Search: Directly Selecting for Evolvability in order to Study and Produce It (2016, methodology): “The second domain, biped locomotion, is adapted from [17].”
Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions (2020, background): “…(e.g. pursuing different kinds of solutions simultaneously) and optimization (e.g. giving one arguably “best” solution) (Brant & Stanley, 2017; Lehman & Stanley, 2011b; Mouret & Clune, 2015; Pugh et al., 2016; Soros & Stanley, 2014), as those solely focusing on optimization often lead to…”
Exploration in Deep Reinforcement Learning: A Survey (2022, methodology): “One of the earliest methods called novelty search was devised in [74] and [75].”