Designing neural networks through neuroevolution

By Kenneth O. Stanley, Jeff Clune, Joel Lehman, Risto Miikkulainen. 2019

683 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Neural Architecture Search & AutoML, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Neural Architecture Search & AutoML

In Evolutionary Computation

Citations in the corpus

Primer: Searching for Efficient Transformers for Language Modeling (2021, methodology): “Given a search space of TensorFlow programs, we use evolution [11, 12, 13, 14, 15, 16, 17] to search for models that achieve as low of a validation loss as possible given a fixed amount of training compute.”
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch (2020, neutral): “Reviews provide more thorough background [21, 81, 95].”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “However they have also been applied to learn learning rules [183], optimizers [184], architectures [27], [123] and data augmentation strategies [135] in supervised learning.”
Neural network models and deep learning - a primer for biologists (2019, background): “This evolutionary approach is intuitive and has recently shown promise (Such et al., 2017; Stanley et al., 2019), but it is not usually the most efficient solution.”
A Survey on Evolutionary Neural Architecture Search (2020, methodology): “[28] went through a review of neuroevolution, aiming at revealing the weight optimization rather than the architecture of neural networks.”
DATA: Differentiable ArchiTecture Approximation (2019, neutral): “, evolution-based NAS [13, 18, 26, 41, 43, 44, 45, 47], reinforcement learning-based NAS [2, 3, 21, 42, 56, 59, 60], and gradient-based NAS [11, 34, 35, 46, 53], which has achieved significant successes in a multitude of fields, including image classification [4, 12, 21, 30, 31, 34, 44, 53, 60], sem”
Biological underpinnings for lifelong learning machines (2022, background): “mechanisms have mimicked neurogenesis by adding new neurons as needed 171–174 .”
Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods (2020, methodology): “…Mnih et al. 2016; Ecoffet et al. 2019; Nair et al. 2015; Horgan et al. 2018; Jader-berg et al. 2018; Sutton and Barto 2018) and population-based methods (Salimans et al. 2017; Jaderberg et al. 2017; Such et al. 2017; Conti et al. 2018; Wang et al. 2019; Stanley et al. 2019; Cully et al. 2015).”
Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions (2020, methodology): “CPPNs are trained with the NEAT algorithm (Stanley & Miikkulainen, 2002), a neuroevolution (Stanley et al., 2019) algorithm that learns both the architecture (i.e. the topology) and the weights of CPPNs. Speciﬁcally, in this work, the NEAT-Python library (McIntyre et al.) is used to initialize and…”
Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning (2017, neutral): “Our mutation-based approach is close to neuroevolution approaches (Stanley et al., 2019).”
Learning to Convolve: A Generalized Weight-Tying Approach (2019, background): “In a different line of work Stanley et al. (2019) propose the HyperNEAT framework, which is an evolutionary algorithm for learning structured network weights.”
Effective Diversity in Population-Based Reinforcement Learning (2020, methodology): “Population-based methods are particularly prominent in the Neuroevolution community (Stanley et al., 2019), but have recently been of increased interest in RL (Jung et al.”
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (2019, background): “There are pros and cons to us-ing evolutionary methods [176, 96, 146, 195, 167], and beneﬁts to hybrid approaches that combine evolutionary concepts (searching in the parameter space) with policy-gradient concepts (searching in the action space) [131, 46].”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “We expect in the future to see more research on automatically learning complex disparity estimation architectures and their activation functions, using, for xample, the neuro-evolution theory [154], [155], which will free the need for manual network design.”