An evolutionary algorithm that constructs recurrent neural networks

By P. Angeline, G. M. Saunders, Jordan B. Pollack. 1994

Subfields: Model Architectures, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Neural Architecture Search & AutoML

Citations in the corpus

AutoGrow: Automatic Layer Growing in Deep Convolutional Networks (2019, neutral): “lain4Net-6-6-6-6 in Table 11. DNNs in Table 11. Even with random initialization in new layers, the accuracy converges stably. 4 RELATED WORK Neural Architecture Search (NAS) [42] and neural evolution [1, 22, 26, 30, 34] can search network architectures from a gigantic search space. In NAS, the depth”
Regularized Evolution for Image Classifier Architecture Search (2018, background): “A traditional approach is neuro-evolution of topologies [1, 32, 41].”
Weight Agnostic Neural Networks (2019, background): “Our work has connections to existing work not only in deep learning, but also to various other ﬁelds: Architecture Search Search algorithms for neural network topologies originated from the ﬁeld of evolutionary computing in the 1990s [1, 8, 17, 25, 33, 40, 53, 59, 70, 71, 81, 86, 116, 117].”
Real-Time Federated Evolutionary Neural Architecture Search (2020, neutral): “Different from the previous neuro-evolution techniques [35], [36] that aim to optimize both the weights and architecture of neural networks, EA based NAS only optimizes the model architecture itself, and the model parameters are trained using conventional gradient descent methods [10]–[13].”
Automated Machine Learning: State-of-The-Art and Open Challenges (2019, neutral): “Many evolutionary approaches based on genetic algorithms are used to optimize the neural networks architecture and weights [117, 116, 2] while others rely on hierarchical evolution [74].”
Efficient Progressive Neural Architecture Search (2018, background): “Optimization of neural network hyperparameters is a problem that has been tackled since the early nineties [1, 3].”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “Popular algorithms include early work of Angeline et al. (1994) and Yao and Liu (1997) to evolve neural networks, Neuroevolution of Augmenting Topologies (NEAT) (Stanley and Miikkulainen, 2002), Analog Genetic Encoding (AGE) (Mattiussi and Flo-reano, 2007), and HyperNEAT (Stanley et al., 2009).”
Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells (2018, background): “Few decades ago, such an algorithm was based on evolutionary programming strategies where best seen so far architectures underwent mutations and their most promising off-springs were bound to continue evolving [2].”
RC-DARTS: Resource Constrained Differentiable Architecture Search (2019, methodology): “EA-based methods Early attemps using evolutionary algorithm to optimize neural architecture includes [1, 27, 8].”
Evolutionary algorithms in control systems engineering: a survey (2002, background): “Angeline, Saunders, and Pollack (1994) attempted the simultaneous evolution of NN structure and weights using EP.”
A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks (2009, background): “, neuroevolution methods), which traditionally evolve either fixed-topology networks [18, 45] or arbitrary random-topology networks [3, 22, 60].”
DATA: Differentiable ArchiTecture Approximation With Distribution Guided Sampling (2020, background): “Historically, evolutionary algorithms have already been heavily investigated and applied in evolving neural architectures [46], [47], [48].”
Sequence Learning (2001, background): “Also Evolutionary Algorithms (Genetic Algorithms) can be used to train recurrent networks [52] [9].”
Neural Architecture Search for Deep Image Prior (2020, methodology): “Genetic algorithms (GAs) were used to optimize both the architecture and weights [1, 33], rather than rely upon backprop, in order to reduce the GA evaluation bottleneck; however, this is not practical for contemporary CNNs.”
Efficient Reinforcement Learning Through Evolving Neural Network Topologies (2002, background): “TWEANNs typically start with an initial population of random topologies (Angeline et al. 1993; Dasgupta and McGregor 1992; Gruau et al. 1996; Zhang and Muhlenbein (1)In rare cases when the fitness of the entire population does not improve for more than 20 generations, only the top two species are al”