Designing Neural Networks using Genetic Algorithms

By Geoffrey F. Miller, Peter M. Todd, Shailesh U. Hegde. 1989

Subfields: Neural Architecture Search & AutoML, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Neural Architecture Search & AutoML

In Evolutionary Computation

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “In practical applications, however, current artificial networks of spiking neurons cannot yet compete with the best traditional deep NNs (e.g., compare MNIST results of Section 5.19).”
Automated Machine Learning: State-of-The-Art and Open Challenges (2019, neutral): “[86] considered an evolutionary algorithm to propose the architecture of the neural network and network weights as well.”
Bayesian Learning of Neural Network Architectures (2019, methodology): “Neural network architecture search has long been a topic of research and diverse methods such as evolutionary algorithms [Todd, 1988, Miller et al., 1989, Kitano, 1990], reinforcement learning [Zoph and Le, 2016] or Bayesian optimisation [Bergstra et al., 2013, Mendoza et al., 2016] have been…”
You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimization (2018, methodology): “ Recently, there has been growing interest in developing methods to generate neural network architecture automatically. One heavily investigated direction is evolutionary algorithm (Meyer-Lee et al.; Miller et al. (1989); Real et al. (2017); Stanley & Miikkulainen (2002)). They designed the modi”
Efficient Architecture Search by Network Transformation (2017, methodology): “round and Related Work Automatic Architecture Designing: Neuro-evolution algorithms which mimic the evolution processes in the nature, are one of the earliest automatic architecture designing methods [18, 28]. Recently, authors in [20] showed that neuro-evolution algorithms are capable of constructi”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1, except that C is now an FNN or RNN trained by evolutionary algorithms [200, 255, 105, 56, 68] applied to NNs [165, 321, 180, 259, 72, 90, 89, 110, 94], or by policy gradient methods [314, 315, 316, 274, 18, 1, 63, 128, 313, 210, 192, 191, 256, 85, 312, 190, 82, 93][245, Sec.”
Reinforcement Learning for Architecture Search by Network Transformation (2017, methodology): “Speciﬁcally, the number of ﬁlters is chosen from [6 , 8 , · · · , 32] and the number of units is chosen from [16 , 18 , · · · , 42] .”
One Big Net For Everything (2018, background): “Without a teacher, reward-maximizing programs of an RNN must be learned through repeated trial and error, e.g., through artiﬁcial evolution [39, 107, 83, 40, 16, 18, 103, 14, 89, 88] [76, Sec.”
AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search (2019, background): “Evolutionary Algorithm (EA) : Evolutionary algorithms represent each neural network as a string of genes and search the architecture space by mutation and recombinations [28, 38, 37, 13, 7, 20, 33, 43, 27, 39, 32].”
Hybrid Genetic Algorithms: A Review (2006, methodology): “It can be utilized to adjust the neural network weights [82]-[84] their topology [85]-[88] and learning rules [89], [90].”
Neural networks for the prediction and forecasting of water resource variables: a review of modelling issues and applications (2000, methodology): “Genetic algorithms provide alternatives to evolutionary algorithms and have also been used successfully to determine optimal network architectures (e.g. Miller et al., 1989).”
Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor (1993, background): “[15] have described representat ion schemes in which the anatomical prop erties of the network structure are encoded as bit-strings.”
Improving One-Shot NAS by Suppressing the Posterior Fading (2019, unknown): “Early neural architecture search (NAS) [24, 22, 29, 39, 2, 35] methods normally involves reinforcement learning or neuro-evolution.”
Explorations in Evolutionary Robotics (1993, background): “This view is expressed in Maturana and Varela (1987), Beer (1992), and van Gelder (1992), and will not be developed further here.”
Evolutionary Algorithms for Neural Network Design and Training (1995, background): “The crossover operator in [51] can only exchange whole sets of incoming connections to a unit.”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “RL is also very different from traditional black box optimization (BBO) [36, 57, 19, 9] such as neuroevolution [29, 59, 14, 12] which can be used to solve complex RL problems in partially observable environments [15] through iterative discovery of better and better parameters of an adaptive…”
Inductive Transfer for Neural Architecture Optimization (2019, methodology): “Neuro-evolution is a well-studied method which started with evolving the network parameters (Miller et al., 1989) and was later extended to evolve the architecture as well (Stanley & Miikkulainen, 2002).”