Optimizing deep learning hyper-parameters through an evolutionary algorithm
By Steven R. Young, Derek C. Rose, T. Karnowski, Seung-Hwan Lim, Robert M. Patton. 2015
435 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dacheng Tao
- Quoc V. Le
- Sergey Levine
- Yoshua Bengio
- Trevor Darrell
- Ruslan Salakhutdinov
- Mohit Bansal
- Pieter Abbeel
In Signal Processing
- Karen Simonyan
- Dacheng Tao
- Ali Razavi
- Oriol Vinyals
- Jeff Donahue
- Wojciech M. Czarnecki
- Chang Xu
- Koray Kavukcuoglu
Citations in the corpus
- Population Based Training of Neural Networks (2017, methodology): “The same problem is also encountered where genetic algorithms are used in place of Bayesian optimisation for hyperparameter evolution (Young et al., 2015).”
- Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “work to reduce the input dimensionality. The study does not use evolution on the deep preprocessing networks itself, but demonstrates nevertheless the evolutionary design of a deep neural controller. Young et al. (2015) used an evolutionary algorithm to optimize two parameters of a deep network: the”
- Automated Machine Learning: From Principles to Practices (2018, methodology): “…learning configurations, the iterative differentiation methods first iteratively update the model parameter for n steps to obtain 𝑤 𝑛 ( 𝛼 ) for the optimization of lower level objective, and then uses the updated result 𝑤 𝑛 ( 𝛼 ) to approximate ∗ ( 𝛼 ) to obtain the approximated gradient.”
- Machine learning and applications in ultrafast photonics (2020, background): “One can then continuously increase the volume of training data until the validation error stagnates.”
- Evolving Deep Networks Using HPC (2017, methodology): “Gradient free approaches include grid search, random search [6, 8], coordinate descent [15], Bayesian methods [7, 23], and evolutionary algorithms [11, 18, 20, 24, 27, 28].”
- Neural networks designing neural networks: Multi-objective hyper-parameter optimization (2016, background): “This relies on human intuition and expert knowledge of the target application in conjunction with extensive trial and error [7, 22].”
- Evolutionary Generative Adversarial Networks (2018, positive): “architectures through an evolutionary search [57], [58].”