Towards designing artificial neural networks by evolution

By Xin Yao, Y. Liu. 1998

Designing artificial neural networks (ANNs) for different applications has been a key issue in the ANN field. At present, ANN design still relies heavily on human experts who have sufficient knowledge about ANNs and the problem to be solved. As ANN complexity increases, designing ANNs manually becomes more difficult and unmanageable. Simulated evolution offers a promising approach to tackle this problem. This paper describes an evolutionary approach to design ANNs. The ANNs designed by the evolutionary process are referred to as evolutionary ANNs (EANNs). They represent a special class of ANNs in which evolution is another fundamental form of adaptation in addition to learning (also known as weight training). This paper describes an evolutionary programming (EP) based system to evolve both architectures and connection weights (including biases) of ANNs. Five mutation operators have been proposed in our evolutionary algorithm. In order to improve the generalisation ability of evolved ANNs, these five operators are applied sequentially and selectively. Validation sets have also been used in the evolutionary process in order to improve generalisation further. The evolutionary algorithm allows ANNs to grow as well as shrink during the evolutionary process. It incorporates the weight learning process as part of its mutation process. The whole EANN system can be regarded as a hybrid evolution and learning system. Extensive experimental studies have been carried out to test this EANN system. This paper gives some of the experimental results which show the effectiveness of the system.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Weight Agnostic Neural Networks (2019, background): “Our work has connections to existing work not only in deep learning, but also to various other ﬁelds: Architecture Search Search algorithms for neural network topologies originated from the ﬁeld of evolutionary computing in the 1990s [1, 8, 17, 25, 33, 40, 53, 59, 70, 71, 81, 86, 116, 117].”
A new evolutionary system for evolving artificial neural networks (1997, methodology): “It was also tested on the two-spiral problem [16].”
A new strategy for adaptively constructing multilayer feedforward neural networks (2003, methodology): “In [36] the authors use a GA whose populations is a set of neural networks that have been initialized to diJerent number of hidden layers and where a pruning method is also invoked to reduce the architecture of the oversized networks.”
Evolution of neural networks using Cartesian Genetic Programming (2010, background): “Direct encoding are generally thought of as better at fine tuning and generating compact architecture [1], [12], [25], [3], [19], [26], [14].”
Compositional Pattern Producing Networks : A Novel Abstraction of Development (2007, background): “It is fortuitous that CPPNs and ANNs are virtually the same from a structural perspective because many methods already exist for evolving ANN topology and weights [5, 32, 58, 61, 73, 83, 84].”
Compositional pattern producing networks: A novel abstraction of development (2007, methodology): “methods already exist for evolving ANN topology and weights [5, 32, 58, 61, 73, 83 , 84].”
Evolving Neural Networks through Augmenting Topologies (2002, background): “…appear in the phenotype (Angeline et al., 1993; Braun and Weisbrod, 1993; Dasgupta and McGregor, 1992; Fullmer and Miikkulainen, 1992; Krishnan and Ciesielski, 1994; Lee and Kim, 1996; Maniezzo, 1994; Opitz and Shavlik, 1997; Pujol and Poli, 1998; Yao and Liu, 1996; Zhang and M ¨ uhlenbein, 1993).”
Evolving Artiicial Neural Networks (2007, background): “Very compact ANNs with good generalization ability have been evolved [185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195].”
G-Prop-III: Global Optimization of Multilayer Perceptrons using an Evolutionary Algorithm (1999, methodology): “[19] Xin Yao and Yong Liu.”
Metaheuristic design of feedforward neural networks: A review of two decades of research (2017, background): “Yao and Liu [134] identified evolution at various components of FNN that fall into the spectrum of metaheuristic design of FNN shown in Fig.”
Meta learning evolutionary artificial neural networks (2004, background): “The interest in evolutionary search procedures for designing ANN architecture has been growing in recent years as they can evolve towards the optimal architecture without outside interference, thus eliminating the tedious trial and error work of manually finding an optimal network [2] [6] [7] [14] [”