Evolving Deep Convolutional Neural Networks for Image Classification

By Yanan Sun, Bing Xue, Mengjie Zhang, Gary G. Yen. 2017

659 citations recorded by the source corpus.

Evolutionary paradigms have been successfully applied to neural network designs for two decades. Unfortunately, these methods cannot scale well to the modern deep neural networks due to the complicated architectures and large quantities of connection weights. In this paper, we propose a new method using genetic algorithms for evolving the architectures and connection weight initialization values of a deep convolutional neural network to address image classification problems. In the proposed algorithm, an efficient variable-length gene encoding strategy is designed to represent the different building blocks and the potentially optimal depth in convolutional neural networks. In addition, a new representation scheme is developed for effectively initializing connection weights of deep convolutional neural networks, which is expected to avoid networks getting stuck into local minimum that is typically a major issue in the backward gradient-based optimization. Furthermore, a novel fitness evaluation method is proposed to speed up the heuristic search with substantially less computational resource. The proposed algorithm is examined and compared with 22 existing algorithms on nine widely used image classification tasks, including the state-of-the-art methods. The experimental results demonstrate the remarkable superiority of the proposed algorithm over the state-of-the-art designs in terms of classification error rate and the number of parameters (weights).

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mengjie Zhang
2. Bing Xue
3. Yanan Sun
4. Gary G. Yen
5. Yaochu Jin
6. Handing Wang
7. Sam Nolen
8. Colin White

Citations in the corpus

• Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor (2020, methodology): “A common way to accelerate the fitness evaluation of CNN is to use a performance predictor [28].”
• Completely Automated CNN Architecture Design Based on Blocks (2020, methodology): “To this end, the exact optimization algorithms (e.g., the gradient-based algorithms) are incapable of or ineffective in solving the architecture optimization problem [7], [8].”
• Evolving convolutional neural networks through grammatical evolution (2019, methodology): “The construction of CNN architectures can be expressed as a hyperparameter optimization problem [1, 2], where layers and layer parameters are all considered variables that can be optimized using search techniques, such as evolutionary algorithms [7, 9, 10].”
• A Survey on Evolutionary Neural Architecture Search (2020, methodology): “are randomly generated in the limited space, and it has been adopted by many methods, such as [9], [21], and [67].”
• Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification (2018, background): “Experimental evidence shows that the reinforcement learning-based designs often require more extensive computational resources than the ones based on the evolutionary algorithms [22].”
• A Study on Encodings for Neural Architecture Search (2020, methodology): “Other work has used intermediate encodings to reduce the complexity of the DAG [16, 4], or added summary statistics to the encoding of feedforward networks [17].”
• NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search (2022, methodology): “In addition, since previous studies using EA for NAS mostly evaluate the performance of offspring through the training and validation procedures [4]–[8], the search cost is catastrophic for ordinary researchers.”