Evolving Deep Convolutional Neural Networks by Variable-Length Particle Swarm Optimization for Image Classification

By Yanan Sun, Bing Xue, Mengjie Zhang. 2018

184 citations recorded by the source corpus.

Convolutional neural networks (CNNs) are one of the most effective deep learning methods to solve image classification problems, but the best architecture of a CNN to solve a specific problem can be extremely complicated and hard to design. This paper focuses on utilising Particle Swarm Optimisation (PSO) to automatically search for the optimal architecture of CNNs without any manual work involved. In order to achieve the goal, three improvements are made based on traditional PSO. First, a novel encoding strategy inspired by computer networks which empowers particle vectors to easily encode CNN layers is proposed; Second, in order to allow the proposed method to learn variable-length CNN architectures, a Disabled layer is designed to hide some dimensions of the particle vector to achieve variable-length particles; Third, since the learning process on large data is slow, partial datasets are randomly picked for the evaluation to dramatically speed it up. The proposed algorithm is examined and compared with 12 existing algorithms including the state-of-art methods on three widely used image classification benchmark datasets. The experimental results show that the proposed algorithm is a strong competitor to the state-of-art algorithms in terms of classification error. This is the first work using PSO for automatically evolving the architectures of CNNs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Xue
2. Mengjie Zhang
3. Yanan Sun
4. Gary G. Yen
5. Yuqiao Liu
6. Amany M. Sarhan
7. M. Arafa
8. Mohamed Reyad

In Computer Vision and Pattern Recognition

1. Bing Xue
2. Mengjie Zhang
3. Yanan Sun
4. Gary G. Yen
5. Yuqiao Liu
6. Amany M. Sarhan
7. M. Arafa
8. Mohamed Reyad

In Neurology

1. Amany M. Sarhan
2. M. Arafa
3. Mohamed Reyad
4. Bing Xue
5. Mengjie Zhang
6. Yanan Sun
7. Gary G. Yen

Citations in the corpus

• A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification (2018, methodology): “The IP-Based Encoding Strategy (IPES) [20] has demonstrated its powerfulness in particle swarm optimisation for evolving deep CNNs, but it has a critical drawback which is that the maximum depth of the CNN architectures has to be set before the commencement of the evolutionary process.”
• A modified Adam algorithm for deep neural network optimization (2023, methodology): “In the CNN model, a convolutional layer often incorporates with the ReLU activation function to be as one layer and then it followed by a pooling layer [56].”
• A Survey on Evolutionary Neural Architecture Search (2020, background): “Note that some works claimed their use of the variable-length encoding strategy, where each individual is designed with a maximal length, and then the placeholder is used to indicate the gene’s validation of the gene [97].”