A survey of the recent architectures of deep convolutional neural networks

By Asifullah Khan, Anabia Sohail, Umme Zahoora, Aqsa Saeed Qureshi. Artificial Intelligence Review, 2019

2,740 citations recorded by the source corpus.

Deep Convolutional Neural Network (CNN) is a special type of Neural Networks, which has shown exemplary performance on several competitions related to Computer Vision and Image Processing. Some of the exciting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, and Speech Recognition. The powerful learning ability of deep CNN is primarily due to the use of multiple feature extraction stages that can automatically learn representations from the data. The availability of a large amount of data and improvement in the hardware technology has accelerated the research in CNNs, and recently interesting deep CNN architectures have been reported. Several inspiring ideas to bring advancements in CNNs have been explored, such as the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. However, the significant improvement in the representational capacity of the deep CNN is achieved through architectural innovations. Notably, the ideas of exploiting spatial and channel information, depth and width of architecture, and multi-path information processing have gained substantial attention. Similarly, the idea of using a block of layers as a structural unit is also gaining popularity. This survey thus focuses on the intrinsic taxonomy present in the recently reported deep CNN architectures and, consequently, classifies the recent innovations in CNN architectures into seven different categories. These seven categories are based on spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. Additionally, the elementary understanding of CNN components, current challenges, and applications of CNN are also provided.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Information Systems

View Paper: https://www.semanticscholar.org/paper/8452941a175c899628c679523da952b18f63e335

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Ling Shao
2. Deng-Ping Fan
3. Luc Van Gool
4. Ping Luo
5. Ming-Ming Cheng
6. Wenhai Wang
7. Dacheng Tao
8. Tong Lu

In Electrical and Electronic Engineering

1. Ling Shao
2. S. Cong
3. Yang Zhou
4. Ping Luo
5. Daiqi Shi

In Information Systems

1. Ling Shao
2. Haijun Lv
3. Ning Chen
4. Yongquan Yang
5. Luca Maria Aiello
6. Ping Luo
7. Bruno Lepri

Citations in the corpus

• A scoping review of transfer learning research on medical image analysis using ImageNet (2020, background): “A more comprehensive description of each model can be found elsewhere [14].”
• A Review on Deep Learning in UAV Remote Sensing (2021, methodology): “This layer is regularly used to solve issues with covariance-shift within feature-maps [Khan et al., 2020].”
• A Survey of Deep Learning-Based Object Detection (2019, background): “To get into this field, we recommend readers refer to [12]–[14] for more details of early methods.”
• Convolutional Neural Networks for Image-Based High-Throughput Plant Phenotyping: A Review (2020, background): “and deep layers are dependent upon shallow layers [18, 19].”
• Object detection using YOLO: challenges, architectural successors, datasets and applications (2022, methodology): “With increase in depth of network by series of convolutions, pooling layers and fully connected layers, it has resulted in 138 M parameters approximately [31].”
• A Survey on Vision--Language--Action Models for Embodied AI (2024, neutral): “Therefore, we mainly summarize approaches for these tasks, but numerous excellent surveys on visual models, ranging from convolutional neural networks (CNNs) [236] to Transformers [237], offer more detailed insights.”
• A review of convolutional neural network architectures and their optimizations (2022, background): “classified various recent CNNs architectures according to the design pattern of processing units (Khan et al. 2020).”
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions (2021, background): “More recent advances can be found in recent survey/review papers [30, 52].”
• A Survey on Deep Learning for Human Mobility (2020, background): “They are effective in computer vision applications such as object recognition [150, 171], image classification and segmentation [56, 107], movement or event recognition [192], and more [99].”
• Emerging opportunities and challenges for the future of reservoir computing (2024, background): “Despite large-scale success of deep learning approaches in producing exciting new possibilities 1 – 7 , such methods generally rely on training big models of neural networks posing severe limitations on their deployment in the most common applications 8 .”
• A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2021, background): “Convolutional neural networks (CNNs or ConvNets) are a class of deep feed-forward neural networks whose main purpose is to extract spatial patterns from visual input signals [20,22].”
• A Survey on ensemble learning under the era of deep learning (2021, background): “The primary reason for this phenomenon is that modern deep neural networks (Rawat and Wang 2017; Liu et al. 2017a; Alam et al. 2020; Khan et al. 2020) usually have parameters ranging from millions to billions which makes the time and space overheads for the training and testing stages of EDL much gr”
• Mapping the NFT revolution: market trends, trade networks, and visual features (2021, background): “[33] Asifullah Khan, Anabia Sohail, Umme Zahoora, and Aqsa Saeed Qureshi.”