Recent advances in convolutional neural networks

By Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, Gang Wang, Jianfei Cai and 1 more. 2015

6,013 citations recorded by the source corpus.

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging on the rapid growth in the amount of the annotated data and the great improvements in the strengths of graphics processor units, the research on convolutional neural networks has been emerged swiftly and achieved state-of-the-art results on various tasks. In this paper, we provide a broad survey of the recent advances in convolutional neural networks. We detailize the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation. Besides, we also introduce various applications of convolutional neural networks in computer vision, speech and natural language processing.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, background): “Modern technologies based on pattern recognition, machine learning and data-driven artificial intelligence, especially after the advent of deep learning, have reported impressive performance in a variety of application domains, from classical pattern recognition tasks like speech and object recognit”
Data Augmentation Generative Adversarial Networks (2017, background): “They have been demonstrated in variety of domains (Gu et al., 2015) spanning from image classification (Krizhevsky et al.”
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, background): “Recent advances of visual recognition in digital images using deep convolutional networks [24, 57, 61, 13] also inspired the research in video analysis.”
A survey on deep learning in medical image analysis (2017, background): “The goal of this section is to provide a formal introduction and deﬁnition of the deep learning concepts, techniques and architectures that we found in the medical image analysis papers surveyed in this work.”
Artificial Intelligence Meets Flexible Sensors: Emerging Smart Flexible Sensing Systems Driven by Machine Learning and Artificial Synapses (2023, neutral): “2c) [56, 57].”
A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images (2020, neutral): “These kernels convolve an entire input using “stride(s)” so that the dimensions of an output volume become integers [46].”
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization (2020, background): “” Even though it is still an open question why CNNs work so well for various applications [17, 59], there are some commonly accepted “intuitions” of how different layers help this model class succeed.”
End-to-end encrypted traffic classification with one-dimensional convolution neural networks (2017, background): “In recent years, the successful applications of CNN in NLP all used 1D-CNN, for example, sentiment analysis and text classification [21][22].”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “More specifically, the target Q network and E-networks with CNN [97] are used to optimise the stability of policy exploration, and reduce the risk of the policy exploration, ultimately enhancing the wireless security in UAV communication against jamming.”
How Artificial Intelligence Is Shaping Medical Imaging Technology: A Survey of Innovations and Applications (2023, background): “The LeNet-5 architecture, developed for the recognition of handwritten digits [20], is a fundamental milestone for convolutional neural networks (CNNs) [21,22].”
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning (2017, background): “Sec-ondly, there is an exposure bias between the training and the testing (Ranzato et al., 2016; Wiseman and Rush, 2016; Gu et al., 2017a).”
Semantic Segmentation With Context Encoding and Multi-Path Decoding (2020, neutral): “computer vision tasks [25], such as image classification [28],”
Transformative Potential of AI in Healthcare: Definitions, Applications, and Navigating the Ethical Landscape and Public Perspectives (2024, methodology): “Inspired by the visual perception abilities of living organisms, CNNs consist of multiple layers and employ backpropagation algorithms to directly recognize visual patterns without extensive preprocessing [17].”
Large-Scale Generative Data-Free Distillation (2020, background): “Previous studies have observed that different layers of a deep convolutional network are likely to perform different tasks [18, 34, 38], i.e. , lower layers tend to detect low-level features such as edges and curves, while higher layers learn to encode more abstract features.”
Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study (2020, background): “Refer to [147], the feature value at location ( x , y ) in the k -th feature map of M -th layer can be calculated as follows: where X Mx , y is the input patch centered at location ( x , y ) , W Mk is the weight vector of the k -th ﬁlter, and b Mk is bias Algorithm 2 Recurrent neural network 1:…”