Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
By Laith Alzubaidi, Jinglan Zhang, A. Humaidi, Ayad Al-dujaili, Ye Duan, O. Al-Shamma, José I. Santamaría, M. Fadhel, Muthana Al-Amidie, Laith Farhan. Journal of Big Data, 2021
6,716 citations recorded by the source corpus.
In the last few years, the deep learning (DL) computing paradigm has been deemed the Gold Standard in the machine learning (ML) community. Moreover, it has gradually become the most widely used computational approach in the field of ML, thus achieving outstanding results on several complex cognitive tasks, matching or even beating those provided by human performance. One of the benefits of DL is the ability to learn massive amounts of data. The DL field has grown fast in the last few years and it has been extensively used to successfully address a wide range of traditional applications. More importantly, DL has outperformed well-known ML techniques in many domains, e.g., cybersecurity, natural language processing, bioinformatics, robotics and control, and medical information processing, among many others. Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it. Therefore, in this contribution, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of DL. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of DL and including those enhancements recently added to the field. In particular, this paper outlines the importance of DL, presents the types of DL techniques and networks. It then presents convolutional neural networks (CNNs) which the most utilized DL network type and describes the development of CNNs architectures together with their main features, e.g., starting with the AlexNet network and closing with the High-Resolution network (HR.Net). Finally, we further present the challenges and suggested solutions to help researchers understand the existing research gaps. It is followed by a list of the major DL applications. Computational tools including FPGA, GPU, and CPU are summarized along with a description of their influence on DL. The paper ends with the evolution matrix, benchmark datasets, and summary and conclusion.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/0084f3cb0a1754272151c5268a783f24bf5676a0
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- M. Fadhel
- Thinagaran Perumal
- Norwati Mustapha
- Laith Farhan
- Amany M. Sarhan
- M. Arafa
- S. Cong
- José I. Santamaría
In Computer Vision and Pattern Recognition
- Thinagaran Perumal
- Norwati Mustapha
- Amany M. Sarhan
- M. Arafa
- S. Cong
- Muhammad Ali Shoaib
- Yang Zhou
- Laith Alzubaidi
In Radiology, Nuclear Medicine and Imaging
Citations in the corpus
- Measuring biological age using omics data (2022, methodology): “Deep neural networks are an especially exciting approach that has shown promise in several different fields when applied to large-scale data 200 .”
- Deep Learning with Graph Convolutional Networks: An Overview and Latest Applications in Computational Intelligence (2023, result): “In comparison to older methods, the introduction of CNN has resulted in signifcant advances in the disciplines of image processing and natural language processing, such as machine translation, image recognition, and speech recognition, among others [9].”
- A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU (2023, methodology): “deep learning particularly advantageous for automating feature learning across diverse tasks [6].”
- An advanced deep learning models-based plant disease detection: A review of recent research (2023, methodology): “DL models, such as CNNs and DBNs, have been applied plant lesion identification based on image analysis and classification, providing better accuracy and robustness compared to traditional image processing methods (Sladojevic et al., 2016; Alzubaidi et al., 2021; Dhaka et al., 2021).”
- A modified Adam algorithm for deep neural network optimization (2023, background): “Deep neural network (DNN) is a type of artificial neural network (ANN) with multiple hidden layers between the input and output layers [1].”
- A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications (2023, methodology): “The DL models require massive data volume to display exceptional performance [1], as portrayed in Fig.”
- A review of convolutional neural network architectures and their optimizations (2022, background): “Similarly, there are also some well-known studies which discussed different CNNs algorithms and their applications (LeCun et al. 2010; Srinivas et al. 2016; Alzubaidi et al. 2021).”