Deep learning applications and challenges in big data analytics

By Maryam Mousaarab Najafabadi, Flavio Villanustre, Taghi M. Khoshgoftaar, Naeem Seliya, Randall Wald, Edin A. Muharemagic. 2015

2,363 citations recorded by the source corpus.

Big Data Analytics and Deep Learning are two high-focus of data science. Big Data has become important as many organizations both public and private have been collecting massive amounts of domain-specific information, which can contain useful information about problems such as national intelligence, cyber security, fraud detection, marketing, and medical informatics. Companies such as Google and Microsoft are analyzing large volumes of data for business analysis and decisions, impacting existing and future technology. Deep Learning algorithms extract high-level, complex abstractions as data representations through a hierarchical learning process. Complex abstractions are learnt at a given level based on relatively simpler abstractions formulated in the preceding level in the hierarchy. A key benefit of Deep Learning is the analysis and learning of massive amounts of unsupervised data, making it a valuable tool for Big Data Analytics where raw data is largely unlabeled and un-categorized. In the present study, we explore how Deep Learning can be utilized for addressing some important problems in Big Data Analytics, including extracting complex patterns from massive volumes of data, semantic indexing, data tagging, fast information retrieval, and simplifying discriminative tasks. We also investigate some aspects of Deep Learning research that need further exploration to incorporate specific challenges introduced by Big Data Analytics, including streaming data, high-dimensional data, scalability of models, and distributed computing. We conclude by presenting insights into relevant future works by posing some questions, including defining data sampling criteria, domain adaptation modeling, defining criteria for obtaining useful data abstractions, improving semantic indexing, semi-supervised learning, and active learning.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Signal Processing

Citations in the corpus

Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, background): “Data validation and trustworthiness should be checked at each level of big data analytics, especially when we are dealing with online streams of input data to an analytic engine [216].”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “self driving cars [12], [180], surveillance [13], maleware detection [34], [107], drones and robotics [155], [178], and voice command recognition [7].”
Demystifying Parallel and Distributed Deep Learning (2018, methodology): “e measures on the approaches, we analyze their concurrency and average parallelism using the Work-Depth model[21]. 1.1 Related Surveys Other surveys in the field focus on applications of deep learning[175], neural networks and their history [143 ,154 209 236 ], scaling up deep learning 17 , and hard”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “Najafabadi [29] An overview of deep learning applications for big data analytics.”
Deep learning for smart fish farming: applications, opportunities and challenges (2020, background): “…data abstractions in DL provides a certain degree of simplification for big data analytics tasks, especially when analyzing massive volumes of data, performing data tagging, information retrieval, or conducting discriminative tasks such as classification and prediction (Najafabadi et al., 2015).”
Digital Twin: Enabling Technologies, Challenges and Open Research (2020, background): “The identification and highlighting of these topics will help in analysing other papers and seeing where this research fits in [63].”
Uncertainty in big data analytics: survey, opportunities, and challenges (2019, background): “, separate variations within an image, such as a light, various materials, and shapes) [56], however current deep learning models incur a high computational cost.”
A Survey of Deep Learning Techniques for Weed Detection from Images (2021, positive): “Image Enhancement and Denoising Nkemelu et al. (2018) investigated the importance of image pre-processing operation by training the CNN model with raw data and processed data.”
Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions (2023, background): “Deep neural networks are e ﬀ ective in hybrid learning as well as supervised, unsupervised, and reinforcement learning [57]. than m and q must be equal to or less than r.”