A survey of machine learning for big data processing

By Junfei Qiu, Qi-hui Wu, Guoru Ding, Yuhua Xu, Shuo Feng. 2016

846 citations recorded by the source corpus.

There is no doubt that big data are now rapidly expanding in all science and engineering domains. While the potential of these massive data is undoubtedly significant, fully making sense of them requires new ways of thinking and novel learning techniques to address the various challenges. In this paper, we present a literature survey of the latest advances in researches on machine learning for big data processing. First, we review the machine learning techniques and highlight some promising learning methods in recent studies, such as representation learning, deep learning, distributed and parallel learning, transfer learning, active learning, and kernel-based learning. Next, we focus on the analysis and discussions about the challenges and possible solutions of machine learning for big data. Following that, we investigate the close connections of machine learning with signal processing techniques for big data processing. Finally, we outline several open issues and research trends.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, methodology): “[15] reviewed several traditional machine learning techniques along with several advanced techniques including DL for processing general big data.”
A Big Data Analytics Method for Tourist Behaviour Analysis (2017, positive): “[58]) to improve predictive capabilities and identification accuracy of tourism demand forecasting.”
Social media big data analytics: A survey (2019, neutral): “Perhaps, (Bello-Orgaz, Jung, & Camacho, 2016; L'Heureux, Grolinger, ElYamany, & Capretz, 2017; Qiu, Wu, Ding, Xu, & Feng, 2016) o ﬀ ered the most relevant related work.”
Big Data technologies: A survey (2017, neutral): “Qiu et al. (2016) gives a brief review of conventional and advanced machine learning algorithms.”
Influence of Data Splitting on Performance of Machine Learning Models in Prediction of Shear Strength of Soil (2021, background): “emain advantage of ML is that it can subjectively analyze unlimited amounts of data and give reliable outcomes and assessment [69].”
Contour Stella Image and Deep Learning for Signal Recognition in the Physical Layer (2020, background): “W ITH the growing diversity and complexity of wireless communication systems, the wireless big data era has arrived [1], [2].”
A Survey on Distributed Machine Learning (2019, neutral): “In contrast to prior surveys on distributed machine learning ([120][124]) or related � elds ([153][87][123][122][171][144]) we apply a wholistic view to the problem and discuss the practical aspects of state-of-the-art machine learning from a distributed systems angle.”
Smart Anomaly Detection in Sensor Systems (2020, background): “This notwithstanding, such methods still need to be refined and improved to tackle the vast amount of data produced by sensor systems [6].”
A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security (2018, methodology): “Several classical ML methods were studied [30] along with advanced methods containing DL for handling common big data.”
The role of big data analytics in Internet of Things (2017, background): “R I P Deep learning [57], machine learning [58], and artiﬁcial intelligence [59] are key technologies that provide value added applications using IoT and big data.”
Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks (2019, background): “by the big data processing capability of ML [17].”
Monitoring Misuse for Accountable 'Artificial Intelligence as a Service' (2020, background): “ML works to uncover patterns in data, to build and refine representative models of that data [28].”
A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends (2019, background): “…and challenges [265] Deep Learning for Natural Language Processing [266] Recommender A survey of machine learning for big data processing [276] Deep learning in big data Analytics: A comparative study [277] Deep learning applications and challenges in big data analytics [278] This review…”
A comprehensive survey on recent metaheuristics for feature selection (2022, background): “This rapid increase in data sizes brought by Big Data has become an important issue for recent machine learning algorithms [1, 2].”