Toward Scalable Systems for Big Data Analytics: A Technology Tutorial

By Han Hu, Yonggang Wen, Tat-seng Chua, Xuelong Li. 2014

1,093 citations recorded by the source corpus.

Recent technological advancements have led to a deluge of data from distinctive domains (e.g., health care and scientific sensors, user-generated data, Internet and financial companies, and supply chain systems) over the past two decades. The term big data was coined to capture the meaning of this emerging trend. In addition to its sheer volume, big data also exhibits other unique characteristics as compared with traditional data. For instance, big data is commonly unstructured and require more real-time analysis. This development calls for new system architectures for data acquisition, transmission, storage, and large-scale data processing mechanisms. In this paper, we present a literature survey and system tutorial for big data analytics platforms, aiming to provide an overall picture for nonexpert readers and instill a do-it-yourself spirit for advanced audiences to customize their own big-data solutions. First, we present the definition of big data and discuss big data challenges. Next, we present a systematic framework to decompose big data systems into four sequential modules, namely data generation, data acquisition, data storage, and data analytics. These four modules form a big data value chain. Following that, we present a detailed survey of numerous approaches and mechanisms from research and industry communities. In addition, we present the prevalent Hadoop framework for addressing big data challenges. Finally, we outline several evaluation benchmarks and potential research directions for big data systems.

Subfields: Computer Networks and Communications, Information Systems, Management Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Management Information Systems

Citations in the corpus

Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, background): “Beyond the aforementioned properties, researchers [18], [20] have identified other characteristics such as: • Big data can be a byproduct or footprint of a digital activity or IoT interplay.”
A comprehensive review of big data analytics throughout product lifecycle to support sustainable smart manufacturing: A framework, challenges and future research directions (2019, background): “ A robust and scalable IT architecture to support various application requirements and optimization tasks for all lifecycle stakeholders (Hu et al., 2014).”
Tourism analytics with massive user-generated content: a case study of Barcelona. (2015, methodology): “Hu et al. (2014) propose a definition for big data analytics based on the software, hardware, and aim of analysis: 'Big data analytics is the process of using analysis algorithms running on powerful supporting platforms to uncover potentials concealed in big data, such as hidden patterns or unknown…”
Understanding big data analytics capabilities in supply chain management: Unravelling the issues, challenges and implications for practice (2017, background): “suitable for scalability, storing data of different types that aggregate batch wise, real-time or in near real-time, and also systems to support fast querying and retrieval of data (Hu et al., 2014). In addition, sharing of data generated from an entity to other parties in a supply chain could have ”
A Survey on Deep Learning in Big Data (2017, background): “Do you need to have some numerical estimate of how strong the fit is into each group? If you answer yes, then you probably should look into a density estimation algorithm [4].”
Big Data Analytics in Operations Management (2018, background): “Hu et al. (2014) present an overview of big data, from its deﬁnition, history to paradigms.”
The use of Big Data Analytics in healthcare (2022, background): “Modern data analytics techniques explore and leverage unique data characteristics even from high-speed data streams and sensor data [15, 16, 31, 55].”
A Survey on Internet of Things From Industrial Market Perspective (2015, background): “Therefore, data collected through multiple sensors need to be fused together [33].”
Review of Internet of Things (IoT) in Electric Power and Energy Systems (2018, background): “Hadoop has a number of advantages including hardware infrastructure scalability, cost efficiency, data type flexibility, and fault tolerance, which makes it a leading candidate for storing, managing, and processing big data [137].”
A Survey on Information and Communication Technologies for Industry 4.0: State-of-the-Art, Taxonomies, Perspectives, and Challenges (2019, background): “These peculiar requirements, challenging—by definition of Big Data—the available technologies, have spun significant innovation over data management techniques and tools in the last two decades, also leveraging Cloud Computing as an enabler for the new distributed paradigms: we refer to [122] for an”
Small business awareness and adoption of state-of-the-art technologies in emerging and developing markets, and lessons from the COVID-19 pandemic (2020, background): “Big data analytics is the technology designed to economically extract value from massive volumes of a wide variety of data by enabling high-velocity capture, discovery, and analysis (Chen, Mao, and Liu 2014; Hu et al. 2014).”
Big data caching for networking: moving from cloud to edge (2016, background): “Indeed the potential offered by big data has spurred great interest from industry, government and academics (see [4] and references therein).”
An integrated big data analytics-enabled transformation model: Application to health care (2018, background): “We identi ﬁ ed three architectural components of big data analytics from its tools and functionalities, namely data aggregation, data analysis, and data interpretation, by reviewing the relevant academic literature (e.g., [4,22]) and technology tutorials (e.g., [23,24]).”
Big data framework for analytics in smart grids (2017, methodology): “[2] was presented; it decomposed big data nto four sequential modules, namely data generation, data acquiition, data storage, and data analytics.”