Big Data: Tutorial and guidelines on information and process fusion for analytics algorithms with MapReduce

By Sergio Ramírez-Gallego, Alberto Fernández, Salvador Garcia, Min Chen, Francisco Herrera. 2018

132 citations recorded by the source corpus.

Abstract We live in a world were data are generated from a myriad of sources, and it is really cheap to collect and storage such data. However, the real benefit is not related to the data itself, but with the algorithms that are capable of processing such data in a tolerable elapse time, and to extract valuable knowledge from it. Therefore, the use of Big Data Analytics tools provide very significant advantages to both industry and academia. The MapReduce programming framework can be stressed as the main paradigm related with such tools. It is mainly identified by carrying out a distributed execution for the sake of providing a high degree of scalability, together with a fault-tolerant scheme. In every MapReduce algorithm, first local models are learned with a subset of the original data within the so-called Map tasks. Then, the Reduce task is devoted to fuse the partial outputs generated by each Map. The ways of designing such fusion of information/models may have a strong impact in the quality of the final system. In this work, we will enumerate and analyze two alternative methodologies that may be found both in the specialized literature and in standard Machine Learning libraries for Big Data. Our main objective is to provide an introduction of the characteristics of these methodologies, as well as giving some guidelines for the design of novel algorithms in this field of research. Finally, a short experimental study will allow us to contrast the scalability issues for each type of process fusion in MapReduce for Big Data Analytics.

Subfields: Artificial Intelligence, Management Information Systems, Management Science and Operations Research, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In ML Systems & Hardware

Citations in the corpus

Evolutionary Fuzzy Systems for Explainable Artificial Intelligence: Why, When, What for, and Where to? (2019, methodology): “main types [67]: 1) Local approaches (also known as approximate models)”