MapReduce: a flexible data processing tool

By Joseph Dean, Sanjay Ghemawat. CACM, 2010

1,429 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/7339c20b5c5ab96260beb5162d38bcd8258461df

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Rubao Lee
2. K. Birman
3. Xiaodong Zhang
4. Yuhua Xu
5. Shuo Feng
6. Jimmy J. Lin
7. Robbert van Renesse
8. Chris Dyer

In Information Systems

1. Xiaodong Zhang
2. Chin-Teng Lin
3. Yuhua Xu
4. Shuo Feng
5. Mukesh Prasad
6. Jimmy J. Lin
7. Robbert van Renesse
8. K. Birman

Citations in the corpus

• An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics (2010, methodology): “MapReduce paradigm – Hadoop employs a Map/ Reduce execution engine [15-17] to implement its faulttolerant distributed computing system over the large data sets stored in the cluster’s distributed file system.”
• A Survey of Large Scale Data Management Approaches in Cloud Environments (2011, background): “From their discussion with some users of the MapReduce framework, Dean and Ghemawat [57] have indicated that MapReduce is very useful for handling data processing and data loading in a heterogenous system with many different storage systems.”
• Running Smart Grid Control Software on Cloud Computing Architectures (2011, background): “…systems have exceptional capabilities for moving data from the Internet into the cloud (web crawlers), indexing and searching that data (MapReduce [16], Chord [3], Dynamo [17], etc.), managing files that might contain petabytes of information (BigTable [13], the Google File System [20],…”
• Want to scale in centralized systems? Think P2P (2015, neutral): “Traditional centralized approaches achieve this by constructing the KNN graph offline and exploiting elastic cloud platforms to massively parallelize the recommendation jobs on numerous nodes [36, 37].”
• A survey of machine learning for big data processing (2016, background): “MapReduce [75, 76], a powerful programming framework, enables the automatic paralleling and distribution of computation applications on large clusters of commodity machines.”
• Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases (2017, background): “These features enable Spanner to support consistent backups, consistent distributed query processing [26], and atomic schema updates, all at global scale, and even in the presence of ongoing transactions.”
• Dremel (2010, neutral): “In this section, we address the following challenges: lossless representation of record structure in a columnar format (Section 4.1), fast encoding (Section 4.2), and efficient record assembly (Section 4.3).”
• The Internet of Things: A Survey from the Data-Centric Perspective (2013, neutral): “In this context, Google’s MapReduce framework [33] provides an effective method for analysis of the sensor data, especially when the nature of the computations involve linearly computable statistical functions over the elements of the data streams (such as MIN, MAX, SUM, MEAN etc.”
• Hadoop-GIS: A High Performance Spatial Data Warehousing System over MapReduce (2013, background): “Comparisons of MapReduce and parallel databases for structured data are discussed in [29, 20, 30].”
• The Hadoop distributed filesystem: Balancing portability and performance (2010, methodology): “The MapReduce programming model has emerged as a scalable way to perform data-intensive computations on commodity cluster computers [8], [9].”
• A review of clustering techniques and developments (2017, background): “Hadoop is one such big data processing tool [169-171].”
• Book Reviews: Data-Intensive Text Processing with MapReduce by Jimmy Lin and Chris Dyer (2010, background): “However, see Dean and Ghemawat’s counterarguments [47] and recent attempts at hybrid architectures [1].”
• Data Classification: Algorithms and Applications (2014, methodology): “As stated in [37]: “The run-time system takes care of the details of partitioning the input data, scheduling the program’s execution across a set of machines, handling machine failures, and managing the required inter-machine communication.”