A comparison of approaches to large-scale data analysis

By Andrew Pavlo, Erik Paulson, A. Rasin, Daniel J. Abadi, David J. DeWitt, Samuel Madden, M. Stonebraker. SIGMOD Conference, 2009

1,252 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems, Signal Processing

View Paper: https://www.semanticscholar.org/paper/18b89ae68d587b5150fbf2e933bcbc1c3404bee9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Signal Processing

Citations in the corpus

A Survey of Large Scale Data Management Approaches in Cloud Environments (2011, methodology): “[75] have conducted a large scale comparison between the Hadoop implementation of MapReduce framework and parallel SQL database management systems in terms of performance and development complexity.”
BigDataBench: A big data benchmark suite from internet services (2014, methodology): “This section presents our methodology and decisions on BigDataBench.”
Runtime measurements in the cloud (2010, methodology): “For this, we consider the following relation (as suggested in [25]), UserVisits(UV)=(sourceIP, visitedURL, visitDate, adRevenue), which is a simplified version of a relation containing information about the revenue generated by user visits to webpages.”
Data-intensive applications, challenges, techniques and technologies: A survey on Big Data (2014, neutral): “In [139], Andrew Pavlo gave an overall discussion on properties of Map/Reduce framework, as well as other approaches to large-scale data analysis.”
Shark: SQL and rich analytics at scale (2012, methodology): “Hadoop was shown by [25] to excel at data loading, as its data loading throughput was ﬁve to ten times higher than that of MPP databases.”
SkewTune: mitigating skew in mapreduce applications (2012, background): “Reducing Launch Overhead: Launching a new MapReduce job is relatively expensive compared to launching a job in a database management system (DBMS) because every task and job starts from scratch [28].”
Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads (2012, methodology): “The data comes from standard logging tools in Hadoop; no additional tools were necessary.”
HaLoop (2010, background): “Although parallel database systems [5] may also serve these data analysis applications, they can be expensive, difficult to administer, and lack fault-tolerance for long-running queries [16].”
MapReduce and parallel DBMSs: friends or foes? (2010, methodology): “Hence, the two technologies are complementary, and we expect MR-style systems performing ETL to live directly upstream from DBMSs.”
Book Reviews: Data-Intensive Text Processing with MapReduce by Jimmy Lin and Chris Dyer (2010, background): “With colleagues, they ran a series of benchmarks that demonstrated the supposed superiority of column-oriented parallel databases over Hadoop [120, 144].”
A survey of multiple classifier systems as hybrid systems (2014, background): “nected with the big data analysis problem [84].”
The Snowflake Elastic Data Warehouse (2016, result): “Vectorized execution means that, in contrast to MapReduce for example [42], Snowﬂake avoids materialization of intermediate results.”
Predicting the Performance of Virtual Machine Migration (2010, methodology): “We use SPEC CPU [15] for CPU bound workloads, SPECweb [16] for webserver workloads, SPECsfs [17] for I/O intensive workloads, and MapReduce tasks [18] for non-interactive workloads.”