Hive - a petabyte scale data warehouse using Hadoop

By Ashish Thusoo, Joydeep Sen Sarma, Namit Jain, Zheng Shao, Prasad Chakka, Ning Zhang, Suresh Anthony, H. Liu, R. Murthy. IEEE International Conference on Data Engineering, 2010

1,044 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems, Management Information Systems

View Paper: https://www.semanticscholar.org/paper/4c7bfa933c11c7a802c2fa9c1dc475dba36a2bd5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Management Information Systems

Citations in the corpus

Understandable Big Data: A survey (2015, methodology): “4. the value of the use of semantics in data integration and in the design of future DBMS. Similar challenges have been identified by S. Auer and J. Lehmann [37].”
A Survey of Large Scale Data Management Approaches in Cloud Environments (2011, background): “HiveQL supports data definition (DDL) statements which can be used to create, drop and alter tables in a database [74].”
Low Latency Geo-distributed Data Analytics (2015, background): “These workloads consist of a mix of Spark [59] and Hive [49] queries.”
Shark: fast data analysis using coarse-grained distributed memory (2012, methodology): “Compared with disk-oriented warehouse solutions and batch infrastructures such as Apache Hive [3], Shark excels at ad-hoc, exploratory queries by exploiting inter-query temporal locality and also leverages the intra-query locality inherent in iterative machine learning algorithms.”
Data warehousing and analytics infrastructure at facebook (2010, methodology): “This compression scheme is based on PAX[8] storage layout for data records.”
Shark: SQL and rich analytics at scale (2012, methodology): “It inherits Hive’s schema-on-read capability and nested data types [28].”
Inside the Social Network's (Datacenter) Network (2015, methodology): “Samples are simultaneously stored into Hive [38] tables for long-term analysis.”
The Anatomy of the Facebook Social Graph (2011, methodology): “Unless otherwise noted, calculations were performed on a Hadoop cluster with 2,250 machines, using the Hadoop/Hive data analysis framework developed at Facebook [45, 46].”
Spanner: Google's Globally-Distributed Database (2012, neutral): “They point to several other efforts that have been made to explore database functionality layered on distributed key-value stores [1, 4, 7, 41] as evidence that the two worlds are converging.”
CIEL: A Universal Execution Engine for Distributed Data-Flow Computing (2011, methodology): “Pig and Hive implement the relational algebra using a DAG of MapReduce jobs on Hadoop [32, 37]; DryadLINQ and SCOPE implement it using a Dryad graph [14, 39].”
Apache Hadoop YARN: yet another resource negotiator (2013, background): “High-level frameworks like Pig[24] and Hive[30] often compose a workflow of MapReduce jobs in a DAG, each filtering, aggregating, and projecting data at every stage in the computation.”
Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks (2014, background): “Furthermore, many high-level programming interfaces [5, 8, 46], such as Pig [38] and FlumeJava [19], compile programs into multiple MapReduce jobs that run sequentially.”
Spark SQL: Relational Data Processing in Spark (2015, background): “Spark offers a functional programming API similar to other recent systems [20, 11], where users manipulate distributed collections called Resilient Distributed Datasets (RDDs) [39].”
RCFile: A fast and space-efficient data placement structure in MapReduce-based warehouse systems (2011, methodology): “Two widely used Hadoop-based warehouse systems are Hive [4][5] in Facebook and Pig [6] in Yahoo! These MapReduce-based warehouse systems cannot directly control storage disks in clusters.”
Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics (2021, background): “Second, there is also broad investment in SQL engines that run directly against data lake storage, such as Spark SQL, Presto, Hive, and AWS Athena [3, 11, 45, 50].”