Hadoop-GIS: A High Performance Spatial Data Warehousing System over MapReduce

By Ablimit Aji, Fusheng Wang, Hoang Vo, Rubao Lee, Qiaoling Liu, Xiaodong Zhang, Joel H. Saltz. Proceedings of the VLDB Endowment, 2013

608 citations recorded by the source corpus.

Support of high performance queries on large volumes of spatial data becomes increasingly important in many application domains, including geospatial problems in numerous fields, location based services, and emerging scientific applications that are increasingly data- and compute-intensive. The emergence of massive scale spatial data is due to the proliferation of cost effective and ubiquitous positioning technologies, development of high resolution imaging technologies, and contribution from a large number of community users. There are two major challenges for managing and querying massive spatial data to support spatial queries: the explosion of spatial data, and the high computational complexity of spatial queries. In this paper, we present Hadoop-GIS – a scalable and high performance spatial data warehousing system for running large scale spatial queries on Hadoop. Hadoop-GIS supports multiple types of spatial queries on MapReduce through spatial partitioning, customizable spatial query engine RESQUE, implicit parallel spatial query execution on MapReduce, and effective methods for amending query results through handling boundary objects. Hadoop-GIS utilizes global partition indexing and customizable on demand local spatial indexing to achieve efficient query processing. Hadoop-GIS is integrated into Hive to support declarative spatial queries with an integrated architecture. Our experiments have demonstrated the high efficiency of Hadoop-GIS on query response and high scalability to run on commodity clusters. Our comparative experiments have showed that performance of Hadoop-GIS is on par with parallel SDBMS and outperforms SDBMS for compute-intensive queries. Hadoop-GIS is available as a set of library for processing spatial queries, and as an integrated software package in Hive.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/4cb4af8e5bb1c4d9569a7920b2a419873540092d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Remote sensing big data computing: Challenges and opportunities (2015, methodology): “In addition, the Hadoop–GIS [57] system for large-scale spatial data processing, search and accessing is also build upon the Hadoop system.”
Big Data and cloud computing: innovation opportunities and challenges (2017, background): “Performance is critical in Big Data era, and accurately and quickly locating data requires a new generation of search engines and query systems (Miyano and Uehara 2012; Aji et al. 2013).”
GeoSpark: a cluster computing framework for processing large-scale spatial data (2015, methodology): “Hadoop-GIS [1] utilizes global partition indexing and customizable on demand local spatial indexing to achieve eﬃcient query processing.”
A Survey of Traffic Prediction: from Spatio-Temporal Data to Intelligent Transportation (2021, methodology): “Later, two distributed systems Spa-tial-Hadoop [81] and Hadoop GIS [82] are designed for spatial data analytics, where the two systems are implemented based on Hadoop.”
SpatialHadoop: A MapReduce framework for spatial data (2015, result): “Similar to Hadoop, a SpatialHadoop cluster contains one master node that breaks a map-reduce job into smaller tasks, carried out by slave nodes.”