Pig latin: a not-so-foreign language for data processing

By Christopher Olston, Benjamin C. Reed, Utkarsh Srivastava, Ravi Kumar, Andrew Tomkins. SIGMOD Conference, 2008

2,082 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems, Signal Processing

View Paper: https://www.semanticscholar.org/paper/3ac939c4ada416858e1c1bff9e6f18aaa98323f3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Signal Processing

Citations in the corpus

The jabberwocky programming environment for structured social computing (2011, methodology): “Inspired by the Pig [19] and Sawzall [21] languages for data mining, Dog deﬁnes a small but powerful set of primitives for requesting computational work from either people or machines, and for interfacing with workers (and their properties) through Dormouse.”
A comparison of join algorithms for log processing in MaPreduce (2010, background): “Our results provide insights that are unique to the MapReduce platform and offer guidance on when to use a particular join algorithm on this platform.”
BigDansing: A System for Big Data Cleansing (2015, methodology): “We also showed that one could use a declarative system ( e.g., Hive [32], Pig [28], or Shark [1]) on top of one of these platforms and re-implement the data quality rules using its query language.”
A survey on provenance: What for? What form? What from? (2017, methodology): “These applications are very diverse with respect to the use cases and platforms they support, e.g., Big Data processing using the Pig language and running onHadoopMapReduce clusters [116], grid-based scientific workflows [181], scientific experiments in various contexts [50,88,169], visualizations [”
One Trillion Edges: Graph Processing at Facebook-Scale (2015, background): “Other companies that use Apache Pig [35], a high-level language for expressing data analysis programs, can execute similar graph preparation steps as a series of simple queries.”
The Internet of Things: A Survey from the Data-Centric Perspective (2013, neutral): “Pig also provides a scripting language [83] called PigLatin, which provides all the familiar constructs of SQL such as projections, joins, sorting, grouping etc.”
GraphLab: A New Framework For Parallel Machine Learning (2010, methodology): “Implementations of this abstraction include Dryad [Isard et al., 2007] and Pig Latin [Olston et al., 2008].”
Coflow: a networking abstraction for cluster applications (2012, neutral): “Objective MapReduce [2, 13] Yes Write to the DFS None Minimize completion time Dataflow with Barriers [5, 21, 22] Yes Write to the DFS None Minimize completion time Dataflow w/o Explicit Barriers [9, 10, 12, 17, 27] Yes (Implicit) Input not ready None Minimize completion time Dataflow with Cycles [2”
A Survey of Data-Intensive Scientific Workflow Management (2015, background): “Pig [107] is an interactive, or script-based, execution environment atop MapReduce.”
Pregel: a system for large-scale graph processing (2010, background): “cilitate aggregation [41] and SQL-like queries [40, 47], but these extensions are usually not ideal for graph algorithms that often better fit a message passing model.”
Design and Implementation of a General Purpose Parallel Programming System (1996, positive): “At run time, vertices are processes communicating with each other through the channels, and each channel is used to transport a ﬁnite sequence of data records.”
A Survey of Large Scale Data Management Approaches in Cloud Environments (2011, methodology): “Pig Latin is architected such that the parsing of the Pig Latin program and the logical plan construction is independent of the execution platform.”
GraphX: Graph Processing in a Distributed Dataflow Framework (2014, methodology): “In this section we describe these optimizations in the context of classic techniques in traditional database systems including indexing, incremental view maintenance, and join optimizations.”
FlumeJava: easy, efficient data-parallel pipelines (2010, background): “As with Pig Latin, SQL queries are broken down into a series of distinct steps, with variables naming intermediate streams.”
Hadoop++ (2010, background): “And, inspired by DBMSs, other works also focus on providing more flexible query execution plans than MapReduce [15, 22].”