Design and Implementation of a General Purpose Parallel Programming System

By Mark Chu-Carroll, Yuan Yu, Lori L. Pollock, Michael Isard, Dennis Fetterly, Mihai Budiu, Ú. Erlingsson, P. Gunda, J. Currey. 1996

863 citations recorded by the source corpus.

Subfields: ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

Citations in the corpus

TensorFlow: A system for large-scale machine learning (2016, neutral): “DryadLINQ [74] adds a high-level query language that supports more sophisticated algorithms than MapReduce.”
Differentially-private network trace analysis (2010, methodology): “In the specific context of PINQ, because it is based on LINQ, the analyses will also automatically scale to a cluster [32].”
Running Smart Grid Control Software on Cloud Computing Architectures (2011, background): “Thus, cloud computing systems have exceptional capabilities for moving data from the Internet into the cloud (web crawlers), indexing and searching that data (MapReduce [16], Chord [3], Dynamo [17], etc.), managing files that might contain petabytes of information (BigTable [13], the Google File Sys”
Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center (2011, methodology): “Spark leverages Scala to provide a language-integrated syntax similar to DryadLINQ [47]: users invoke parallel operations by applying a function on a special “distributed dataset” object, and the body of the function is captured as a closure to run as a set of tasks in Mesos.”
Swift: A language for distributed parallel scripting (2011, methodology): “Scripting-level use of Dryad is now supported primarily by DryadLINQ [43], which generates Dryad computations from the LINQ extensions to C#.”
A Survey of Large Scale Data Management Approaches in Cloud Environments (2011, background): “Dryad has its own high-level language called DryadLINQ [41] .”
Occupy the cloud: distributed computing for the 99% (2017, methodology): “While the developer has no control of where a stateless function runs (e.g., the developer cannot specify that a stateless function should run on the node storing the function’s input), the benefits of colocating computa- tion and data – a major design goal for prior systems like Hadoop, Spark and D”
Big data analytics on Apache Spark (2016, neutral): “Iterative algorithms DryadLINQ [88], Iterative MapReduce (e.”
Tupleware: Distributed Machine Learning on Small Clusters (2014, methodology): “Other recent frameworks (e.g., Spark, Stratosphere [21], DryadLINQ [35]) have started to bridge the gap between expressiveness and optimizability by allowing users to easily compose arbitrary workﬂows of UDFs.”
The Stratosphere platform for big data analytics (2014, background): “The“BigData” software ecosystem includes distributed ﬁle systems [29], parallel data analysis platforms [8,12,15,22,44,70,74], data programming languages [13,19,37,57,69], and more specialized tools for speciﬁc data domains [53,54].”
Cloud Computing and the DNA Data Race (2010, background): “New parallel programming frameworks, such as DryadLINQ 22 and Pregel 23 , can also help in some cases by providing richer programming abstractions.”
Dremel (2010, methodology): “Other systems for parallel data processing include Scope [6] and DryadLINQ [23], and are discussed in more detail in [7].”