GraphX: Graph Processing in a Distributed Dataflow Framework

By Joseph E. Gonzalez, Reynold Xin, Ankur Dave, Daniel Crankshaw, M. Franklin, Ion Stoica. 2014

1,216 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

SparkNet: Training Deep Networks in Spark (2015, background): “…SparkNet, training a deep network on the output of a SQL query, or a graph computation, or a streaming data source is straightforward due to its general purpose nature and its support for SQL, graph computations, and data streams (Armbrust et al., 2015; Gonzalez et al., 2014; Zaharia et al., 2013).”
MLlib: Machine Learning in Apache Spark (2015, background): “GraphX (Gonzalez et al., 2014) supports large-scale graph processing and provides a powerful API for implementing learning algorithms that can naturally be viewed as large, sparse graph problems, e.g., LDA (Blei et al., 2003; Bradley, 2015).”
Big data analytics on Apache Spark (2016, background): “In order to make graph analysis more flexible, GraphX extends Spark operatorswith a collection of specialized operators for property graphs [35].”
Addressing the straggler problem for iterative convergent parallel ML (2016, methodology): “For example, the highly-tuned Sparkbased GraphX [27] was shown to approximately match PowerGraph [26], which has been shown [15] to be 2–10X slower than two recent parameter server systems (IterStore [15] and LazyTable [14]) for collaborative filtering via sparse matrix factorization.”
Spark SQL: Relational Data Processing in Spark (2015, methodology): “Advanced Analytics Spark SQL builds on recent work to run advanced analytics algorithms on large clusters, including platforms for iterative algorithms [39] and graph analytics [15, 24].”
DStress: Efficient Differentially Private Computations on Distributed Data (2017, background): “In the age of “big data”, it is well known that many interesting things can be learned by collecting and analyzing large graphs, and a number of tools – including GraphLab [47], PowerGraph [35], and GraphX [36] – have been developed to make such analyses fast and convenient.”
Proposal Scaling Distributed Machine Learning with System and Algorithm Co-design (2016, unknown): “Previous works Graph partitioning has attracted a lot of research interests in scientific computing [28, 47, 73], scaling out large-scale computations [20, 59, 60, 141, 147, 155], graph database [39, 122, 144], search and social network analysis [7, 112, 141], and streaming processing [105, 130, 131”
Incremental, iterative data processing with timely dataflow (2016, methodology): “These challenges have spurred research into specialized distributed graph-processing systems(11) and—more recently—attempts to adapt dataflow systems for graph processing.(12) We used a variety of graph algorithms to evaluate both the expressiveness of the timely dataflow programming model and the p”
Opaque: An Oblivious and Encrypted Distributed Analytics Platform (2017, background): “variety of big data workloads, including complex graph analytics [14] and machine learning [17].”
One Trillion Edges: Graph Processing at Facebook-Scale (2015, methodology): “Vertices Edges LiveJournal [9] 4.8M 69M Twitter 2010 [31] 42M 1.5B UK web graph 2007 [10] 109M 3.7B Yahoo web [8] 1.4B 6.6B Many specialized graph processing frameworks (e.g. [20, 21, 32, 44]) have been developed to run on web and social graphs such as those shown in Table 1.”
EmptyHeaded: A Relational Engine for Graph Processing (2015, methodology): “We compared EmptyHeaded to GraphX [20] which is a graph engine designed for scale-out performance.”
PowerLyra: Differentiated Graph Computation and Partitioning on Skewed Graphs (2019, methodology): “In contrast, PowerGraph [24] and GraphX [26] focus on evenly parallelizing the computation by partitioning edges among machines, which incurs high communication cost among partitioned vertices even with low degrees.”