GraphLab: A New Framework For Parallel Machine Learning

By Yucheng Low, Joseph E. Gonzalez, Aapo Kyrola, Danny Bickson, Carlos Guestrin, Joseph M Hellerstein. Conference on Uncertainty in Artificial Intelligence, 2010

927 citations recorded by the source corpus.

Designing and implementing efficient, provably correct parallel machine learning (ML) algorithms is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. By targeting common patterns in ML, we developed GraphLab, which improves upon abstractions like MapReduce by compactly expressing asynchronous iterative algorithms with sparse computational dependencies while ensuring data consistency and achieving a high degree of parallel performance. We demonstrate the expressiveness of the GraphLab framework by designing and implementing parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and Compressed Sensing. We show that using GraphLab we can achieve excellent parallel performance on large scale real-world problems.

Subfields: Artificial Intelligence, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/4187caa4d0d329f47e18377a6cd31ef3f580cfcc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Big data analytics on Apache Spark (2016, methodology): “– Graph-Parallel Frameworks: Several large-scale graphparallel frameworks were designed to efficiently execute graph algorithms, such as GraphLab [55], Pregel [57], Kineograph [19] and PowerGraph [34].”
Iterative parameter mixing for distributed large-margin training of structured predictors for natural language processing (2015, methodology): “, 2012), which allows memory read from disk to persist in main memory for faster re-use, and the GraphLab framework (Low et al., 2010), which allows better for asynchronous jobs.”
Distributed Multi-task Learning for Sensor Network (2017, background): “[12] proposed parallel and distributed learning approach in which a number of processors cooperated together for a single learning task.”
Over-optimization of academic publishing metrics: observing Goodhart’s Law in action (2018, positive): “The framework uses TuriCreate’s SFrame dataframe objects [83] to perform big-data analysis on tens of millions of records to calculate how various properties have changed over time.”
Simple and Scalable Response Prediction for Display Advertising (2014, methodology): “However, the abstraction is rather ill-suited for machine learning algorithms, as several researchers in the field have observed [Low et al. 2010; Zaharia et al. 2012], because it does not easily allow iterative algorithms, such as the typical optimization algorithms used to minimize the objective…”
Minerva: A Scalable and Highly Efficient Training Platform for Deep Learning (2014, positive): “Indeed, our first prototype leverages the gather/scatter/apply API in GraphLab [16].”
Cyclades: Conflict-free Asynchronous Machine Learning (2016, methodology): “GraphLab and PowerGraph support serializable execution via locking mechanisms This is in contrast to our partition-and-allocate coordination which allows us to provide guarantees on speedup.”
MLbase: A Distributed Machine-learning System (2013, neutral): “These primitives currently include efficient implementations of gradient and stochastic gradient descent, mini-batch extensions of map-reduce that naturally support divide-and-conquer approaches such as [21], and graph-parallel primitives as in GraphLab [20].”
Direction-optimizing Breadth-First Search (2012, background): “On mobile clients, graph algorithms are important components of recognition and machine-learning applications [19,28].”
Strategies and Principles of Distributed Machine Learning on Big Data (2015, methodology): “Similar non-uniform convergence has been observed and exploited in PageRank, another iterative-convergent algorithm [53].”
FENNEL: streaming graph partitioning for massive scale graphs (2014, methodology): “Many popular graph processing platforms such as Pregel [23] that builds on MapReduce [7], and its open source cousin Apache Giraph, PEGASUS [13] and GraphLab [22] use as a default partitioner Hash Partition of vertices, which corresponds to assigning each vertex to one of the k partitions uniformly ”
Fast Differentially Private Matrix Factorization (2015, background): “GraphLab Create is a closed source data analysis platform [18].”