Map-Reduce for Machine Learning on Multicore

By Cheng-Tao Chu, Sang Kyun Kim, Yi-An Lin, YuanYuan Yu, Gary R. Bradski, Andrew Y. Ng, Kunle Olukotun. 2006

1,288 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

Citations in the corpus

TensorFlow: A system for large-scale machine learning (2016, neutral): “Batch dataﬂow systems Starting with MapReduce [22], batch dataﬂow systems have been applied to a large number of machine learning algorithms [71], and more recent systems have focused on increasing expressivity and performance.”
Learning in a Large Function Space: Privacy-Preserving Mechanisms for SVM Learning (2009, methodology): “However, note that the SVM primal can be iteratively optimized where each iterate is a function of sums over the data (Chapelle, 2007; Chu et al., 2006).”
Towards Making Systems Forget with Machine Unlearning (2015, methodology): “Our unlearning approach is inspired by prior work on speeding up machine learning algorithms with MapReduce [33].”
Slow Learners are Fast (2009, methodology): “Grid Computing: Computational paradigms such as MapReduce [4] and Hadoop are well suited for the parallelization of batch-style algorithms [17].”
Distributed GraphLab: A Framework for Machine Learning in the Cloud (2012, background): “MapReduce [8, 9], Dryad [19] and Pregel [25]) fail to support these critical properties.”
Demystifying Parallel and Distributed Deep Learning (2018, background): “Prior to these works, the potential scaling of MapReduce was studied [42] on a variety of machine-learning problems, including NNs, promoting the need to shift from single-processor learning to distributed memory systems.”
Distributed learning for Random Vector Functional-Link networks (2015, background): “So far, a large body of research has gone into developing fully distributed, decentralized learning algorithms, including works on diffusion adaptation [21,34], learning by consensus [16], distributed learning on commodity clusters architectures [8], adaptation on WSNs [5,32], distributed online lea”
Parallelized Stochastic Gradient Descent (2010, methodology): “Algorithm Latency tolerance MapReduce Network IO Scalability Distributed subgradient [3, 9] moderate yes high linear Distributed convex solver [7] high yes low unclear Multicore stochastic gradient [5] low no n.”
Large-scale deep unsupervised learning using graphics processors (2009, methodology): “…such as logistic regression, linear SVMs and others can be easily implemented in parallel on multicore architectures, by having each core perform the required computations for a subset of input examples, and then combining the results centrally (Dean & Ghemawat, 2004; Chu et al., 2006).”
Large-scale image classification: Fast feature extraction and SVM training (2011, methodology): “Hadoop is an open source implementation of MapReduce computation framework and a distributed file system [3].”
Multi‐center machine learning in imaging psychiatry: A meta‐model approach (2017, neutral): “, 2014) 3) Computational complexity – increasing with large number of subjects, especially in cases when particular classifiers exhibit higher-order time or memory complexity (Chu et al., 2007).”
An Introduction to Conditional Random Fields (2010, neutral): “[22].”
Iterative parameter mixing for distributed large-margin training of structured predictors for natural language processing (2015, background): “Thus, a more efficient strategy to continue increasing computational power, and the one adopted recently in practice, is to keep the performance of a single computing core fixed, but to add more computing cores (Chu et al., 2007).”
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing (2012, methodology): “have shown that iterated MapReduce can also be used to implement other common learning algorithms [11].”