Large-scale matrix factorization with distributed stochastic gradient descent

By Rainer Gemulla, Erik Nijkamp, Peter J. Haas, Yannis Sismanis. 2011

704 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition, Optimization & Training, Recommender Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Optimization & Training

In Recommender Systems

Citations in the corpus

Recommendations as Treatments: Debiasing Learning and Evaluation (2016, methodology): “So, we can readily draw upon existing optimization algorithms (i.e., Gemulla et al., 2011; Yu et al., 2012) that can efﬁciently solve the training problem at scale.”
Perturbed Iterate Analysis for Asynchronous Stochastic Optimization (2015, background): “Recent advances in parallel and distributed computing technologies have generated renewed interest in the theoretical understanding and practical implementation of parallel stochastic algorithms [15–20].”
More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server (2013, methodology): “The entries Dab can then be distributed over multiple workers, and their gradients computed in parallel [12].”
Petuum: A New Platform for Distributed Machine Learning on Big Data (2013, methodology): “We begin by formalizing ML algorithms as iterativeconvergent programs, which encompass a large space of modern ML, such as stochastic gradient descent (SGD)[18] and coordinate descent (CD) [10] for convex optimization problems, proximal methods [19] for more complex constrained optimization, as well”
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System (2019, background): “Furthermore, unlike ALS, parallelization of SGD is challenging, and numerous approaches have been proposed to distribute the learning of matrix factorization [14, 15, 16, 17].”
Byzantine-Tolerant Machine Learning (2017, methodology): “In most SGD-based learning algorithms used today [3,5,6,25], the choice function consists in computing the average of the input vectors.”
Guaranteed Matrix Completion via Non-Convex Factorization (2014, background): “Index Terms —Matrix completion, matrix factorization, nonconvex optimization, alternating minimization, SGD, perturbation analysis.”
Fast Matrix Factorization for Online Recommendation with Implicit Feedback (2016, background): “Thus, sophisticated strategies are required to control the possible losses introduced by parallelization [7].”
PyTorch-BigGraph: A Large-scale Graph Embedding System (2019, background): “Convergence under SGD to a stationary or chain-recurrent point, is still guaranteed under this modiﬁ-cation (see (Gemulla et al., 2011), Sec.”
Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization (2018, methodology): “Large-scale matrix factorization: In our next set of experiments, we consider a large-scale matrix factorization problem (Gemulla et al., 2011; S¸ims¸ekli et al., 2015; S¸ims¸ekli et al., 2017), where the goal is to obtain the MAP solution of the following probabilistic model: Here, Y ∈ R R × S is…”
On Model Parallelization and Scheduling Strategies for Distributed Machine Learning (2014, methodology): “In particular, numerous data-parallel solutions from both algorithmic [28, 10] and system [7, 25] angles have been proposed to speed up inference and learning on Big Data.”
Distributed large-scale natural graph factorization (2013, background): “Several sequential and distributed algorithms have been proposed to tackle this problem over the last four years [11] [17] [25] [26] [16].”
FlexiFaCT: Scalable Flexible Factorization of Coupled Tensors on Hadoop (2014, methodology): “Thus, FlexiFaCT includes several recent methods [9], [11], as special cases.”
A Fast Parallel Stochastic Gradient Method for Matrix Factorization in Shared Memory Systems (2015, methodology): “CCD++ is a parallel coordinate descent method that is considered state-of-the-art for matrix factorization.”
Logistic Matrix Factorization for Implicit Feedback Data (2014, methodology): “To scale up computation we employ a sharding technique similar to those used by other latent factor models [4, 7].”