LOCO: Distributing Ridge Regression with Random Projections
By Christina Heinze, Brian McWilliams, Nicolai Meinshausen, Gabriel Krummenacher. 2014
37 citations recorded by the source corpus.
We propose LOCO, an algorithm for large-scale ridge regression which distributes the features across workers on a cluster. Important dependencies between variables are preserved using structured random projections which are cheap to compute and must only be communicated once. We show that LOCO obtains a solution which is close to the exact ridge regression solution in the fixed design setting. We verify this experimentally in a simulation study as well as an application to climate prediction. Furthermore, we show that LOCO achieves significant speedups compared with a state-of-the-art distributed algorithm on a large-scale regression problem.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jakub Konecný
- Peter Richtárik
- Martin Jaggi
- Virginia Smith
- Martin Takác
- Michael I. Jordan
- Chenxin Ma
- Sebastian U. Stich
In Computer Vision and Pattern Recognition
- Jakub Konecný
- Martin Takác
- Peter Richtárik
- Martin Jaggi
- Virginia Smith
- Michael I. Jordan
- Chenxin Ma
- Nicolai Meinshausen
Citations in the corpus
- CoCoA: A General Framework for Communication-Efficient Distributed Optimization (2016, methodology): “At the other extreme, there are distributed methods that use only a single round of communication, such as Mann et al. (2009); Zinkevich et al. (2010); Zhang et al. (2013); McWilliams et al. (2014); and Heinze et al. (2016).”
- Adding vs. Averaging in Distributed Primal-Dual Optimization (2015, methodology): “At the other extreme, there are distributed methods using only a single round of communication, such as (Zhang et al., 2013; Zinkevich et al., 2010; Mann et al., 2009; McWilliams et al., 2014).”
- Distributed optimization with arbitrary local solvers (2015, methodology): “At the other extreme, there are distributed methods using only a single round of communication, such as [24, 36, 38, 75, 78].”
- Stochastic, Distributed and Federated Optimization for Machine Learning (2017, methodology): “At the other extreme, there are distributed methods using only a single round of communication, such as [198, 203, 114, 76, 75].”
- DUAL-LOCO: Distributing Statistical Estimation Using Random Projections (2015, methodology): “Recently, LOCO was proposed to solve ridge regression when the data is distributed across the features [11].”
- Preserving privacy between features in distributed estimation (2017, background): “Lemma 5 (Concatenating random features (Lemma 3 from Heinze et al. (2014))) .”