DUAL-LOCO: Distributing Statistical Estimation Using Random Projections
By Christina Heinze, Brian McWilliams, Nicolai Meinshausen. 2015
41 citations recorded by the source corpus.
We present DUAL-LOCO, a communication-efficient algorithm for distributed statistical estimation. DUAL-LOCO assumes that the data is distributed according to the features rather than the samples. It requires only a single round of communication where low-dimensional random projections are used to approximate the dependences between features available to different workers. We show that DUAL-LOCO has bounded approximation error which only depends weakly on the number of workers. We compare DUAL-LOCO against a state-of-the-art distributed optimization method on a variety of real world datasets and show that it obtains better speedups while retaining good accuracy.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Jakub Konecný
- Martin Jaggi
- Christina Heinze-Deml
- Celestine Mendler-Dünner
- Simone Forte
- Virginia Smith
- Peter Richtárik
- Francis R. Bach
Citations in the corpus
- CoCoA: A General Framework for Communication-Efficient Distributed Optimization (2016, methodology): “If only one outer round of communication is performed, this is similar in spirit to one-shot communication schemes, which attempt to completely solve for and then combine locallycomputed models (see, e.g., Mann et al., 2009; Zhang et al., 2013; Heinze et al., 2016).”
- Stochastic, Distributed and Federated Optimization for Machine Learning (2017, methodology): “At the other extreme, there are distributed methods using only a single round of communication, such as [198, 203, 114, 76, 75].”
- Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems (2017, methodology): “Cats ImageNet dataset as presented in [6], where we additionally double the number of samples, while using single precision floating point numbers.”
- Preserving privacy between features in distributed estimation (2017, methodology): “In Heinze et al. (2016), λ is tuned “globally”, i.e. the local predictions for a particular λ are communicated, added and thus evaluated on the global objective.”