Communication-efficient distributed statistical learning
By Michael I. Jordan, Jason D. Lee, Yun Yang. 2016
19 citations recorded by the source corpus.
We present the Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical learning problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For low-dimensional estimation, CSL provably improves upon the averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax optimal estimator with minimal communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of MCMC algorithms even in a non-distributed setting. The methods are illustrated through empirical studies.
Subfields: Artificial Intelligence, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jialei Wang
- Nati Srebro
- Peter Richtárik
- Tong Zhang
- Ilias Diakonikolas
- Sebastian U. Stich
- Martin Jaggi
- Jakub Konecný
In Statistics and Probability
- Ilias Diakonikolas
- Daniel M. Kane
- Nati Srebro
- Jialei Wang
- Sankeerth Rao
- Mladen Kolar
- Themis Gouleakis
- Tong Zhang
Citations in the corpus
- Communication-Efficient Distributed Learning of Discrete Probability Distributions (2017, background): “In recent years, a number of statistical estimation problems have been algorithmically studied in the distributed setting [3, 16, 15, 40, 21, 30, 24, 33, 5, 29].”
- Communication and Memory Efficient Testing of Discrete Distributions (2019, background): “A series of works Zhang et al. (2013); Garg et al. (2014); Braverman et al. (2016); Jordan et al. (2016); Diakonikolas et al. (2017b); Han et al. (2018b,a) studied distributed learning in both parametric and nonparametric settings obtaining (nearly-) matching upper and lower bounds.”
- Efficient Distributed Learning with Sparsity (2016, background): “Concurrent work of (Jordan et al., 2016) (personal communication) also improved computational efficiency of Lee et al.”