Partitioning Data on Features or Samples in Communication-Efficient Distributed Optimization?
By Chenxin Ma, Martin Takác. 2015
11 citations recorded by the source corpus.
In this paper we study the effect of the way that the data is partitioned in distributed optimization. The original DiSCO algorithm [Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss, Yuchen Zhang and Lin Xiao, 2015] partitions the input data based on samples. We describe how the original algorithm has to be modified to allow partitioning on features and show its efficiency both in theory and also in practice.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Peter Richtárik
- Jakub Konecný
- Virginia Smith
- Martin Jaggi
- Konstantin Mishchenko
- Martin Takác
- Nicolas Loizou
- Chenxin Ma
Citations in the corpus
- Communication trade-offs for synchronized distributed SGD with large step size (2019, background): “Some relevant references are Lee et al. (2014); Ma and Takáč (2015); Smith et al. (2016); Chen et al. (2016b); Fang and Klabjan (2018).”
- Distributed optimization with arbitrary local solvers (2015, background): “…with the massive growth of available datasets, major roadblocks still persist in the distributed setting, where data no longer fits in the memory of a single computer, and computation must be split across multiple machines in a network [3, 7, 12, 18, 22, 28, 32, 34, 37, 46, 50, 58, 60, 62, 76].”
- Stochastic, Distributed and Federated Optimization for Machine Learning (2017, background): “However, with the massive growth of available datasets, major roadblocks still persist in the distributed setting, where data no longer fits in the memory of a single computer, and computation must be split across multiple machines in a network [19, 111, 106, 8, 149, 53, 173, 57, 32, 108, 166, 167, ”
- Distributed learning with compressed gradient differences (2019, methodology): “The key methods in this sphere include CoCoA and its variants (Jaggi et al., 2014; Ma et al., 2015; 2017b;a; Smith et al., 2018), DANE (Shamir et al., 2014), DiSCO (Zhang and Xiao, 2015; Ma and Takáč, 2015), DANCE (Jahani et al., 2018) and AIDE (Reddi et al., 2016).”