2016 Ieee International Conference on Big Data (big Data) Consensus Optimization with Delayed and Stochastic Gradients on Decentralized Networks

By Benjamin Sirb, Xiaojing Ye

Subfields: Optimization & Training, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Sebastian U. Stich
2. Martin Jaggi
3. Peter Richtárik
4. Ji Liu
5. Hanlin Tang
6. Anastasia Koloskova
7. Shusen Wang
8. Tong Zhang

In ML Systems & Hardware

1. Sebastian U. Stich
2. Martin Jaggi
3. Hanlin Tang
4. Anastasia Koloskova
5. Tong Zhang
6. Ji Liu
7. Ce Zhang
8. Shusen Wang

Citations in the corpus

• Communication Efficient Decentralized Training with Multiple Local Updates (2019, methodology): “Decentralized (stochastic) algorithms used to tackle the failure of being centralized as a compromise and used to be studied as consensus optimization in the control community [28, 50, 36].”
• Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication (2019, positive): “…case without any communication compression, and for consensus stepsize γ = 1 as in exact gossip (E-G) , Choco-SGD (Algorithm 2) recovers the following standard variant of decentralized SGD with gossip (similar e.g. to (Sirb & Ye, 2016; Lian et al., 2017)), stated for illustration in Algorithm 3.”
• Distributed Learning over Unreliable Networks (2018, background): “…gradient descent, such as the compression of communication, e.g Seide and Agarwal (2016), decentralized communication Lian et al. (2017a); Sirb and Ye (2016); Lan et al. (2017); Tang et al. (2018a); Stich et al. (2018), and asynchronous communication Lian et al. (2017b); Zhang et al.…”
• Communication Compression for Decentralized Training (2018, methodology): “Decentralized algorithms are mostly applied to solve the consensus problem [Zhang et al., 2017b, Lian et al., 2017a, Sirb and Ye, 2016], where the network topology is decentralized.”
• Communication-Efficient Local Decentralized SGD Methods (2019, background): “They were studied as consensus optimization in the control community [25, 45, 31].”
• Decentralization Meets Quantization (2018, methodology): “Decentralized algorithms are mostly applied to solve the consensus problem [Lian et al., 2017a, Sirb and Ye, 2016, Zhang et al., 2017b], where the network topology is decentralized.”