DeepSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression

By Hanlin Tang, Xiangru Lian, Shuang Qiu, Lei Yuan, Ce Zhang, T. Zhang, Ji Liu. 2019

35 citations recorded by the source corpus.

Subfields: Efficiency & Compression, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

1. Medhi Bennis
2. Jakub Konecný
3. Oluwasanmi Koyejo
4. Martin Jaggi
5. Ananda Theertha Suresh
6. Qiang Yang
7. Sebastian U. Stich
8. Mehryar Mohri

In Optimization & Training

1. Jakub Konecný
2. Peter Richtárik
3. Martin Jaggi
4. Oluwasanmi Koyejo
5. Jianyu Wang
6. Gauri Joshi
7. Sebastian U. Stich
8. Medhi Bennis

Citations in the corpus

• Advances and Open Problems in Federated Learning (2019, background): “Characterizing such fundamental trade-offs between accuracy and communication has been of recent interest in theoretical statistics [434, 81, 195, 11, 47, 380].”
• Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning (2019, background): “…to reduce the cost of gradient communication (Al-istarh et al., 2017; Bernstein et al., 2018; Wen et al., 2017; Yu et al., 2018; Chen et al., 2018; Wang et al., 2019; Stich et al., 2018; Suresh et al., 2017; Sattler et al., 2019; Tang et al., 2018; 2019; Assran et al., 2019; Acharya et al., 2019).”
• 1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed (2021, unknown): “…to the promising efficiency of this method, error compensation has been applied into many related area (Zheng et al., 2019; Phuong and Phong, 2020; Yu et al., 2019; Shi et al., 2019; Ivkin et al., 2019; Sun et al., 2019; Basu et al., 2019; Vogels et al., 2019) in order to reduce the communication…”