eSGD: Communication Efficient Distributed Deep Learning on the Edge

By Zeyi Tao, Qun Li. 2018

141 citations recorded by the source corpus.

Subfields: Optimization & Training, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Liekang Zeng
2. Victor C. M. Leung
3. Young Geun Kim
4. Ying-Chang Liang
5. Zhi Zhou
6. Qiang Yang
7. Chunyan Miao
8. Xu Chen

In Federated & Distributed Learning

1. Young Geun Kim
2. Zhi Zhou
3. Qiang Yang
4. Xu Chen
5. Dusist Niyato
6. Xiaofei Wang
7. Dinh Thai Hoang
8. Colin Raffel

Citations in the corpus

• Training Neural Networks with Fixed Sparse Masks (2021, background): “An orthogonal approach to reducing communication costs is to have workers only update a small subset of the model’s parameters [2, 13, 44, 45].”
• Machine Learning at the Network Edge: A Survey (2019, methodology): “[89] Local & global update comparison technique in federate learning to reduce communication cost Tao and Li [130] Developed eSGD that is capable of reducing the gradient size of a CNN model by up to 90% Lin et al.”
• All One Needs to Know about Fog Computing and Related Edge Computing Paradigms: A Complete Survey (2018, unknown): “Applications [20, 51, 61, 100, 154, 159, 174, 175, 190, 191, 247, 263, 271, 276, 287, 306, 320, 321, 323, 330–379]”
• AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning (2021, methodology): “laboratively train a shared ML model while keeping all the raw data on device [11, 34, 52, 64, 71, 77, 84, 116, 120, 135, 137].”
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey (2019, background): “• Importance-based Updating: This strategy involves selective communication such that only the important or relevant updates [95] are transmitted in each communication round.”
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, background): “Different from [190]–[192], in the scenario of [196], training”
• Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, methodology): “Inspired by [53], Tao and Li [54] propose edge SGD (eSGD), a family of sparse schemes with both convergence and practical performance guarantees.”