ATP: In-network Aggregation for Multi-tenant Learning

By Chon-In Lao, Yanfang Le, Kshiteej S. Mahajan, Yixi Chen, Wenfei Wu, Aditya Akella, Michael M. Swift. Symposium on Networked Systems Design and Implementation, 2021

335 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/c90a5375329ea537d4ac266297b1c5cca89af19b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Dheevatsa Mudigere
2. Maxim Naumov
3. Yuchen Hao
4. Xiaodong Wang
5. Liangchen Luo
6. Jongsoo Park
7. Jianyu Huang
8. Xing Liu

Citations in the corpus

• High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (2021, background): “SwitchML and ATP [27, 47] leverages programmable network switches to perform in-network aggregation for cross-rack bandwidth reduction in datacenter environments.”
• Scaling Distributed Machine Learning with In-Network Aggregation (2019, unknown): “Alternatively, ATP [54] – a follow up work to ours – explores”
• Efficient sparse collective communication and its application to accelerate distributed deep learning (2020, unknown): “SwitchML [58], Mellanox SHARP [44] and ATP [38] recently demonstrated the feasibility of streaming collective aggregation protocols where the aggregation takes place within network switches.”
• Software-hardware co-design for fast and scalable training of deep learning recommendation models (2021, background): “SwitchML and ATP [30, 53] leverages programmable network switches to perform in-network aggregation for cross-rack bandwidth reduction in datacenter environments.”
• MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud (2022, background): “SwitchML [50] and ATP [27] use the programmable switches as gradient aggregation servers to reduce the communication overheads.”