Scalable deep learning on distributed GPUs with a GPU-specialized parameter server

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

The Non-IID Data Quagmire of Decentralized Machine Learning (2019, methodology): “We implement and evaluate SkewScout in a GPU parameter server system (Cui et al., 2016) based on Caffe (Jia et al., 2014).”
GeePS: scalable deep learning on distributed GPUs with a GPU-specialized parameter server (2016, background): “MXNet [6] and Poseidon [35] are two concurrently developed ([12]) systems for multi-GPU deep learning.”
TF-Replicator: Distributed Machine Learning for Researchers (2019, methodology): “Examples of parameter server-based systems for machine learning include DistBelief (Dean et al., 2012), GeePS (Cui et al., 2016) and Project Adam (Chilimbi et al., 2014).”
TicTac: Accelerating Distributed Deep Learning with Communication Scheduling (2018, methodology): “Smarter interleaving of computation and communication : This approach is adopted by several layer-by-layer systems (Arnold, 2016; Cui et al., 2016; Zhang et al., 2017) where the models are sequential and obtaining the order is trivial (Cui et al., 2014).”
3LC: Lightweight and Effective Traffic Compression for Distributed Machine Learning (2018, background): “Parameter servers , or simply servers , store a partition of the global model, which consists of parameters (trainable variables).”