MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
By Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, Zheng Zhang. 2015
2,339 citations recorded by the source corpus.
MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends declarative symbolic expression with imperative tensor computation. It offers auto differentiation to derive gradients. MXNet is computation and memory efficient and runs on various heterogeneous systems, ranging from mobile devices to distributed GPU clusters. This paper describes both the API design and the system implementation of MXNet, and explains how embedding of both symbolic expression and tensor operation is handled in a unified fashion. Our preliminary experiments reveal promising results on large scale deep neural network applications using multiple GPU machines.
Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, ML Systems & Hardware
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
- Cong Xie
- Oluwasanmi Koyejo
- Indranil Gupta
- Ion Stoica
- Neil Zhenqiang Gong
- Eric P. Xing
- M. Franklin
- Phillip B. Gibbons
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Trevor Darrell
- Yi Yang
- Jitendra Malik
- Alan L. Yuille
In Electrical and Electronic Engineering
- Carole-Jean Wu
- Oluwasanmi Koyejo
- Indranil Gupta
- Cong Xie
- Naiyan Wang
- Song Han
- Alex Smola
- Torsten Hoefler
In ML Systems & Hardware
- Cong Xie
- Oluwasanmi Koyejo
- Indranil Gupta
- Eric P. Xing
- Ion Stoica
- Alex Smola
- Sebastian Raschka
- Torsten Hoefler
Citations in the corpus
- Array programming with NumPy (2020, methodology): “For example, the NumPy API is implemented by several popular tensor computation libraries including CuPy3, JAX4, and Apache MXNet5.”
- TensorFlow: A system for large-scale machine learning (2016, neutral): “MXNet [12] is a recent system that uses a parameter server to scale training, supports GPU acceleration, and includes a flexible programming model with interfaces for many languages.”
- Training Deep Nets with Sublinear Memory Cost (2016, methodology): “We our method on top of MXNet [6], which statically allocate all the intermediate feature maps before computation.”
- An Introduction to Probabilistic Programming (2018, neutral): “, 2017), MxNET (Chen et al., 2016), and CNTK (Seide and Agarwal, 2016).”
- AllenNLP: A Deep Semantic Natural Language Processing Platform (2018, background): “Most existing deep-learning toolkits are designed for general machine learning (Bergstra et al., 2010; Yu et al., 2014; Chen et al., 2015; Abadi et al., 2016; Neubig et al., 2017), and can require signifi-cant effort to develop research infrastructure for particular model classes.”
- Dual Path Networks (2017, methodology): “We implement the DPNs using MXNet [2] on a cluster with 40 K80 graphic cards.”
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size (2016, methodology): “Finally, we use our design strategies to construct SqueezeNet , which is comprised mainly of Fire modules.”
- Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, methodology): “We use MXNet [12] to implement federated learning and attacks.”
- DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks (2017, methodology): “We use MxNet as our neural network framework [5].”
- Deep Interest Network for Click-Through Rate Prediction (2017, methodology): “Here we reuse the open-sourced deep learning frameworks, like tensorflow, mxnet, theano[18, 19, 20]etc.”
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “General distributed training libraries in PyTorch [67], TensorFlow [68], MXNet [69], and distributed training-specialized libraries such as Horovod [70] and BytePS [71] are designed for distributed training in data centers.”
- Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates (2019, methodology): “Most of the existing large-scale distributed deeplearning frameworks, such as Tensorflow (Abadi et al., 2016), PyTorch (Steiner et al., 2019), and MXNet Chen et al. (2015) support either PS or AllReduce.”
- Secure Evaluation of Quantized Neural Networks (2019, background): “We identify the quantization techniques used in both TensorFlow, PyTorch and MXNet and described in [38] as particularly well suited for MPC.”
- Practical Distributed Learning: Secure Machine Learning with Communication-Efficient Local Updates (2019, methodology): “We implement the federated optimization algorithms using the MXNET (Chen et al., 2015) framework.”
- Exploiting Unintended Feature Leakage in Collaborative Learning (2018, methodology): “We use MxNet [8] and Lasagne [12] to implement deep neural networks and Scikit-learn [48] for conventional ML models.”