Comparative Study of Deep Learning Software Frameworks

By Soheil Bahrampour, Naveen Ramakrishnan, Lukas Schott, Mohak Shah. 2015

168 citations recorded by the source corpus.

Deep learning methods have resulted in significant performance improvements in several application domains and as such several software frameworks have been developed to facilitate their implementation. This paper presents a comparative study of five deep learning frameworks, namely Caffe, Neon, TensorFlow, Theano, and Torch, on three aspects: extensibility, hardware utilization, and speed. The study is performed on several types of deep learning architectures and we evaluate the performance of the above frameworks when employed on a single machine for both (multi-threaded) CPU and GPU (Nvidia Titan X) settings. The speed performance metrics used here include the gradient computation time, which is important during the training phase of deep networks, and the forward time, which is important from the deployment perspective of trained networks. For convolutional networks, we also report how each of these frameworks support various convolutional algorithms and their corresponding performance. From our experiments, we observe that Theano and Torch are the most easily extensible frameworks. We observe that Torch is best suited for any deep architecture on CPU, followed by Theano. It also achieves the best performance on the GPU for large convolutional and fully connected networks, followed closely by Neon. Theano achieves the best performance on GPU for training and deployment of LSTM networks. Caffe is the easiest for evaluating the performance of standard deep architectures. Finally, TensorFlow is a very flexible framework, similar to Theano, but its performance is currently not competitive compared to the other studied frameworks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, background): “Each framework has its own strength based on its supported DL architectures, optimization algorithms, and ease of development and deployment [46].”
Benchmarking State-of-the-Art Deep Learning Software Tools (2016, methodology): “[27] did the similar work with us, but they only used a single architecture of GPU (i.”
Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey (2019, methodology): “– Comparing the most well-known DL frameworks (Bahrampour et al. 2015; Shi et al. 2016; Akiba 2017; Karmanov et al. 2018; Liu et al. 2018) (e.”
Predicting process behaviour using deep learning (2016, background): “A number of software frameworks for deep-learning have become available recently (Bahrampour et al., 2015).”
Deep Learning in Mining Biological Data (2020, background): “) in CPU and GPU platforms. The numbers in (a, c) denote the number of CPU threads employed in the benchmarking process, and in (d-f) denote the batch size. In case of DA the batch size was 64. times [155,156]. Moreover, to the best of our knowledge, there exist two main efforts that provide the ben”
Convolutional Neural Networks for Inverse Problems in Imaging: A Review (2017, background): “For a comparison of these libraries, refer to [35]*; here, we provide a basic overview.”
Deep Learning in Medical Ultrasound Analysis: A Review (2019, unknown): “[84].”
DAWNBench : An End-to-End Deep Learning Benchmark and Competition (2017, background): “Most existing benchmarks for deep learning performance [1, 2, 10], and some research papers [4, 5, 8, 35], only measure proxy metrics such as the time to process one minibatch of data.”
Deep Learning vs. Traditional Computer Vision (2019, background): “high-powered GPUs[29] and TPUs [30] for training and AI accelerated platforms such as VPUs for inference [31]) for developers of AI.”
The Deep Learning Compiler: A Comprehensive Survey (2020, background): “Previous works [10, 25, 35, 70, 82, 100] have compared the performance of DL frameworks on different applications (e.”
Deep learning in agriculture: A survey (2018, background): “Moreover, there exist various tools and platforms allowing researchers to experiment with DL (Bahrampour et al., 2015).”
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark (2018, background): “Many prior ML benchmarks use throughput (either per-kernel or per-iteration) as a metric [6,11,12,21,30,58].”
Deep learning in bioinformatics (2016, methodology): “Core Speed for batch* (ms) Multi-GPU Distributed Strengths [56,57]”
A Survey of Deep Convolutional Neural Networks Applied for Prediction of Plant Leaf Diseases (2021, background): “It is the most accessible framework to evaluate the performance of deep architectures [115].”
A review of the use of convolutional neural networks in agriculture (2018, background): “Moreover, there are various tools and platforms that allow researchers to experiment with DL (Bahrampour et al., 2015).”