Benchmarking TPU, GPU, and CPU Platforms for Deep Learning

By Yu Emma Wang, Gu-Yeon Wei, David M. Brooks. arXiv.org, 2019

325 citations recorded by the source corpus.

Training deep learning models is compute-intensive and there is an industry-wide trend towards hardware specialization to improve performance. To systematically benchmark deep learning platforms, we introduce ParaDnn, a parameterized benchmark suite for deep learning that generates end-to-end models for fully connected (FC), convolutional (CNN), and recurrent (RNN) neural networks. Along with six real-world models, we benchmark Google's Cloud TPU v2/v3, NVIDIA's V100 GPU, and an Intel Skylake CPU platform. We take a deep dive into TPU architecture, reveal its bottlenecks, and highlight valuable lessons learned for future specialized system design. We also provide a thorough comparison of the platforms and find that each has unique strengths for some types of models. Finally, we quantify the rapid performance improvements that specialized software stacks provide for the TPU and GPU platforms.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/f5dcdf616c3af51210631d39981de7c131682887

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Hardware and Architecture

Citations in the corpus

Simple and Efficient ways to Improve REALM (2021, methodology): “We perform exact MIPS search by leveraging the efficiency of large matrix multiplications of TPUs (Wang et al., 2019) to compute the retrieval score, Sretr, for∼13M passages of corpus and extract c passages having the highest scores.”
Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond (2021, background): “Fortunately, recent developments in computer-processor architecture have empowered the computing resources required for deep learning [36].”
The Deep Learning Compiler: A Comprehensive Survey (2020, background): “ack of large developer community. Chainer [87] was once the preferred framework for dynamic computation graphs, however replaced by MXNet, PyTorch and TensorFlow with similar features. Previous works [10, 25, 35, 70, 82, 100] have compared the performance of DL frameworks on different applications (”
DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference (2020, background): “To amortize this cost, GPUs often require larger batch sizes to exhibit speedup over CPUs, as shown in Figure 9 [35].”
Physics-Informed Neural Network for Ultrasound Nondestructive Quantification of Surface Breaking Cracks (2020, background): “The recent advances in the machine learning algorithms along with the growth in data and computing resources [1] have transformative results across various scientific disciplines including but not limited to image recognition [2], cognitive sciences [3], genomics [4] and bioinformatics [5].”
Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset (2020, background): “Also, with the advent of nanotechnology semiconductors, a new generation of Tensor Processing Units (TPUs) and Graphical Processing Units (GPUs) can provide an extraordinary computation capability for data-driven methods [27].”