Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?

By Eriko Nurvitadhi, Ganesh Venkatesh, Jaewoong Sim, Debbie Marr, Randy Huang, J. Hock, Yeong Tat Liew, Krishnan Srivatsan, Duncan J. M. Moss, S. Subhaschandra and 1 more. Symposium on Field Programmable Gate Arrays, 2017

456 citations recorded by the source corpus.

Subfields: Electrical and Electronic Engineering, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/a04166adc56ba373c20ffd704ba5e141e8dc6489

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

1. Tal Ben-Nun
2. Kurt Keutzer
3. Torsten Hoefler
4. Bichen Wu
5. Dusist Niyato
6. Aiman H. El-Maleh
7. Victor C. M. Leung
8. Kees A. Vissers

In Hardware and Architecture

1. Aiman H. El-Maleh
2. Tal Ben-Nun
3. Torsten Hoefler
4. Kurt Keutzer
5. Sadiq M. Sait
6. Xiaofei Wang
7. Ahmad Shawahna

Citations in the corpus

• Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs (2018, methodology): “[32] implements a ternary neural network on high-end Intel FPGAs and achieves higher performance/Watt than Titan X GPU.”
• Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, background): “To minimize on-chip memory requirement, the weight matrix can be stored in a dense format [110].”
• A survey of deep learning techniques for autonomous driving (2019, methodology): “ep learning algorithms can be analyzed from four major perspectives: efficiency and power, raw computing power, flexibility and functional safety. Our study is based on the research published by Intel (Nurvitadhi et al., 2017), Microsoft (Ovtcharov et al., 2015) and UCLA (Cong et al., 2018). By reduci”
• MnnFast: A Fast and Scalable System Architecture for Memory-Augmented Neural Networks (2019, background): “As shown in figure 12(b), H2D latency differences between the worst case and the ideal case are getting larger as the number of GPUs increases.”
• Demystifying Parallel and Distributed Deep Learning (2018, methodology): “Accelerators are usually programmed with special languages such as NVIDIA’s CUDA, OpenCL, or in the case of FPGAs using hardware design languages.”
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, methodology): “NVIDIA’ GPUs based on Turing architecture [37]; 2) Field Programmable Gate Array (FPGA)-based hardware [51],”
• FINN-R (2018, unknown): “We constrain our comparison to only measured results and exclude extrapolated numbers ([45],[41],[44][39]).”
• FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, background): “[61] recently evaluated emerging DNN algorithms on latest generations of GPUs (i.”