Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer
By Yongming Shen, Michael Ferdman, Peter Milder. 2017
106 citations recorded by the source corpus.
Convolutional neural networks (CNNs) are used to solve many challenging machine learning problems. Interest in CNNs has led to the design of CNN accelerators to improve CNN evaluation throughput and efficiency. Importantly, the bandwidth demand from weight data transfer for modern large CNNs causes CNN accelerators to be severely bandwidth bottlenecked, prompting the need for processing images in batches to increase weight reuse. However, existing CNN accelerator designs limit the choice of batch sizes and lack support for batch processing of convolutional layers. We observe that, for a given storage budget, choosing the best batch size requires balancing the input and weight transfer. We propose Escher, a CNN accelerator with a flexible data buffering scheme that ensures a balance between the input and weight transfer bandwidth, significantly reducing overall bandwidth requirements. For example, compared to the state-of-the-art CNN accelerator designs targeting a Virtex-7 690T FPGA, Escher reduces the accelerator peak bandwidth requirements by 2.4x across both fully-connected and convolutional layers on fixed-point AlexNet, and reduces convolutional layer bandwidth by up to 10.5x on fixed-point GoogleNet.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yongming Shen
- Vikas Chandra
- Liangzhen Lai
- Naveen Suda
- Michael Ferdman
- Peter Milder
- Nitin Rathi
- Hadi Esmaeilzadeh
In Computer Vision and Pattern Recognition
- Yongming Shen
- Michael Ferdman
- Peter Milder
- Vikas Chandra
- Hadi Esmaeilzadeh
- Liangzhen Lai
- Bashir I. Morshed
- G. Diamos
In Electrical and Electronic Engineering
- Nitin Rathi
- Yongming Shen
- Kaushik Roy
- Michael Ferdman
- Peter Milder
- Sparsh Mittal
- Hadi Esmaeilzadeh
- Bashir I. Morshed
In Efficiency & Compression
- Vikas Chandra
- Hadi Esmaeilzadeh
- Michael Ferdman
- Bashir I. Morshed
- G. Diamos
- Yongming Shen
- Syed Kamrul Islam
- Jianlin Cheng
Citations in the corpus
- Beyond human-level accuracy: computational challenges in deep learning (2019, background): “Layer parallelism can also reduce the memory requirements for model weights [33, 34].”
- Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “The map batching technique balances the heavy transfer of parameters in dense layers by kernel reuse eliminating the transfer to an external memory [111].”
- Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network (2017, background): “Alwani, et. al [52] propose layer-fusion, that combines multiple convolutional layers to save off-chip accesses for FPGA acceleration of CNNs. Es-cher [53] propose a CNN FPGA accelerator using flexible buffering that balances the off-chip accesses for inputs and weights in CNNs.”
- AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, unknown): “…[60], XNOR-POP [61], DRISA [62], 3DICT [63], NAND-NET [134], SCOPE [135], Promise [136] Light Models EIE [64], SC-DCNN [65], SCNN [66], Escher [67], LookNN [68], Bit-Pragmatic DNN [69], Bit Fusion [70], Cn-vlutin [99], TIE [137], Laconic [138], ADM-NN [139], Gist [140] Co-Design DPS-CNN [71],…”
- Maximizing CNN accelerator efficiency through resource partitioning (2016, background): “[25] optimizes batch sizes to reduce off-chip data transfer.”
- DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization (2021, background): “Many proposals reduce the memory cost by data buffering [48], trading computations for memory [49], and data reuse through efficient dataflows [50].”