Maximizing CNN accelerator efficiency through resource partitioning

By Yongming Shen, Michael Ferdman, Peter Milder. 2016

336 citations recorded by the source corpus.

Convolutional neural networks (CNNs) are revolutionizing machine learning, but they present significant computational challenges. Recently, many FPGA-based accelerators have been proposed to improve the performance and efficiency of CNNs. Current approaches construct a single processor that computes the CNN layers one at a time; the processor is optimized to maximize the throughput at which the collection of layers is computed. However, this approach leads to inefficient designs because the same processor structure is used to compute CNN layers of radically varying dimensions. We present a new CNN accelerator paradigm and an accompanying automated design methodology that partitions the available FPGA resources into multiple processors, each of which is tailored for a different subset of the CNN convolutional layers. Using the same FPGA resources as a single large processor, multiple smaller specialized processors increase computational efficiency and lead to a higher overall throughput. Our design methodology achieves 3.8x higher throughput than the state-of-the-art approach on evaluating the popular AlexNet CNN on a Xilinx Virtex-7 FPGA. For the more recent SqueezeNet and GoogLeNet, the speedups are 2.2x and 2.0x.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

Beyond human-level accuracy: computational challenges in deep learning (2019, background): “Layer parallelism can also reduce the memory requirements for model weights [33, 34].”
Simba: Scaling Deep-Learning Inference with Multi-Chip-Module-Based Architecture (2019, background): “Tile-based architectures have frequently been proposed for deeplearning accelerator designs [1, 2, 13, 15, 22, 25, 30, 42, 51, 53, 56, 58, 61, 75].”
Timeloop: A Systematic Approach to DNN Accelerator Evaluation (2019, background): “There is an abundance of excellent recent work on DNN accelerators [6], [13], [22], [26], [27], [32], [35]–[37].”
Machine Learning at Facebook: Understanding Inference at the Edge (2019, background): “Recent work on hardware for machine learning and efficient training and inference has substantially advanced the state-of-the-art [41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66].”
A High-Throughput and Power-Efficient FPGA Implementation of YOLO CNN for Object Detection (2019, methodology): “To reduce the size of block RAM, studies in [4], [12], [18], [20], and [25] save the intermediate data for each layer in off-chip memory.”
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, methodology): “[192] proposed a multi-CLP accelerator system for CNNs where the”
Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer (2017, background): “[15], [17], [29]–[31] use vector dot product arrays.”
Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search (2019, methodology): “With such vision, a series of works for implementing neural networks on FPGAs have been carried out [3, 4, 8, 9, 11, 13, 15].”
A survey of FPGA-based accelerators for convolutional neural networks (2018, methodology): “[40] note that FPGA-based accelerators use CONV layer engines (CLE) that process consecutive CNN layers one-at-a-time.”
MAERI: Enabling Flexible Dataflow Mapping over DNN Accelerators via Reconfigurable Interconnects (2018, background): “This work also opens up exciting opportunities in compiler designs that can take arbitrary DNNs and map them efficiently over a Maeri-like fabric.”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, background): “Due to the high demand of computing power in DL applications, we have recently witnessed a phenomenal trend in which the landscape of computing has shifted from general-purpose processors to domain-specific architectures [16–110].”
Hardware/Software Co-Exploration of Neural Architectures (2019, neutral): “…to 35FPS, which can satisfy most cameras; whereas for ImageNet, due to the more complicated architectures and the limited resource, we set the speciﬁcation to 10FPS. Finally, for both data and weights, we apply the commonly used 16-bit ﬁxed-point data, as that in [21], [29], [30], and [39].”
A Configurable Cloud-Scale DNN Processor for Real-Time AI (2018, background): “There has been a large body of research in DNN accelerators targeting FPGAs, in particular for accelerating Convolutional Neural Networks (CNNs), such as [42], [63]–[94].”