SCNN: An accelerator for compressed-sparse convolutional neural networks

By Angshuman Parashar, Minsoo Rhu, Anurag Mukkara, A. Puglielli, Rangharajan Venkatesan, Brucek Khailany, Joel S. Emer, S. Keckler, William (Bill) J. Dally. 2017

1,254 citations recorded by the source corpus.

Convolutional Neural Networks (CNNs) have emerged as a fundamental technology for machine learning. High performance and extreme energy efficiency are critical for deployments of CNNs, especially in mobile platforms such as autonomous vehicles, cameras, and electronic personal assistants. This paper introduces the Sparse CNN (SCNN) accelerator architecture, which improves performance and energy efficiency by exploiting the zero-valued weights that stem from network pruning during training and zero-valued activations that arise from the common ReLU operator. Specifically, SCNN employs a novel dataflow that enables maintaining the sparse weights and activations in a compressed encoding, which eliminates unnecessary data transfers and reduces storage requirements. Furthermore, the SCNN dataflow facilitates efficient delivery of those weights and activations to a multiplier array, where they are extensively reused; product accumulation is performed in a novel accumulator array. On contemporary neural networks, SCNN can improve both performance and energy by a factor of 2.7× and 2.3×, respectively, over a comparably provisioned dense CNN accelerator.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

To prune, or not to prune: exploring the efficacy of pruning for model compression (2017, methodology): “The resulting pruned model typically has sparse connection matrices, so efﬁcient inference using these sparse models requires purpose-built hardware capable of loading sparse matrices and/or performing sparse matrix-vector operations (Zhang et al., 2016; Han et al., 2016; Parashar et al., 2017).”
AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, methodology): “Fine-grained pruning method prunes neural networks based on individual connections to achieve sparsity in both weights and activations, which is able to achieve higher compression ratio and can be accelerated with specialized hardware such as [18, 17, 39].”
A Configurable Cloud-Scale DNN Processor for Real-Time AI (2018, background): “The commercial viability of DNNs has spawned a wave of products from large companies and startups, including Google’s TPU [10], [43], [44], Nvidia’s SCNN [45], Wave Computing [46], Graphcore [3], Movidius [47], and many others (complete list in [48]).”
Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks (2018, background): “Sparsity in DNN models can be exploited by accelerators [41], [42].”
MASR: A Modular Accelerator for Sparse RNNs (2019, methodology): “Then we show how the underlying lane micro-architecture handles sparsity in weights and activations.”
Timeloop: A Systematic Approach to DNN Accelerator Evaluation (2019, background): “Also, although Timeloop already accounts for the energy savings due to sparsity, future work includes modeling architectures that save both time and energy [1], [15], [29], [39].”
SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training (2020, methodology): “We compare SIGMA against other state-of-the-art accelerators: TPU [4], EIE [19], SCNN [33], OuterSPACE [32], Eyeriss v2 [11], Packed Systolic [26], and Cambricon-X [47].”
Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices (2018, background): “For instance, EIE targets fully-connected (FC) layers [41], while SCNN targets convolutional (CONV) layers [42].”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “The SCNN [244] supports compressed processing for convolutional layers, and EIE [243] supports processing for dense layers.”
Inducing and Exploiting Activation Sparsity for Fast Neural Network Inference (2020, methodology): “This observation has been leveraged by hardware accelerators, e.g. (Albericio et al., 2016; Han et al., 2016; Parashar et al., 2017), and reference (Rhu et al., 2018) performed an analysis of naturally-occurring activation sparsity.”
Energy-Constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking (2018, positive): “Many dedicated DNN hardware chips (a.k.a., Neural Processing Units, NPUs) (Jouppi et al., 2017; Chen et al., 2016; Han et al., 2016; Parashar et al., 2017) have been developed to directly beneﬁt from model sparsity, and are already widely available in today’s consumer devices such as Apple iPhoneX,…”
Understanding Reuse, Performance, and Hardware Cost of DNN Dataflow: A Data-Centric Approach (2018, methodology): “DNN accelerators achieve high performance by exploiting parallelism over hundreds of processing elements (PEs) and high energy efficiency by maximizing data reuse within PEs and on-chip scratchpads [1, 9, 11, 19, 31, 38].”
SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization (2019, background): “Specialized accelerators on FPGA or ASIC show great potential in bringing a much higher speedup and power efﬁciency for sparse DNNs [6, 2, 21, 1, 28].”
DeepStore: In-Storage Acceleration for Intelligent Queries (2019, methodology): “DNN Accelerator: Several accelerator designs have been proposed to speed up the training and inference computation of popular DNN models [28, 29, 34, 44, 49, 50, 52, 60, 65, 68, 71, 75, 78].”
Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions (2018, background): “Such a sparsity-promotion operation may marginally decrease compression performance as it restricts the ﬂexibility of setting centroids, but can gain more in both speedup and energy savings (Parashar et al., 2017).”
DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference (2020, background): “Building upon the performance bottlenecks, a variety of software and hardware solutions have been proposed to optimize traditional DNNs [13]–[15], [17], [48]–[68].”
Dynamic Channel Pruning: Feature Boosting and Suppression (2018, background): “This has prompted custom accelerator solutions that exploit sparse weights (Parashar et al., 2017; Han et al., 2016).”