SnaPEA: Predictive Early Activation for Reducing Computation in Deep Convolutional Neural Networks

By Vahideh Akhlaghi, Amir Yazdanbakhsh, K. Samadi, Rajesh K. Gupta, Hadi Esmaeilzadeh. 2018

168 citations recorded by the source corpus.

Deep Convolutional Neural Networks (CNNs) perform billions of operations for classifying a single input. To reduce these computations, this paper offers a solution that leverages a combination of runtime information and the algorithmic structure of CNNs. Specifically, in numerous modern CNNs, the outputs of compute-heavy convolution operations are fed to activation units that output zero if their input is negative. By exploiting this unique algorithmic property, we propose a predictive early activation technique, dubbed SnaPEA. This technique cuts the computation of convolution operations short if it determines that the output will be negative. SnaPEA can operate in two distinct modes, exact and predictive. In the exact mode, with no loss in classification accuracy, SnaPEA statically re-orders the weights based on their signs and periodically performs a single-bit sign check on the partial sum. Once the partial sum drops below zero, the rest of computations can simply be ignored, since the output value will be zero in any case. In the predictive mode, which trades the classification accuracy for larger savings, SnaPEA speculatively cuts the computation short even earlier than the exact mode. To control the accuracy, we develop a multi-variable optimization algorithm that thresholds the degree of speculation. As such, the proposed algorithm exposes a knob to gracefully navigate the trade-offs between the classification accuracy and computation reduction. Compared to a state-of-the-art CNN accelerator, SnaPEA in the exact mode, yields, on average, 28% speedup and 16% energy reduction in various modern CNNs without affecting their classification accuracy. With 3% loss in classification accuracy, on average, 67.8% of the convolutional layers can operate in the predictive mode. The average speedup and energy saving of these layers are 2.02x and 1.89x, respectively. The benefits grow to a maximum of 3.59x speedup and 3.14x energy reduction. Compared to static pruning approaches, which are complimentary to the dynamic approach of SnaPEA, our proposed technique offers up to 63% speedup and 49% energy reduction across the convolution layers with no loss in classification accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey (2020, background): “32(a) monitors the sign of the accumulated activation in each PE and terminates its computation once a negative value is detected [242].”
Machine Learning at Facebook: Understanding Inference at the Edge (2019, background): “Recent work on hardware for machine learning and efficient training and inference has substantially advanced the state-of-the-art [41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66].”
A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation (2020, unknown): “Specifically, SnaPEA [75] exploits the unique”
MASR: A Modular Accelerator for Sparse RNNs (2019, methodology): “Given MASR’s modular design, compact activation storage provides the same area beneﬁt to all design variants; input and hidden-state memories are maintained outside of the PEs/lanes.”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, background): “The number of operations can be lessened by observing the sign of accumulated activation in the PE [240].”
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications (2018, background): “ltiplications, thereby offering signiﬁcant improvements in energy efﬁciency [49]. Akhlaghi et al. propose early stopping of convolution when the output is expected to be non-positive followed by ReLU [1]. Sharma et al. present a systolic array accelerator called BitFusion which supports variable bit”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, methodology): “…DPS-CNN [71], Minerva [72], MoDNN [73], MeDNN [74], AdaLearner [75], Stitch-X [76], PIM-DNN [77], Scalpel [78], CirCNN [79], CoSMIC [80], SnaPEA [81], OLAcce [82], Prediction-based DNN [83], PERMDNN [84], MnnFast [141], DNN Computation Reuse [142], vDNN [143], Compressing-DMA-Engine [144],…”
MnnFast: A Fast and Scalable System Architecture for Memory-Augmented Neural Networks (2019, background): “Value prediction [47, 54, 84] can increase pruning candidates [2, 65].”
Boosting the Performance of CNN Accelerators with Dynamic Fine-Grained Channel Gating (2019, methodology): “Table 4(a) and 4(b) compare CGNet to prior arts [5, 9, 24] on AlexNet and ResNet-18 for ImageNet.”
Planaria: Dynamic Architecture Fission for Spatial Multi-Tenant Acceleration of Deep Neural Networks (2020, background): “The need for higher speed and efficiency in DNN execution has led to an explosion of DNN accelerators [1, 3, 71–93] that has even made their way to operational datacenters (Google’s TPU [1], NVIDIA T4 [2], Microsoft Brainwave [3], etc.”
Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network (2017, unknown): “Snapea [59] employs early termination to skip computations.”
SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization (2019, background): “Specialized accelerators on FPGA or ASIC show great potential in bringing a much higher speedup and power efﬁciency for sparse DNNs [6, 2, 21, 1, 28].”
Understanding Reuse, Performance, and Hardware Cost of DNN Dataflow: A Data-Centric Approach (2018, background): “Figure 2: Abstract DNN accelerator architecture model which is pervasive in many state-of-the-art accelerators [3, 11, 19, 31, 38].”
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing (2019, background): “Most research efforts within the architecture community have focused on accelerating the compute-intensive, highly-regular computational patterns found in fully-connected (FC), convolution (CNN), and recurrent (RNN) neural networks [5], [6], [10], [15], [18], [24], [27], [34], [39], [42], [48], [56]”