SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization

By Shijie Cao, Lingxiao Ma, Wencong Xiao, Chen Zhang, Yunxin Liu, Lintao Zhang, Lanshun Nie, Zhi Yang. 2019

80 citations recorded by the source corpus.

In this paper we present a novel and general method to accelerate convolutional neural network (CNN) inference by taking advantage of feature map sparsity. We experimentally demonstrate that a highly quantized version of the original network is sufficient in predicting the output sparsity accurately, and verify that leveraging such sparsity in inference incurs negligible accuracy drop compared with the original network. To accelerate inference, for each convolution layer our approach first obtains a binary sparsity mask of the output feature maps by running inference on a quantized version of the original network layer, and then conducts a full-precision sparse convolution to find out the precise values of the non-zero outputs. Compared with existing work, our approach avoids the overhead of training additional auxiliary networks, while is still applicable to general CNN networks without being limited to certain application domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search (2019, background): “They attempted to add extra controllers to select which computations are executed [5, 3, 34, 25, 15, 7, 46, 41, 42].”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, background): “For instance, activations are 40% and 70% sparse in CNN and FC layers, respectively, [220] due to ReLU activations.”
Forward and Backward Information Retention for Accurate Binary Neural Networks (2019, background): “Although much progress has been made on bina-rizing DNNs, the existing quantization methods remains a signiﬁcant drop of accuracy compared with the full-precision counterparts [14, 39, 2, 28, 57, 48, 9, 40, 43].”
Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation (2020, background): “rom model-predicted foreground masks at low resolution [32], from a small auxiliary layer that supplements each convolutional layer [7], and from a highly quantized version of the convolutional layer [1]. Our work instead reconstructs activation maps by interpolation from a sparse set of samples sel”
Binary Neural Networks: A Survey (2020, background): “In [123], SeerNet considers feature-map sparsity through low-bit quantization, which is applicable to general convolutional neural networks and tasks.”