Boosting the Performance of CNN Accelerators with Dynamic Fine-Grained Channel Gating

By Weizhe Hua, Yuan Zhou, Christopher De Sa, Zhiru Zhang, G. Suh. 2019

48 citations recorded by the source corpus.

This paper proposes a new fine-grained dynamic pruning technique for CNN inference, named channel gating, and presents an accelerator architecture that can effectively exploit the dynamic sparsity. Intuitively, channel gating identifies the regions in the feature map of each CNN layer that contribute less to the classification result and turns off a subset of channels for computing the activations in these less important regions. Unlike static network pruning, which removes redundant weights or neurons prior to inference, channel gating exploits dynamic sparsity specific to each input at run time and in a structured manner. To maximize compute savings while minimizing accuracy loss, channel gating learns the gating thresholds together with weights automatically through training. Experimental results show that the proposed approach can significantly speed up state-of-the-art networks with a marginal accuracy loss, and enable a trade-off between performance and accuracy. This paper also shows that channel gating can be supported with a small set of extensions to a CNN accelerator, and implements a prototype for quantized ResNet-18 models. The accelerator shows an average speedup of 2.3× for ImageNet when the theoretical FLOP reduction is 2.8×, indicating that the hardware can effectively exploit the dynamic sparsity exposed by channel gating.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Song Han
3. Gao Huang
4. Yulin Wang
5. Hanrui Wang
6. Le Yang
7. Yizeng Han
8. Honghui Wang

In Computer Vision and Pattern Recognition

1. Shiji Song
2. Song Han
3. Gao Huang
4. Yulin Wang
5. Yizeng Han
6. Le Yang
7. Hanrui Wang
8. Luc Van Gool

In Efficiency & Compression

1. Song Han
2. Gao Huang
3. Hanrui Wang
4. Shiji Song
5. Yulin Wang
6. Le Yang
7. Yizeng Han
8. Zhekai Zhang

In ML Systems & Hardware

1. Song Han
2. Gao Huang

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, methodology): “tions, including pixel-level dynamic computation [114], [275], [276], adaptive channel pruning [277], [278] and early exiting [279], have also been tailored together with hard-”
• A review of convolutional neural network architectures and their optimizations (2022, methodology): “proposed the fine-grained dynamic method called channel gating to prune CNNs inference (Hua et al. 2019).”
• SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning (2020, background): “Many of them leverage the sparsity to improve performance [57]– [73].”
• S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search (2019, background): “However, these methods lead to irregular computation at channel level or spatial level, which are not efficiently supported by existing software and hardware devices [43, 47, 16].”