FLightNNs: Lightweight Quantized Deep Neural Networks for Fast and Accurate Inference
By Ruizhou Ding, Zeye (Dexter) Liu, Ting-Wu Chin, Diana Marculescu, R. D. Blanton. 2019
26 citations recorded by the source corpus.
To improve the throughput and energy efficiency of Deep Neural Networks (DNNs) on customized hardware, lightweight neural networks constrain the weights of DNNs to be a limited combination (denoted as k $\in$ {1, 2}) of powers of 2. In such networks, the multiply-accumulate operation can be replaced with a single shift operation, or two shifts and an add operation. To provide even more design flexibility, the k for each convolutional filter can be optimally chosen instead of being fixed for every filter. In this paper, we formulate the selection of k to be differentiable, and describe model training for determining k-based weights on a per-filter basis. Over 46 FPGA-design experiments involvmg eight configurations and four data sets reveal that lightweight neural networks with a flexible k value (dubbed FLightNNs) fully utilize the hardware resources on Field Programmable Gate Arrays (FPGAs), our experimental results show that FLightNNs can achieve $2\times$ speedup when compared to lightweight NNs with k = 2, with only 0.1% accuracy degradation. Compared to a 4-bit fixed-point quantization, FLightNNs achieve higher accuracy and up to $2\times$ inference speedup, due to their lightweight shift operations. In adchtion, our experiments also demonstrate that FLightNNs can achieve higher computational energy efficiency for ASIC implementation.CCS CONCEPTS• Computing methodologies $\rightarrow$ Machine learning; • Hardware$\rightarrow$Electronic design automation.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ruizhou Ding
- Di Wang
- Bodhi Priyantha
- Dimitrios Lymberopoulos
- Dimitrios Stamoulis
- Diana Marculescu
- Jie Liu
- Jun Du
In Computer Vision and Pattern Recognition
- Ruizhou Ding
- Di Wang
- Bodhi Priyantha
- Dimitrios Lymberopoulos
- Diana Marculescu
- Dimitrios Stamoulis
- Jie Liu
- Jun Du
Citations in the corpus
- Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization (2019, methodology): “We draw inspiration from weight-based conditions that have been successfully used for quantization-related decisions [36], [37] and we use the group Lasso term .”
- Joint architecture and knowledge distillation in CNN for Chinese text recognition (2019, background): “Different from constraining the weights to +1 or − 1, Ding et al. [38,39] replaced the multiply-accumulate operation with one shift or a constrained number of shifts and adds, which can make trade-offs between accuracy and computational consumption.”