A Quantization-Friendly Separable Convolution for MobileNets

By Tao Sheng, Chen Feng, Shaojie Zhuo, Xiaopeng Zhang, Liang Shen, Mickey Aleksic. 2018

126 citations recorded by the source corpus.

As deep learning (DL) is being rapidly pushed to edge computing, researchers invented various ways to make inference computation more efficient on mobile/IoT devices, such as network pruning, parameter compression, and etc. Quantization, as one of the key approaches, can effectively offload GPU, and make it possible to deploy DL on fixed-point pipeline. Unfortunately, not all existing networks design are friendly to quantization. For example, the popular lightweight MobileNetV1, while it successfully reduces parameter size and computation latency with separable convolution, our experiment shows its quantized models have large performance gap against its float point models. To resolve this, we analyzed the root cause of quantization loss and proposed a quantization-friendly separable convolution architecture. By evaluating the image classification task on ImageNet2012 dataset, our modified MobileNetV1 model can archive 8-bit inference top-1 accuracy in 68.03%, almost closed the gap to the float pipeline.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware (2018, background): “Removing these activations improves convergence and accuracy (Chollet, 2017; Sheng et al., 2018), while also making the network more outsourcing-friendly (i.e., it is possible to verify a separable convolution in a single step).”
Quantizing deep convolutional networks for efficient inference: A whitepaper (2018, background): “Note that other approaches like weight regularization can also improve the accuracy of quantization post training, please see [22].”
Relaxed Quantization for Discretized Neural Networks (2018, unknown): “Nevertheless, there is encouraging recent work (Zhang et al., 2018) on non-uniform grids that can be implemented with bit operations.”
UNIQ (2018, methodology): “In the majority of the applications, further reduction of precision quickly degrades performance; hence, non-trivial techniques are required to carry it out.”
AI Benchmark: Running Deep Neural Networks on Android Smartphones (2018, methodology): “It can be optimized further for mobile usage by quantization [28,54] — converting its weights and activations from FLOAT32 to INT8 8-bit fixed point representation.”
Pruning and Quantization for Deep Neural Network Acceleration: A Survey (2021, background): “Weight regularization can slightly improve the accuracy of quantized networks by penalizes weights with large magnitudes [205].”
Data-Free Quantization Through Weight Equalization and Bias Correction (2019, methodology): “Other methods to improve quantization need architecture changes or training with quantization in mind from the start [1, 21, 31, 33, 36].”