Fully Quantized Network for Object Detection

By Rundong Li, Yan Wang, Feng Liang, Hongwei Qin, Junjie Yan, Rui Fan. 2019

229 citations recorded by the source corpus.

Efficient neural network inference is important in a number of practical domains, such as deployment in mobile settings. An effective method for increasing inference efficiency is to use low bitwidth arithmetic, which can subsequently be accelerated using dedicated hardware. However, designing effective quantization schemes while maintaining network accuracy is challenging. In particular, current techniques face difficulty in performing fully end-to-end quantization, making use of aggressively low bitwidth regimes such as 4-bit, and applying quantized networks to complex tasks such as object detection. In this paper, we demonstrate that many of these difficulties arise because of instability during the fine-tuning stage of the quantization process, and propose several novel techniques to overcome these instabilities. We apply our techniques to produce fully quantized 4-bit detectors based on RetinaNet and Faster R-CNN, and show that these achieve state-of-the-art performance for quantized detectors. The mAP loss due to quantization using our methods is more than 3.8x less than the loss from existing methods.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022, neutral): “…different bit-widths have been studied: 1-bit methods (Courbariaux and Bengio, 2016; Rastegari et al., 2016; Courbariaux et al., 2015), 2 to 3-bit (Zhu et al., 2017; Choi et al., 2019), 4-bits (Li et al., 2019), more bits (Courbariaux et al., 2014), or a variable amount of bits (Gong et al., 2019).”
8-bit Optimizers via Block-wise Quantization (2021, methodology): “, 2019), 4-bits (Li et al., 2019), more bits (Courbariaux et al.”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Another approach is to learn/impose this clipping range during NN training [36, 144, 268, 278].”
Forward and Backward Information Retention for Accurate Binary Neural Networks (2019, background): “…(DNNs), especially convolutional neural networks (CNNs), have been well demonstrated in a ∗ Corresponding author wide variety of computer vision applications such as image classiﬁcation [30, 46, 47, 49, 50, 56, 63, 53], object detection [17, 16, 42, 45, 34] and semantic segmentation [15, 65].”
Binary Neural Networks: A Survey (2020, background): “…binary neural network, in the past years a large amount of research has been attracted on this topic from the ﬁelds of computer vision and machine learning [1, 2, 12, 28], and has been applied to various popular tasks such as image classiﬁcation[59, 60, 61, 62, 63], detection [64, 65], and so on.”
ZeroQ: A Novel Zero Shot Quantization Framework (2020, methodology): “Our results are comparable to the recent results of FQN [25], even though it is not a zero-shot quantization method (i.e., it uses the full training dataset and requires ﬁne-tuning).”
Post-Training Quantization for Vision Transformer (2021, methodology): “For the similar quantized model size, the proposed method outperforms percentile-based method [18] by 3.”
Distilling Object Detectors via Decoupled Features (2021, background): “Techniques such as quantization [19, 58, 31, 57, 62], pruning [2, 17, 20], network design [55, 49, 15, 18] and knowledge distillation [56, 6] have been developed to overcome this dilemma and achieve an efficient inference on detection task.”
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks (2019, methodology): “Using HAWQ-V2 on mixed-precision weight quantization with uniform 4-bit activations can achieve a state-of-the-art performance of 34.1 mAP, which is 1.6 mAP higher than [13] with an even smaller model size.”
Towards Unified INT8 Training for Convolutional Neural Network (2019, methodology): “As for the hyperparameters, we follow the same rules described in [35].”
CoDeNet: Efficient Deployment of Input-Adaptive Object Detection on Embedded FPGAs (2021, methodology): “Even with quantization-aware fine-tuning, quantizing the detection models with naive quantization schemes can cause around 10% AP degradation on the COCO dataset [19].”