Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

By Benoit Jacob, Skirmantas Kligys, Menglong Zhu, Matthew Tang, Andrew G. Howard, Hartwig Adam, Dmitry Kalenichenko. 2017

4,475 citations recorded by the source corpus.

The rising popularity of intelligent mobile devices and the daunting computational cost of deep learning-based models call for efficient and accurate on-device inference schemes. We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficiently than floating point inference on commonly available integer-only hardware. We also co-design a training procedure to preserve end-to-end model accuracy post quantization. As a result, the proposed quantization scheme improves the tradeoff between accuracy and on-device latency. The improvements are significant even on MobileNets, a model family known for run-time efficiency, and are demonstrated in ImageNet classification and COCO detection on popular CPUs.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Yi Yang

In Electrical and Electronic Engineering

  1. Young Geun Kim
  2. Song Han
  3. Carole-Jean Wu
  4. Tim Dettmers
  5. Luke S. Zettlemoyer
  6. Quoc V. Le
  7. Kurt Keutzer
  8. Raghuraman Krishnamoorthi

In Efficiency & Compression

  1. Song Han
  2. Kurt Keutzer
  3. Dan Alistarh
  4. Quoc V. Le
  5. Luke S. Zettlemoyer
  6. Tijmen Blankevoort
  7. A. Gholami
  8. Michael W. Mahoney

Citations in the corpus