Any-Precision Deep Neural Networks

By Haichao Yu, Haoxiang Li, Humphrey Shi, Thomas S. Huang, Gang Hua. 2019

80 citations recorded by the source corpus.

We present Any-Precision Deep Neural Networks (Any- Precision DNNs), which are trained with a new method that empowers learned DNNs to be flexible in any numerical precision during inference. The same model in runtime can be flexibly and directly set to different bit-width, by trun- cating the least significant bits, to support dynamic speed and accuracy trade-off. When all layers are set to low- bits, we show that the model achieved accuracy compara- ble to dedicated models trained at the same precision. This nice property facilitates flexible deployment of deep learn- ing models in real-world applications, where in practice trade-offs between model accuracy and runtime efficiency are often sought. Previous literature presents solutions to train models at each individual fixed efficiency/accuracy trade-off point. But how to produce a model flexible in runtime precision is largely unexplored. When the demand of efficiency/accuracy trade-off varies from time to time or even dynamically changes in runtime, it is infeasible to re-train models accordingly, and the storage budget may forbid keeping multiple models. Our proposed framework achieves this flexibility without performance degradation. More importantly, we demonstrate that this achievement is agnostic to model architectures. We experimentally validated our method with different deep network backbones (AlexNet-small, Resnet-20, Resnet-50) on different datasets (SVHN, Cifar-10, ImageNet) and observed consistent results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Le Yang
5. Yizeng Han
6. Honghui Wang
7. Quoc V. Le
8. Alan L. Yuille

In Computer Vision and Pattern Recognition

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Yizeng Han
5. Le Yang
6. Honghui Wang
7. Luc Van Gool
8. Alan L. Yuille

In Efficiency & Compression

1. Shiji Song
2. Gao Huang
3. Yulin Wang
4. Le Yang
5. Yizeng Han
6. Honghui Wang

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, background): “Instead of either skipping a layer, or executing it thoroughly with a full numerical precision, a line of work [68], [69] studies adaptive bit-width for different layers conditioned on the resource budget.”
• Dynamic Network Quantization for Efficient Video Inference (2021, unknown): “To tackle this problem, we adopt any-precision recognition [27, 64] that makes a single model be flexible to any numerical precision during the inference.”