Trained Ternary Quantization

By Chenzhuo Zhu, Song Han, Huizi Mao, William (Bill) J. Dally. 2016

1,089 citations recorded by the source corpus.

Deep neural networks are widely used in machine learning applications. However, the deployment of large neural networks models can be difficult to deploy on mobile devices with limited power budgets. To solve this problem, we propose Trained Ternary Quantization (TTQ), a method that can reduce the precision of weights in neural networks to ternary values. This method has very little accuracy degradation and can even improve the accuracy of some models (32, 44, 56-layer ResNet) on CIFAR-10 and AlexNet on ImageNet. And our AlexNet model is trained from scratch, which means it's as easy as to train normal full precision model. We highlight our trained quantization method that can learn both ternary values and ternary assignment. During inference, only ternary values (2-bit weights) and scaling factors are needed, therefore our models are nearly 16x smaller than full-precision models. Our ternary models can also be viewed as sparse binary weight networks, which can potentially be accelerated with custom circuit. Experiments on CIFAR-10 show that the ternary models obtained by trained quantization method outperform full-precision models of ResNet-32,44,56 by 0.04%, 0.16%, 0.36%, respectively. On ImageNet, our model outperforms full-precision AlexNet model by 0.3% of Top-1 accuracy and outperforms previous ternary models by 3%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022, neutral): “…different bit-widths have been studied: 1-bit methods (Courbariaux and Bengio, 2016; Rastegari et al., 2016; Courbariaux et al., 2015), 2 to 3-bit (Zhu et al., 2017; Choi et al., 2019), 4-bits (Li et al., 2019), more bits (Courbariaux et al., 2014), or a variable amount of bits (Gong et al.,…”
8-bit Optimizers via Block-wise Quantization (2021, methodology): “, 2015), 2-bit/3 values (Zhu et al., 2017; Choi et al., 2019), 4-bits (Li et al.”
To prune, or not to prune: exploring the efficacy of pruning for model compression (2017, background): “In addition, an emerging area of research is low precision networks where the parameters and/or activations are quantized to 4 bits or fewer (Courbariaux et al., 2015; Lin et al., 2015; Hubara et al., 2016; Rastegari et al., 2016; Zhu et al., 2016).”
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function (2020, background): “Non-architectural modifications: This involves a number of network compression techniques including low-bit quantization [9] to as few as two [49] or even one bit [17].”
Once for All: Train One Network and Specialize it for Efficient Deployment (2019, methodology): “…efﬁcient deep learning, including network pruning that removes redundant units (Han et al., 2015) or redundant channels (He et al., 2018; Liu et al., 2017), and quantization that reduces the bit width for the weights and activations (Han et al., 2016; Courbariaux et al., 2015; Zhu et al., 2017).”
Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search (2018, methodology): “TTQ uses ternary weights (stored by 2 bits) to quantize a network.”
AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, background): “Quantization [55,10,41] and special convolution implementations [36,48,29,3] can also speed up the neural networks.”
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion (2019, background): “For example, one might want to use a smaller or more resource-efficient network architecture while deploying on edge inference devices [44, 66, 76], or to adapt the network to the inference hardware [9, 63, 71], or for continuously learning to classify new image classes [29, 34], etc.”
Bayesian Compression for Deep Learning (2017, methodology): “As an extreme example, [13, 48, 57, 76] and [11] trained NN to use only binary or tertiary weights with floating point gradients.”
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (2017, methodology): “This methodology, embraced by approaches such as Ternary weight networks (TWN [22]), Binary Neural Networks (BNN [14]), XNOR-net [27], and more [8, 21, 26, 33, 34, 35], is the focus of our investigation.”
TSM: Temporal Shift Module for Efficient Video Understanding (2018, background): “Another way is to prune, quantize and compress an existing model for efficient deployment [16, 15, 29, 59, 18, 47].”
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (2018, methodology): “Representative techniques include quantization [6, 34, 29, 54, 52, 21, 39, 53, 22, 3, 55, 9, 18], pruning [13, 12, 17, 36], low-rank decomposition [7, 8, 24, 27, 38, 51, 49], hashing [4], and deliberate architecture design [23, 19, 50].”
ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation (2018, background): “[36] show that there is only a 2% top-5 accuracy loss for ResNet-18 when using a 3-bit representation for weights compared to its full-precision counterpart.”
SqueezeNext: Hardware-Aware Neural Network Design (2018, background): “Based on this observation, there has been many successful attempts to compress a trained network [17, 8, 21, 29].”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Another approach is to learn/impose this clipping range during NN training [36, 144, 268, 278].”
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks (2018, background): “Some works [Zhu et al. , 2017; Zhou et al. , 2017] focus on compressing the CNNs by using low-precision weights .”