Binary Neural Networks: A Survey

By Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, Nicu Sebe. 2020

577 citations recorded by the source corpus.

Abstract The binary neural network, largely saving the storage and computation, serves as a promising technique for deploying deep models on resource-limited devices. However, the binarization inevitably causes severe information loss, and even worse, its discontinuity brings difficulty to the optimization of the deep network. To address these issues, a variety of algorithms have been proposed, and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these algorithms, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing the quantization error, improving the network loss function, and reducing the gradient error. We also investigate other practical aspects of binary neural networks such as the hardware-friendly design and the training tricks. Then, we give the evaluation and discussions on different tasks, including image classification, object detection and semantic segmentation. Finally, the challenges that may be faced in future research are prospected.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022, neutral): “For additional related work, please see the survey of Qin et al. (2020).”
8-bit Optimizers via Block-wise Quantization (2021, background): “See also Qin et al. (2020) for a survey on binary neural networks.”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Here, we brieﬂy discuss each branch, and we refer the interested readers to [191] for more details. a) Quantization Error Minimization: The ﬁrst branch of solutions aims to minimize the quantization error, i.e., the gap between the real values and the quantized values [19, 34, 62, 102, 149, 155,…”
Forward and Backward Information Retention for Accurate Binary Neural Networks (2019, background): “Although much progress has been made on bina-rizing DNNs, the existing quantization methods remains a signiﬁcant drop of accuracy compared with the full-precision counterparts [14, 39, 2, 28, 57, 48, 9, 40, 43].”
Pruning and Quantization for Deep Neural Network Acceleration: A Survey (2021, background): “Simons [206] and Qin [188] provide an in-depth review of BNNs.”