Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network

By Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Joon Kyung Kim, Vikas Chandra, Hadi Esmaeilzadeh. International Symposium on Computer Architecture, 2017

561 citations recorded by the source corpus.

Hardware acceleration of Deep Neural Networks (DNNs) aims to tame their enormous compute intensity. Fully realizing the potential of acceleration in this domain requires understanding and leveraging algorithmic properties of DNNs. This paper builds upon the algorithmic insight that bitwidth of operations in DNNs can be reduced without compromising their classification accuracy. However, to prevent loss of accuracy, the bitwidth varies significantly across DNNs and it may even be adjusted for each layer individually. Thus, a fixed-bitwidth accelerator would either offer limited benefits to accommodate the worst-case bitwidth requirements, or inevitably lead to a degradation in final accuracy. To alleviate these deficiencies, this work introduces dynamic bit-level fusion/decomposition as a new dimension in the design of DNN accelerators. We explore this dimension by designing Bit Fusion, a bit-flexible accelerator, that constitutes an array of bit-level processing elements that dynamically fuse to match the bitwidth of individual DNN layers. This flexibility in the architecture enables minimizing the computation and the communication at the finest granularity possible with no loss in accuracy. We evaluate the benefits of Bit Fusion using eight real-world feed-forward and recurrent DNNs. The proposed microarchitecture is implemented in Verilog and synthesized in 45 nm technology. Using the synthesis results and cycle accurate simulation, we compare the benefits of Bit Fusion to two state-of-the-art DNN accelerators, Eyeriss and Stripes. In the same area, frequency, and process technology, Bit Fusion offers 3.9x speedup and 5.1x energy savings over Eyeriss. Compared to Stripes, Bit Fusion provides 2.6x speedup and 3.9x energy reduction at 45 nm node when Bit Fusion area and frequency are set to those of Stripes. Scaling to GPU technology node of 16 nm, Bit Fusion almost matches the performance of a 250-Watt Titan Xp, which uses 8-bit vector instructions, while Bit Fusion merely consumes 895 milliwatts of power.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/69e220145b5a7886f9c92da8a253b6cc97181f57

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, background): “The PEs in the bit fusion [126] architecture are combined differently to implement spatial ﬂexibility in bit widths.”
MASR: A Modular Accelerator for Sparse RNNs (2019, methodology): “We then simplify the neural networks to establish an efﬁcient baseline RNN for ASR.”
Boosting the Performance of CNN Accelerators with Dynamic Fine-Grained Channel Gating (2019, background): “Bit Fusion [38] proposes to reduce the computational cost by choosing different bit widths dynamically which is also applicable to CGNet.”
Planaria: Dynamic Architecture Fission for Spatial Multi-Tenant Acceleration of Deep Neural Networks (2020, background): “BitFusion [88] explores bit-level dynamic composability in its multipliers to support heterogeneity in deeply quantized neural networks.”
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing (2019, unknown): “patterns found in fully-connected (FC), convolution (CNN), and recurrent (RNN) neural networks [3], [6]–[8], [11], [12], [15], [17], [18], [24], [33], [36], [40], [47], [53]–[55], [57], [60], [68], [71]–[74], [76], [79], [82].”
HAQ: Hardware-Aware Automated Quantization With Mixed Precision (2018, methodology): “Besides industry, recently academia also works on the bit-level flexible hardware design: BISMO [27] proposed the bit-serial multiplier to support multiplications of 1 to 8 bits; BitFusion [26] supports multiplications of 2, 4, 8 and 16 bits in a spatial manner.”
HAQ: Hardware-Aware Automated Quantization (2018, methodology): “[25] is a state-of-the-art spatial ASIC design for neural network accelerator.”
UNIQ (2018, background): “Sharma et al. [28] propose dynamic bit-level fusion/decomposition that constitutes an array of bit-level processing elements that dynamically fuse to match the bitwidth of individual DNN layers.”
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction (2021, background): “With the support of scalable function-unit (Sharma et al., 2018), the peak performance of the accelerator is able to achieve corresponding linear improvement as the precision decreases.”
Machine Learning at Facebook: Understanding Inference at the Edge (2019, background): “Recent work on hardware for machine learning and efficient training and inference has substantially advanced the state-of-the-art [41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66].”
HAWQV3: Dyadic Neural Network Quantization (2020, methodology): “…et al., 2020; Chin et al., 2020; Dong et al., 2019; Hubara et al., 2016; Jacob et al., 2018; Kim & Kim, 2021; Park et al., 2018a; Rastegari et al., 2016; Sharma et al., 2018; Song et al., 2020; Zhang et al., 2018; Zhou et al., 2017a; 2016), where low-bit precision is used for weights/activations.”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, background): “Lightweight models are also introduced [64–70] to reduce computational effort.”
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications (2018, background): “present a systolic array accelerator called BitFusion which supports variable bit precision [61].”
DeepAttest: An End-to-End Attestation Framework for Deep Neural Networks (2019, background): “Methodological and architecture-level advancements have been proposed to improve the performance and efficiency of DNN training/execution on diverse platforms [8, 18, 41].”
Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey (2020, methodology): “Two ways are utilized in this context: bit-serial [12], [278] and bit-decomposed [279], which are depicted in Figs.”