FINN: A Framework for Fast, Scalable Binarized Neural Network Inference

By Yaman Umuroglu, Nicholas J. Fraser, Giulio Gambardella, Michaela Blott, Philip Heng Wai Leong, Magnus Jahre, Kees A. Vissers. Symposium on Field Programmable Gate Arrays, 2016

1,186 citations recorded by the source corpus.

Research has shown that convolutional neural networks contain significant redundancy, and high classification accuracy can be obtained even when weights and activations are reduced from floating point to binary values. In this paper, we present FINN, a framework for building fast and flexible FPGA accelerators using a flexible heterogeneous streaming architecture. By utilizing a novel set of optimizations that enable efficient mapping of binarized neural networks to hardware, we implement fully connected, convolutional and pooling layers, with per-layer compute resources being tailored to user-provided throughput requirements. On a ZC706 embedded FPGA platform drawing less than 25 W total system power, we demonstrate up to 12.3 million image classifications per second with 0.31 μs latency on the MNIST dataset with 95.8% accuracy, and 21906 image classifications per second with 283 μs latency on the CIFAR-10 and SVHN datasets with respectively 80.1% and 94.9% accuracy. To the best of our knowledge, ours are the fastest classification rates reported to date on these benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/3b2491ddeeaa7beae4d311b217c292a9e16112cf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

TVM: An Automated End-to-End Optimizing Compiler for Deep Learning (2018, background): “Figure 21 breaks down ResNet inference time into CPU-only execution and CPU+FPGA execution.”
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, methodology): “FINN outperforms the design by Ovtcharov et al. [152] by over 13.8 ˆ for throughput.”
A survey of FPGA-based accelerators for convolutional neural networks (2018, methodology): “Hence, on a typical FPGA, the FxP CNN becomes bound by the memory BW, whereas the BNN nearly reaches the peak of its computational performance [17].”
Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network (2017, methodology): “Several inspiring works have explored ASIC and FPGA accelerators optimized for Binary DNNs. FINN [47] use FPGAs for accelerating Binary DNNs, while YodaNN [48] and BRein [49] propose an ASIC accelerator for Binary DNNs. Kim, et al. [50] decompose the convolution weights for binary CNNs to improve pe”
FINN-R (2018, methodology): “ction. The authors also created a tool to compile a network description into host code which is used to call the CNP. Accelerator frameworks: Numerous new frameworks, including the original FINN tool [58], have been proposed that take a graph based description of neural networks (such as Caffe’s pro”
Ternary Neural Networks with Fine-Grained Quantization (2017, background): “The large and somewhat unique compute requirements for both deep learning training and inference operations motivate the use of customized low precision arithmetic [9, 2, 8, 24, 15, 13] and specialized hardware to run these computations as efﬁciently as possible [5, 25, 21, 19, 11].”
SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks (2018, methodology): “The layer-wise baseline uses no multiplies, as these can absorb into quantization thresholds for activations [29].”
A High-Throughput and Power-Efficient FPGA Implementation of YOLO CNN for Object Detection (2019, methodology): “Therefore, most of the previous FPGA designs for binary neural networks (BNNs) are validated using just tiny data set, such as Cifar-10 or MNIST [19], [20], [30], [31].”
Heterogeneous Bitwidth Binarization in Convolutional Neural Networks (2018, methodology): “There have been several recent binary convolutional neural network implementations on FGPAs and ASICs that provide a baseline we can use to estimate the performance of HBNNs on ZC706 FPGA platforms (Umuroglu et al., 2017) and on ASIC hardware (Alemdar et al., 2017).”
NICE: Noise Injection and Clamping Estimation for Neural Network Quantization (2018, methodology): “…complex scheduling and data movement, FPGA-based inference accelerators has been described as heterogeneous system using OpenCL (Wang et al., 2016a;b; Aydonat et al., 2017) or as standalone accelerator using HLS compilers (Ghaffari & Shariﬁan, 2016; Umuroglu et al., 2017; Zhao et al., 2017).”
Pruning and Quantization for Deep Neural Network Acceleration: A Survey (2021, methodology): “When deploying on the Zynq7000 SoC, FINN has achieved up to 12.36 million images per second on the MNIST dataset with 4.17% loss of accuracy.”