DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning

By Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, Olivier Temam. 2014

1,660 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Hardware and Architecture, Efficiency & Compression, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Hardware and Architecture

In Efficiency & Compression

In ML Systems & Hardware

Citations in the corpus

Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations (2016, background): “…research efforts are invested in speeding up DNNs at run-time on both generalpurpose (Vanhoucke et al., 2011; Gong et al., 2014; Romero et al., 2014; Han et al., 2015b) and specialized computer hardware (Farabet et al., 2011a,b; Pham et al., 2012; Chen et al., 2014a,b; Esser et al., 2015).”
BinaryConnect: Training Deep Neural Networks with binary weights during propagations (2015, background): “This along, with the drive to put deep learning systems on low-power devices (unlike GPUs) is greatly increasing the interest in research and development of specialized hardware for deep networks [11, 12, 13].”
Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, background): “3) Accelerators: Designing specific hardware and circuits is another active research direction aiming to optimize the energy efficiency [191] and memory footprint [192] of DL”
EIE: Efficient Inference Engine on Compressed Deep Neural Network (2016, methodology): “Previous work has used specialized hardware to accelerate DNNs [10]–[12].”
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients (2016, background): “…light of this, substantial research efforts are invested in speeding up DCNNs at both run-time and training-time, on both general-purpose (Vanhoucke et al., 2011; Gong et al., 2014; Han et al., 2015b) and specialized computer hardware (Farabet et al., 2011; Pham et al., 2012; Chen et al., 2014a;b).”
Demystifying Parallel and Distributed Deep Learning (2018, methodology): “While already applied to inference hardware [35], the first successful instance of reduced precision for training [88] was performed with IEEE 754 16-bit float values (“half precision”).”
ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA (2016, methodology): “DianNao [2] implements an array of multiply-add units to map large DNN onto its core architecture.”
Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices (2018, methodology): “igh throughput while consuming less than 5% of the total energy due to its scalability. VI. RELATED WORK Designing accelerators for DNN has been a very active ﬁeld of research over the past few years [1], [6], [13], [18], [19], [27]. Previous work that explored ﬂexible hardware for DNNs include Flex”
Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications (2017, methodology): “[61], Diannao [12], Dadiannao [14] x x x”
Gist: Efficient Data Encoding for Deep Neural Network Training (2018, methodology): “These weights are ﬁxed after training, and hence many different optimizations can be applied to reduce their storage requirements [17], [2], [18], [5].”
Training deep neural networks with low precision multiplications (2014, background): “FPGA and ASIC implementations claim a better power efficiency than general-purpose hardware (Kim et al., 2009; Farabet et al., 2011; Pham et al., 2012; Chen et al., 2014a;b).”
Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks (2017, background): “A series of accelerator designs have also been proposed for CNNs [58, 59, 60, 61, 62, 63, 64, 65, 66, 24, 67].”
SnaPEA: Predictive Early Activation for Reducing Computation in Deep Convolutional Neural Networks (2018, background): “Several accelerators for convolutional neural networks has been proposed [2], [7]–[10], [15]–[23].”
YodaNN: An Ultra-Low Power Convolutional Neural Network Accelerator Based on Binary Weights (2016, methodology): “For computational complexity the total number of multiplications and additions have been used in other publications like in [36], [11], [33], [8].”