DaDianNao: A Machine-Learning Supercomputer

By Yunji Chen, Tao Luo, Shaoli Liu, Shijin Zhang, Liqiang He, Jia Wang, Ling Li, Tianshi Chen, Zhiwei Xu, Ninghui Sun and 1 more. 2014

1,438 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Hardware and Architecture, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Hardware and Architecture

In ML Systems & Hardware

Citations in the corpus

In-datacenter performance analysis of a tensor processing unit (2017, background): “Relative GPU and TPU includes host server overhead.”
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations (2016, background): “…research eﬀorts are invested in speeding up DNNs at run-time on both general-purpose (Vanhoucke et al., 2011; Gong et al., 2014; Romero et al., 2014; Han et al., 2015b) and specialized computer hardware (Farabet et al., 2011a,b; Pham et al., 2012; Chen et al., 2014a,b; Esser et al., 2015).”
Deep Learning with Limited Numerical Precision (2015, methodology): “A recent work (Chen et al., 2014) presents a hardware accelerator for deep neural network training that employs ﬁxed-point computation units, but ﬁnds it necessary to use 32-bit ﬁxed-point representation to achieve convergence while training a convolutional neural network on the MNIST dataset.”
BinaryConnect: Training Deep Neural Networks with binary weights during propagations (2015, background): “This along, with the drive to put deep learning systems on low-power devices (unlike GPUs) is greatly increasing the interest in research and development of specialized hardware for deep networks [11, 12, 13].”
Binarized Neural Networks (2016, background): “…research efforts are invested in speeding up DNNs at run-time on both general-purpose (Vanhoucke et al., 2011; Gong et al., 2014; Romero et al., 2014; Han et al., 2015) and specialized computer hardware (Farabet et al., 2011a;b; Pham et al., 2012; Chen et al., 2014a;b; Esser et al., 2015).”
EIE: Efficient Inference Engine on Compressed Deep Neural Network (2016, background): “DaDianNao [11] and ShiDianNao [12] eliminate the DRAM access by having all weights on-chip (eDRAM or SRAM).”
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients (2016, background): “ at both run-time and training-time, on both general-purpose (Vanhoucke et al., 2011; Gong et al., 2014; Han et al., 2015b) and specialized computer hardware (Farabet et al., 2011; Pham et al., 2012; Chen et al., 2014a;b). Various approaches like quantization (Wu et al., 2015) and sparsiﬁcation (Han”
Training deep neural networks with low precision multiplications (2014, background): “ted deep learning hardware is appearing as well. FPGA and ASIC implementations claim a better power efﬁciency than general-purpose hardware (Kim et al., 2009; Farabet et al., 2011; Pham et al., 2012; Chen et al., 2014a;b). In contrast with general-purpose hardware, dedicated hardware such as ASIC an”
Low precision storage for deep learning (2014, background): “FPGA and ASIC implementations claim a better power efﬁciency than general-purpose hardware (Kim et al. , 2009; Chen et al. , 2014a;b).”
RETROSPECTIVE: Cnvlutin: Ineffectual-Neuron-Free Deep Neural Network Computing (2023, methodology): “Our design, refined over numerous in-depth meetings, was based on DaDianNao’s processing element [1] which exploited spatial and temporal reuse.”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, neutral): “There is another architecture for machine learning called “Dadiannao” [279].”
Simba: Scaling Deep-Learning Inference with Multi-Chip-Module-Based Architecture (2019, methodology): “towards even larger and deeper networks, the ensuing compute and storage requirements motivate large-scale compute capability in DL hardware, which is currently addressed by a combination of large monolithic chips and homogeneous multi-chip board designs [14, 17, 24, 29, 39, 71].”
Maximizing CNN accelerator efficiency through resource partitioning (2016, neutral): “Similar designs are used in [4, 5].”
Virtualizing Deep Neural Networks for Memory-Efficient Neural Network Design (2016, methodology): “ our work are: This work is the ﬁrst to present a detailed, quantitative analysis on GPU-based DNN training, as opposed to recent literature targeting energy-efﬁcient accelerators for DNN inference [20, 21, 22, 23, 24, 25, 26, 27, 28, 29]. To the best of our knowledge, our work is the ﬁrst that pr”
From high-level deep neural models to FPGAs (2016, neutral): “For example, (Da)Diannao [12, 13] provide DNN accelerators with a low-level fine-grained ISA, yet they do not define an ISA to unify DNN accelerators.”
TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory (2017, background): “Early designs [5, 6, 47] used parallel inner-product engines which could be easily programmed for different layer types.”
OD-SGD (2020, background): “Besides, reducing the overhead for accessing main memory also contributes to computation overhead reduction, Dadiannao [26] greatly accelerates the computing speed by introducing more on-chip storage, leading to much fewer accesses to main memory, while Prime [27] takes advantage of…”
ASIC Clouds: Specializing the Datacenter (2016, methodology): “Finally, our DaDianNao-style [22] Convolutional Neural Network ASIC Clouds make use of on-ASIC eDRAM and HyperTransport links between ASICs to scale to large multichip CNN accelerators.”
FlexFlow: A Flexible Dataflow Accelerator Architecture for Convolutional Neural Networks (2017, background): “For example, Systolic architectures [1], [7] can only exploit synapse parallelism, 2D-Mapping architectures [6] neuron parallelism, and Tiling architectures [2], [3] feature map parallelism.”