ShiDianNao: Shifting vision processing closer to the sensor

By Zidong Du, Robert Fasthuber, Tianshi Chen, P. Ienne, Ling Li, Tao Luo, Xiaobing Feng, Yunji Chen, Olivier Temam, Olivier Temam. 2015

519 citations recorded by the source corpus.

In recent years, neural network accelerators have been shown to achieve both high energy efficiency and high performance for a broad application scope within the important category of recognition and mining applications. Still, both the energy efficiency and peiformance of such accelerators remain limited by memory accesses. In this paper, we focus on image applications, arguably the most important category among recognition and mining applications. The neural networks which are state-of-the-art for these applications are Convolutional Neural Networks (CNN), and they have an important property: weights are shared among many neurons, considerably reducing the neural network memory footprint. This property allows to entirely map a CNN within an SRAM, eliminating all DRAM accesses for weights. By further hoisting this accelerator next to the image sensor, it is possible to eliminate all remaining DRAM accesses, i.e., for inputs and outputs. In this paper, we propose such a CNN accelerator, placed next to a CMOS or CCD sensor. The absence of DRAM accesses combined with a careful exploitation of the specific data access patterns within CNNs allows us to design an accelerator which is 60x more energy efficient than the previous state-of-the-art neural network accelerator. We present a fult design down to the layout at 65 nm, with a modest footprint of 4.86 mm2 and consuming only 320 mW, but still about 30x faster than high-end GPUs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

Citations in the corpus

• EIE: Efficient Inference Engine on Compressed Deep Neural Network (2016, neutral): “DaDianNao [11] and ShiDianNao [12] eliminate the DRAM access by having all weights on-chip (eDRAM or SRAM).”
• Deep Learning With Edge Computing: A Review (2019, unknown): “ShiDianNao [80] is another recently proposed custom ASIC, which focuses on efficient memory accesses in order to reduce latency and energy consumption.”
• SCNN: An accelerator for compressed-sparse convolutional neural networks (2017, unknown): “Unlike previous convolutional dataflows [6, 8, 12, 28], the SCNN dataflow only delivers to the multiplier array weights and activations that can all be multiplied with one another in the manner of a Cartesian product.”
• An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics (2016, unknown): “Most research architectures for acceleration of CNNs have focused on specialized architectures to accelerate convolutional layers (e.g. Origami [40]), or convolutional and pooling layers (e.g. ShiDianNao [41] and Eyeriss [42]).”
• A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation (2020, unknown): “CNN accelerators for custom ASIC or FPGA device [1], [2], [3], [61], [62], [63], [64], [65], [66], [67] and RNN accelerators [3], [68], [69], [70] are proposed to accelerate NN processing by exploiting massive parallelism and optimizing data movements.”
• MaxNVM: Maximizing DNN Storage Density and Inference Efficiency with Sparse Encoding and Error Mitigation (2019, unknown): “Research concerning specialized on-chip accelerator systems for DNN image classification is prevalent [2, 6, 9, 10, 22, 30, 34, 66, 69].”
• YodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration (2016, unknown): “Even though most approaches work with fixed-point operations, the number of quantization bits is still kept at 24-bit [28], [40] or 16-bit [17], [18], [27], [42], [45].”
• Going Deeper with Embedded FPGA Platform for Convolutional Neural Network (2016, unknown): “In this case, CNN accelerators such as ShiDianNao [10] which store weights on chip are hard to support those large CNN models.”
• TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory (2017, unknown): “Large on-chip SRAM caches or buffers of up to a few MBytes are commonly used to capture the locality in the fmap and filter weight accesses in order to reduce the main memory traffic [8, 12].”
• CompAct (2019, unknown): “To mitigate these issues, there is growing interest in the design of hardware accelerators to speed-up CNN inference from both industry and academia [4, 5, 9, 17, 27, 30].”
• ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA (2016, unknown): “DaDianNao [3] and ShiDianNao [5] eliminate the DRAM access by having all weights on-chip (eDRAM or SRAM).”
• Understanding Reuse, Performance, and Hardware Cost of DNN Dataflow: A Data-Centric Approach (2018, unknown): “However, the energy efficiency in VGG16 (Figure 10 (b)) is worse than YR-P (Eyeriss [11] style) dataflow, and the runtime is worse than YX-P (Shidiannao [15] style) dataflow in UNet ( Figure 10 (e)).”
• A high performance FPGA-based accelerator for large-scale convolutional neural networks (2016, unknown): “References [17] and [18] have stored the weights on chip to reduce the required bandwidth.”
• Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications (2017, unknown): “However, existing DNN accelerators tend to follow a one-sizefits-all approach by deploying a data type representation which is long enough to work for all computations in different layers and DNNs [12, 13, 19].”
• DRISA: A DRAM-based Reconfigurable In-Situ Accelerator (2017, unknown): “An advanced on-chip data reuse scheme as in ShiDian-Nao [28] is adopted.”