Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks

By Charles Eckert, Xiaowei Wang, Jingcheng Wang, Arun K. Subramaniyan, R. Iyer, D. Sylvester, David T. Blaauw, R. Das. International Symposium on Computer Architecture, 2018

392 citations recorded by the source corpus.

This paper presents the Neural Cache architecture, which re-purposes cache structures to transform them into massively parallel compute units capable of running inferences for Deep Neural Networks. Techniques to do in-situ arithmetic in SRAM arrays, create efficient data mapping and reducing data movement are proposed. The Neural Cache architecture is capable of fully executing convolutional, fully connected, and pooling layers in-cache. The proposed architecture also supports quantization in-cache. Our experimental results show that the proposed architecture can improve inference latency by 8.3× over state-of-art multi-core CPU (Xeon E5), 7.7× over server class GPU (Titan Xp), for Inception v3 model. Neural Cache improves inference throughput by 12.4× over CPU (2.2× over GPU), while reducing power consumption by 50% over CPU (53% over GPU).

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/d0758b9368845a4618dcb4463705b5b5dd085e34

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey (2020, positive): “Besides eNVM devices, traditional memories such as SRAM [270], DRAM [271], and Flash [272] can also be modiﬁed to support the PIM-fashion processing of neural networks.”
Towards spike-based machine intelligence with neuromorphic computing (2019, background): “Most of these works are focused on embedding Boolean bit-wise vector computations inside the memory arrays 120–122 .”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “The traditional SRAM [338], DRAM [339], and Flash [340] memories also can be modiﬁed to realize the PIM implementation of DNN algorithms.”
MnnFast: A Fast and Scalable System Architecture for Memory-Augmented Neural Networks (2019, background): “Processing in memory (PIM) performs read and computation simultaneously while losing reliability [21], which is appealing features for DNN accelerators [19].”
NIPQ: Noise Injection Pseudo Quantization for Automated DNN Optimization (2022, positive): “In particular, linear quantization can easily utilize hardware acceleration via integer arithmetic [3, 4, 5, 6] or bit-serial acceleration [7, 8, 9].”
Planaria: Dynamic Architecture Fission for Spatial Multi-Tenant Acceleration of Deep Neural Networks (2020, background): “There is a large body of work [1, 3, 52– 55, 71–82, 84–88, 91–93, 95, 96, 114–124] for isolated acceleration of DNNs that, although inspired our work, are not focused on multi-tenancy but rather offer various innovations to improve the speed and efficiency of DNN execution.”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, neutral): “…[22] DNN Co-processor NPU [23], DianNao-family [24–27], Cambricon [28], Cambricon-x [29], TPU [30,31], ScaleDeep [32], Stripes [33], Neural Cache [123,124], Diffy [125] FPGA FPGA-DCNN [34], Embedded-FPGA-CNN [35], FPGA-Exploration [36], OpenCL-FPGA-CNN [37] [126], Caffeine [38], DeepBurning…”