Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

By Abhronil Sengupta, Yuting Ye, Robert Y. Wang, Chiao Liu, Kaushik Roy. Frontiers in Neuroscience, 2018

1,229 citations recorded by the source corpus.

Over the past few years, Spiking Neural Networks (SNNs) have become popular as a possible pathway to enable low-power event-driven neuromorphic hardware. However, their application in machine learning have largely been limited to very shallow neural network architectures for simple problems. In this paper, we propose a novel algorithmic technique for generating an SNN with a deep architecture, and demonstrate its effectiveness on complex visual recognition problems such as CIFAR-10 and ImageNet. Our technique applies to both VGG and Residual network architectures, with significantly better accuracy than the state-of-the-art. Finally, we present analysis of the sparse event-driven computations to demonstrate reduced hardware overhead when operating in the spiking domain.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/b20ace3f79c502b29e2e3e0a211880957652e622

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, background): “Most approaches (e.g., Cao et al., 2015; Diehl et al., 2015; Sengupta et al., 2018) circumvented this problem by replacing all pooling operations with average pooling, which is easy to implement in SNNs as a linear operation, but leads to a drop of accuracy.”
Opportunities for neuromorphic computing algorithms and applications (2022, neutral): “,44,46 .”
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks (2020, methodology): “Most previous studies [51, 8, 48] preferred to use the average-pooling in SNNs as they found that max-pooling in SNNs leads to significant information loss.”
RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network (2020, methodology): “ANN-SNN conversion has been shown to be promising approach for building deep SNNs yielding high enough accuracy for complex image recognition [3, 6, 30, 32, 33, 34, 42] and natural language processing tasks [7].”
DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization (2021, methodology): “This leads to adopting a large number of timesteps for high accuracy at the expense of high inference latency [14].”
Spiking Neural Networks and Their Applications: A Review (2022, methodology): “Although some studies have shown SNNs can be used for image classiﬁcation on large datasets such as ImageNet [38,67,68], most applications of SNNs are still limited to less complex datasets such as MNIST [70], N-MNIST [71], and N-Caltech101 [71].”
Deep Residual Learning in Spiking Neural Networks (2021, methodology): “89 4096 Spiking ResNet-34 [49] ANN2SNN 65.”
A review of convolutional neural networks in computer vision (2024, neutral): “To examine the impact of a CNN’s depth on its accuracy, Karen Sengupta et al. (2019) conducted a comprehensive evaluation of the performance of network models with increasing depth using small convolution filters (3 × 3) instead of the previous large convolution kernels (5 × 5) and proposed a series”
Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting (2022, methodology): “Some special techniques have been proposed to reduce the inference latency, such as the subtraction mechanism (Rueckauer et al., 2016; Han et al., 2020), robust normalization Rueckauer et al. (2016), spike-norm (Sengupta et al., 2018), and channel-wise normalization (Kim et al., 2019).”
SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence (2023, methodology): “For quantitative assessment purposes, we evaluated the training and inference performance of modern spiking deep learning frameworks on the Spiking ResNet, which is the spiking version of the classic ResNet ANN structure ( 87 ) and is used frequently by SNN researchers ( 58 , 60 , 62 , 88 , 89 ).”
Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures (2019, methodology): “We also achieved accuracy performance comparable with ANN-SNN converted network (Diehl et al., 2015; Sengupta et al., 2019) for CIFAR-10 dataset while beating all other spikebased training methods.”
Going Deeper With Directly-Trained Larger Spiking Neural Networks (2020, methodology): “The first one is called as the “ANN-SNN conversion methods” (Sengupta et al. 2019; Han, Srinivasan, and Roy 2020), seen as the most popular way to train deep SNNs with high performance, which transforms the real-valued output of ReLU function to binary spikes in the SNN.”
Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation (2020, methodology): “Of all the techniques, conversion from ANN-to-SNN (Diehl et al., 2016; 2015; Sengupta et al., 2019; Hunsberger & Eliasmith, 2015) has yielded state-of-the-art accuracies matching deep ANN performance for Imagenet dataset on complex architectures (such as, VGG (Simonyan & Zisserman, 2014) and ResNet…”
Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection (2019, methodology): “DNN-to-SNN conversion methods, as an alternative approach, have been studied widely in recent years (Cao, Chen, and Khosla 2015; Diehl et al. 2015; Sengupta et al. 2019).”