Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification

By Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, Shih-Chii Liu. Frontiers in Neuroscience, 2017

1,090 citations recorded by the source corpus.

Spiking neural networks (SNNs) can potentially offer an efficient way of doing inference because the neurons in the networks are sparsely activated and computations are event-driven. Previous work showed that simple continuous-valued deep Convolutional Neural Networks (CNNs) can be converted into accurate spiking equivalents. These networks did not include certain common operations such as max-pooling, softmax, batch-normalization and Inception-modules. This paper presents spiking equivalents of these operations therefore allowing conversion of nearly arbitrary CNN architectures. We show conversion of popular CNN architectures, including VGG-16 and Inception-v3, into SNNs that produce the best results reported to date on MNIST, CIFAR-10 and the challenging ImageNet dataset. SNNs can trade off classification error rate against the number of available operations whereas deep continuous-valued neural networks require a fixed number of operations to achieve their classification error rate. From the examples of LeNet for MNIST and BinaryNet for CIFAR-10, we show that with an increase in error rate of a few percentage points, the SNNs can achieve more than 2x reductions in operations compared to the original CNNs. This highlights the potential of SNNs in particular when deployed on power-efficient neuromorphic spiking neuron chips, for use in embedded applications.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/915cc4b359863f256957485c8a60f2cceb78ab5f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook (2021, methodology): “PyTorch and Tensorﬂow models are supported through the SNN Conversion Toolbox [21], and directly training deep SNNs is supported through SLAYER [22].”
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures (2018, methodology): “Although Rueckauer et al. (2017) reports a top-1 SNN error rate of 25.40% for an Inception-V3 network, their ANN is trained with an error rate of 23.88%.”
Spiking Neural Networks and Their Applications: A Review (2022, methodology): “Although some studies have shown SNNs can be used for image classiﬁcation on large datasets such as ImageNet [38,67,68], most applications of SNNs are still limited to less complex datasets such as MNIST [70], N-MNIST [71], and N-Caltech101 [71].”
Spikformer: When Spiking Neural Network Meets Transformer (2022, methodology): “In ANN-to-SNN conversion (Cao et al., 2015; Hunsberger & Eliasmith, 2015; Rueckauer et al., 2017; Bu et al., 2021; Meng et al., 2022; Wang et al., 2022), the high-performance pre-trained ANN is converted to SNN by replacing the ReLU activation layers with spiking neurons.”
DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization (2021, methodology): “The firing thresholds are also fixed [18] or selected based on some heuristics [14], [28].”
Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures (2019, methodology): “In the literature, several methods have been proposed (Cao et al., 2015; Hunsberger and Eliasmith, 2015; Diehl et al., 2016; Rueckauer et al., 2017; Sengupta et al., 2019) for balancing appropriate ratios between neuronal thresholds and synaptic weights of spiking neuron in the case of ANN-SNN conve”
Event-Based Vision: A Survey (2019, background): “learning on frames), and later convert the trained network to an SNN that processes data event-by-event [133], [134], [135], [136], [137].”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “The high-precision activation function of each neuron is converted into either a spike rate [155], [156], [157], [158], [159] or a latency code [160].”
Deep Residual Learning in Spiking Neural Networks (2021, methodology): “ANN to SNN conversion (ANN2SNN) [20, 4, 46, 49, 12, 11, 6, 54, 33] and backpropagation with surrogate gradient [40] are the two main methods to get deep SNNs.”
A review of learning in biologically plausible spiking neural networks (2019, positive): “In the group of DSNNs, first a classical DNN which is composed of neurons with continuous activation values is trained, then the classical DNN is deployed to a DSNN (Cao, Chen, & Khosla, 2015; Rueckauer, Lungu, Hu, & Pfeiffer, 2017).”
SLAYER: Spike Layer Error Reassignment in Time (2018, neutral): “[25] SNN converted from standard ANN 99.”
Opportunities for neuromorphic computing algorithms and applications (2022, neutral): “,44,46 .”
RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network (2020, methodology): “Note that authors in [33] reported a top-1 SNN error rate of 25.04% for an Inception-V3 network, with their ANN trained to an error rate of 23.88%.”
A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS (2018, methodology): “If high accuracy is needed through global gradient-based optimization with deep multi-layer networks, the weights from an ofﬂine-learned ANN topology can be mapped to an SNN in ODIN using dedicated ANN-to-SNN toolboxes (e.g., [59] or [60]).”
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks (2020, methodology): “ANN to SNN conversion ANN to SNN conversion (ANN2SNN) converts a trained non-spiking ANN to an SNN by using the firing rate of each spiking neuron to approximate the corresponding ReLU activation of an analog neuron [24, 7, 50].”