Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing

By Peter U. Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, Michael Pfeiffer. IEEE International Joint Conference on Neural Network, 2015

1,067 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/1cb3a0ff57de3199fe7029db7a1b8bac310fa5f8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization (2021, methodology): “1) begins with training an ANN without the bias term and batch normalization to achieve minimal loss during ANN-to-SNN conversion [13], [14], [23].”
Spiking Deep Networks with LIF Neurons (2015, background): “While one goal of this work is to better understand the brain by trying to reverse engineer it [9], another goal is to build energy-efﬁcient neuromorphic systems that use a similar communication method for image categorization [12, 13].”
Towards spike-based machine intelligence with neuromorphic computing (2019, background): “Recent works 29–31 have proposed practical solutions to determine optimal firing rates, and additional constraints (such as noise or leaky ReLUs) are introduced during training of the DLN to better match the spiking neuron’s firing rate 36 .”
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures (2018, methodology): “ARCHITECTURES: VGG As mentioned previously, our work is based on the proposal outlined by authors in Diehl et al. (2015) wherein the neuron threshold of a particular layer is set equal to the maximum activation of all ReLUs in the corresponding layer (by passing the entire training set through the…”
STDP-based spiking deep neural networks for object recognition (2016, methodology): “Besides, the accuracy of the proposed SDNN is close to the 99.1% accuracy of the totally supervised ratebased SDNN (Diehl et al., 2015) which is indeed the converted version of a traditional DCNN trained by back-propagation.”
Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting (2022, methodology): “t the ANN activation value layer-by-layer (Diehl et al., 2015; 2016).”
Biologically Inspired Sleep Algorithm for Reducing Catastrophic Forgetting in Neural Networks (2020, methodology): “To use this SNN advantage, we utilized an existing algorithm for converting an ANN to SNN (Diehl et al. 2015) and stimulated the SNN with noisy versions of the average input observed during normal training while updating weights during the sleep phase (see pseudocode).”
Event-Based Vision: A Survey (2019, background): “Many approaches use packets of events during training (deep learning on frames), and later convert the trained network to an SNN that processes data event-by-event [121], [122], [123], [124], [125].”
Training Deep Spiking Neural Networks Using Backpropagation (2016, methodology): “Diehl et al. (2015) trained deep neural networks with conventional deep learning techniques and additional constraints necessary for conversion to SNNs.”
Effective sensor fusion with event-based sensors and deep network architectures (2016, methodology): “In the second set of experiments, the frame-based networks were converted to spiking networks as described in [9] and these networks were then tested on spiking inputs.”
Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification (2017, methodology): “We discuss next two types of reset: reset to zero, used e.g., in Diehl et al. (2015), always sets the membrane potential back to a baseline, typically zero.”
Unsupervised learning of digit recognition using spike-timing-dependent plasticity (2015, methodology): “One popular approach is to still rely on backpropagation training but afterwards converting the ANN into a spiking neural network (SNN), which we will call “rate-based learning” (Merolla et al., 2011; O’Connor et al., 2013; Hussain et al., 2014; Neil and Liu, 2014; Diehl et al., 2015).”
A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs) (2017, background): “It has been recently shown [33], [35] that if these networks are designed”
Biologically inspired sleep algorithm for increased generalization and adversarial robustness in deep neural networks (2020, methodology): “Recent work has shown that a near loss-less conversion between ANNs and SNNs can be achieved by propagating activity through a spiking neural network for a given input and counting the number of times that each output neuron ﬁres (Diehl et al. (2015)).”
Going Deeper With Directly-Trained Larger Spiking Neural Networks (2020, methodology): “For SNNs, researchers propose other normalization techniques, such as data-based normalization (Diehl et al. 2015), Spike-Norm (Sengupta et al. 2019) and NeuNorm (Wu et al. 2019).”
Spiking Neural Networks and Their Applications: A Review (2022, background): “In subsequent work, reference [66] proposed data-based normalization to improve the performance in deep SNNs.”
Biologically inspired sleep algorithm for artificial neural networks (2019, methodology): “ on a task. The ANN consisted of ReLU activation units to create positive ﬁring rates and no bias. We used a previously developed algorithm to convert the architecture in the FCN to an equivalent SNN [24]. In short, the weights are transferred directly to the SNN, which consists of leaky integrate a”