Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks

By Wei Fang, Zhaofei Yu, Yanqing Chen, Timothée Masquelier, Tiejun Huang, Yonghong Tian. IEEE International Conference on Computer Vision, 2020

776 citations recorded by the source corpus.

Spiking Neural Networks (SNNs) have attracted enormous research interest due to temporal information processing capability, low power consumption, and high biological plausibility. However, the formulation of efficient and high-performance learning algorithms for SNNs is still challenging. Most existing learning methods learn weights only, and require manual tuning of the membrane-related parameters that determine the dynamics of a single spiking neuron. These parameters are typically chosen to be the same for all neurons, which limits the diversity of neurons and thus the expressiveness of the resulting SNNs. In this paper, we take inspiration from the observation that membrane-related parameters are different across brain regions, and propose a training algorithm that is capable of learning not only the synaptic weights but also the membrane time constants of SNNs. We show that incorporating learnable membrane time constants can make the network less sensitive to initial values and can speed up learning. In addition, we reevaluate the pooling methods in SNNs and find that max-pooling will not lead to significant information loss and have the advantage of low computation cost and binary compatibility. We evaluate the proposed method for image classification tasks on both traditional static MNIST, Fashion-MNIST, CIFAR-10 datasets, and neuromorphic N-MNIST, CIFAR10-DVS, DVS128 Gesture datasets. The experiment results show that the proposed method outperforms the state-of-the-art accuracy on nearly all datasets, using fewer time-steps. Our codes are available at https://github.com/fangwei123456/Parametric-Leaky-Integrate-and-Fire-Spiking-Neuron.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/445fd15f64650d2cfd58833ec6642235888ee584

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

1. Yonghong Tian
2. Federico Corradi
3. Shuicheng Yan
4. Lei Deng
5. Guoqi Li
6. Zhaofei Yu
7. Sander M. Bohte
8. Wei Fang

In Cognitive Neuroscience

1. Yonghong Tian
2. Guoqi Li
3. Liuliang Yuan
4. Man Yao
5. Timothée Masquelier
6. Zhaofei Yu
7. Wei Fang
8. Yanqing Chen

In Electrical and Electronic Engineering

1. Yonghong Tian
2. Guoqi Li
3. Liuliang Yuan
4. Man Yao
5. Timothée Masquelier
6. Zhaofei Yu
7. Wei Fang
8. Yanqing Chen

Citations in the corpus

• Spikformer: When Spiking Neural Network Meets Transformer (2022, methodology): “…neural networks (ANNs), SNNs are able to lift performance by borrowing advanced architectures from ANNs, such as ResNet-like SNNs (Hu et al., 2021a; Fang et al., 2021a; Zheng et al., 2021; Hu et al., 2021b), Spiking Recurrent Neural Networks (Lotfi Rezaabad & Vishwanath, 2020) and Spiking Graph…”
• Deep Residual Learning in Spiking Neural Networks (2021, methodology): “The accuracy of our Wide-7B-Net is slightly lower than the current SOTA method [8] (74.4% v.s. 74.8%), which uses 1.25 times as many simulation time-steps T (20 v.s. 16) and 14.6 times as many the number of parameters (17.4M v.s. 1.19M).”
• Attention Spiking Neural Networks (2022, methodology): “ing [40]) and five-layer TCSA-SNN (following [52]) increase the energy efficiency by up to 3.”
• Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks (2021, background): “In prelimi-nary work, the use of conventional convolutional network layers to extract useful features proved helpful for simple subsequent layers of spiking neurons 35 .”