Unsupervised learning of digit recognition using spike-timing-dependent plasticity

By Peter U. Diehl, Matthew Cook. Frontiers in Computational Neuroscience, 2015

1,436 citations recorded by the source corpus.

In order to understand how the mammalian neocortex is performing computations, two things are necessary; we need to have a good understanding of the available neuronal processing units and mechanisms, and we need to gain a better understanding of how those mechanisms are combined to build functioning systems. Therefore, in recent years there is an increasing interest in how spiking neural networks (SNN) can be used to perform complex computations or solve pattern recognition tasks. However, it remains a challenging task to design SNNs which use biologically plausible mechanisms (especially for learning new patterns), since most such SNN architectures rely on training in a rate-based network and subsequent conversion to a SNN. We present a SNN for digit recognition which is based on mechanisms with increased biological plausibility, i.e., conductance-based instead of current-based synapses, spike-timing-dependent plasticity with time-dependent weight change, lateral inhibition, and an adaptive spiking threshold. Unlike most other systems, we do not use a teaching signal and do not present any class labels to the network. Using this unsupervised learning scheme, our architecture achieves 95% accuracy on the MNIST benchmark, which is better than previous SNN implementations without supervision. The fact that we used no domain-specific knowledge points toward the general applicability of our network design. Also, the performance of our network scales well with the number of neurons used and shows similar performance for four different learning rules, indicating robustness of the full combination of mechanisms, which suggests applicability in heterogeneous biological neural networks.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/713d600bc941cb0625873f946a4b2a7fbe25ed4c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Direct Training for Spiking Neural Networks: Faster, Larger, Better (2018, methodology): “Recently, several normalization methods (e.g. model-based normalization (Diehl et al. 2015), data-based normalization (Diehl et al. 2015 (Bower and Beeman 1998) mainly focus on the biological realistic simulation from neuron functionality to synapse reaction, which are more beneﬁcial for the…”
STDP-based spiking deep neural networks for object recognition (2016, methodology): “The second group consists of SNNs with bioinspired learning rules which have shallow architectures (Brader, Senn, & Fusi, 2007; Diehl & Cook, 2015; Habenschuss, Bill, & Nessler, 2012; Querlioz, Bichler, Dollfus, & Gamrat, 2013; Zhao, Ding, Chen, Linares-Barranco, & Tang, 2015) or only one trainable…”
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks (2017, methodology): “For instance, only recently have some groups begun to train SNNs on datasets such as MNIST (Diehl and Cook, 2015; Guergiuev et al., 2016; Neftci et al., 2016; Petrovici et al., 2017), whereas most previous studies have used smaller artiﬁcial datasets.”
Training Deep Spiking Neural Networks Using Backpropagation (2016, methodology): “ devise learning rules operating directly on spike trains, but so far it has only been possible to train single layers, and use unsupervised learning rules, which leads to a deterioration of accuracy [3, 18, 20]. An alternative approach has recently been introduced by [23], in which a SNN learns fro”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “The more “natural” approach to online learning is perceived to be via unsupervised learning with synaptic plasticity rules, such as STDP [33], [206] and variants of STDP (see Appendix C2) [207], [208], [209], [210].”
Opportunities for neuromorphic computing algorithms and applications (2022, background): “Given that graphs are a special class of objects called relational structures, foundational work of Cook 93 on relational structures has proven to be compatible with neuromorphic hardware, finding application to learning in cortical networks 94 and unsupervised learning tasks 95 .”
Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing (2015, neutral): “STDP-trained network [31] None 95.”
Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection (2019, methodology): “Although STDP is biologically more plausible, the learning performance is signiﬁcantly lower than that of supervised learning.”
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures (2018, methodology): “However, such a “Model-Based Normalization” technique is highly pessimistic since all the fan-in neurons are not supposed to fire at every time-step (Diehl et al., 2015).”
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks (2020, methodology): “Existing approaches exploited the self-organization principle [56, 11, 29], and STDP-based expectation-maximization algorithm [43, 17].”
Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks (2017, neutral): “66% Spiking MLP(STDP) [15] 784-6400 None 95.”
Deep Learning in Spiking Neural Networks (2018, background): “[130] showed that STDP in a two-layer SNN is able to extract discriminative features and patterns from stimuli.”
RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network (2020, methodology): “STDP-based unsupervised [5, 23, 37, 40] and semi-supervised learning algorithms [17, 20, 25, 39] have thus far been restricted to shallow SNNs (with ≤ 5 layers) yielding considerably lower than acceptable accuracy on complex datasets like CIFAR-10 [9, 38].”
Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures (2019, methodology): “STDP-trained two-layer network (consisting of 6,400 output neurons) has been shown to achieve 95% classiﬁcation accuracy on MNIST dataset (Diehl and Cook, 2015).”