Convolutional networks for fast, energy-efficient neuromorphic computing

By Steven K. Esser, Paul A. Merolla, John V. Arthur, A. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, T. Melano, Davis Barch and 6 more. 2016

763 citations recorded by the source corpus.

Significance Brain-inspired computing seeks to develop new technologies that solve real-world problems while remaining grounded in the physical requirements of energy, speed, and size. Meeting these challenges requires high-performing algorithms that are capable of running on efficient hardware. Here, we adapt deep convolutional neural networks, which are today’s state-of-the-art approach for machine perception in many domains, to perform classification tasks on neuromorphic hardware, which is today’s most efficient platform for running neural networks. Using our approach, we demonstrate near state-of-the-art accuracy on eight datasets, while running at between 1,200 and 2,600 frames/s and using between 25 and 275 mW. Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that (i) approach state-of-the-art classification accuracy across eight standard datasets encompassing vision and speech, (ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1,200 and 2,600 frames/s and using between 25 and 275 mW (effectively >6,000 frames/s per Watt), and (iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. This approach allows the algorithmic power of deep learning to be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

Factorization tricks for LSTM networks (2017, methodology): “Multi-group convnets have been widely used to reduce network weights and required compute, for example by Esser et al. (2016).”
Deep learning with coherent nanophotonic circuits (2016, background): “Traditional central processing units (CPUs) are far suboptimal for implementing these algorithms [2]; and a growing effort in academia and industry has been put towards the development of new hardware architectures tailored towards applications in artificial neural networks and deep learning [3].”
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE) (2018, methodology): “EEDN is a convolutional deep neural network architecture that can be trained oﬄine (e.g., on a GPU) and deployed on the IBM TrueNorth chip (Esser et al., 2016).”
When Neurons Fail - Technical Report (2016, background): “More recently, teams from IBM reported [18, 19] a successful neuromorphic implementations of convolutional neural networks that require a running power as low as 25 mW to 275 mW.”
Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting (2022, background): “So far, there are various shapes of surrogate gradients, such as rectangular (Wu et al., 2018; 2019), triangle (Esser et al., 2016; Rathi & Roy, 2020), and exponential (Shrestha & Orchard, 2018) curve.”
When Neurons Fail (2017, background): “More recently, teams from IBM reported [18], [19] a successful neuromorphic implementations of convolutional neural networks that require a running power as low as 25 mW to 275 mW.”
Towards Accurate Binary Convolutional Neural Network (2017, methodology): “It is known that binary activation can use spiking response for event-based computation and communication (consuming energy only when necessary) and therefore is energy-efﬁcient [Esser et al., 2016].”
Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook (2021, methodology): “This formulation of SNN error backpropagation has been successfully demonstrated in recent years [22], [33]–[36], and several implementations have been applied to Loihi networks.”
Biological plausibility and stochasticity in scalable VO2 active memristor neurons (2018, background): “-biased memristor X 2 acts as the voltage-gated K+ channel. Capacitors C 1 and C 2 are the corresponding membrane capacitances. b, Basic steps in action potential (spike) generation of a VO 2 neuron. (1) Resting state, in which both the Na+ +and K channels are closed. A resting potential of 0.2– 0.3”
Memory-Efficient Deep Learning on a SpiNNaker 2 Prototype (2018, methodology): “In contrast, we show 96.6% inference accuracy for MNIST at ≈ 23 µJ per image classification, which is comparable to TrueNorth.”
Deep neural networks are robust to weight binarization and other non-linear distortions (2016, methodology): “The output of a layer (which is also the input to the next layer) is computed using the convolution operation ∗, for example Ak+1 = r(Ak ∗ Proj(Wk)) where r is the typical ReLU activation, and Proj is a projection operator described below.”
Efficient Processing of Deep Neural Networks: A Tutorial and Survey (2017, methodology): “1Note: Recent work using TrueNorth in a stylized fashion allows it to be used to compute reduced precision neural networks [9].”