Backpropagation for Energy-Efficient Neuromorphic Computing

By Steven K. Esser, Rathinakumar Appuswamy, Paul A. Merolla, John V. Arthur, Dharmendra S. Modha. 2015

328 citations recorded by the source corpus.

Subfields: ML Systems & Hardware, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

In Neuroscience

Citations in the corpus

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks (2016, methodology): “A fully binary network at run time presented in [37] using a similar approach to EBP, showing significant improvement in energy efficiency.”
Binarized Neural Networks (2016, methodology): “We argue that the proposed BBP algorithm can be implemented in hardware and is expected to be much more efficient in terms of area, speed, and energy consumption than full precision DNNs, which used floating-point multiply-accumulators.”
When Neurons Fail (2017, background): “More recently, teams from IBM reported [18], [19] a successful neuromorphic implementations of convolutional neural networks that require a running power as low as 25 mW to 275 mW.”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “A fully binary NN (BNN) is presented in [136] using EBP demonstrating improved energy efficiency on the neuromorphic chip.”
Evaluating the Energy Efficiency of Deep Convolutional Neural Networks on CPUs and GPUs (2016, background): “Although the mainstream approach for training deep convolutional neural networks is using CPUs and GPUs, researchers have recently started to explore the use of other architectures and devices, including FPGA [36], RRAM [37], neuromorphic processors [38] and Tetra-Parallel architecture [39].”
Convolutional networks for fast, energy-efficient neuromorphic computing (2016, background): “This approach draws inspiration from the spiking neurons and lowprecision synapses of the brain (8) and builds on work showing that deep learning can create networks with constrained connectivity (9), low-precision synapses (10, 11), low-precision neurons (12–14), or both low-precision synapses and ”
On the Robustness of a Neural Network (2017, background): “A year ago, teams from IBM reported [13], [14] a successful neuromorphic implementation of neural networks that requires a running power as low as 25 mW to 275 mW to perform image recognition tasks that would require orders of magnitude more energy on a classical computer.”
CaffePresso: An optimized library for Deep Learning on embedded accelerator-based platforms (2016, background): “While there is significant interest in design of ASICs customized for deep learning [5, 8, 6, 2, 3], along with large FPGA-based [10, 14, 7] and GPU-based accelerators [13] for high-performance systems, we investigate the potential of commercial off-the-shelf SoC hardware for efficiently implementin”
Deep neural networks are robust to weight binarization and other non-linear distortions (2016, result): “While normalizing across the entire layer seems crude compared to a filter-wise normalization (as in [7]), we have found that the two cases lead to similar results.”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, background): “[17], TrueNorth [18–20], MT-spike [21], PT-spike [22] DNN Co-processor NPU [23], DianNao-family [24–27], Cambricon [28], Cambricon-x [29], TPU [30,31], ScaleDeep [32], Stripes [33],”
Training Deep Spiking Neural Networks Using Backpropagation (2016, methodology): “An alternative approach has recently been introduced by [23], in which a SNN learns from spikes, but requires keeping statistics for computing stochastic gradient descent (SGD) updates in order to approximate a conventional ANN.”
Neuro-Inspired Computing With Emerging Nonvolatile Memorys (2018, methodology): “, at least three orders of magnitude better than the 28-nm TrueNorth chip implementation of the same task [111], with a similar accuracy.”
SLAYER: Spike Layer Error Reassignment in Time (2018, methodology): “There are different approaches to overcome the loss of accuracy such as introducing extra constraints on neuron firing rate [23], scaling the weights [23–25], constraining the network parameters [20], formulating an equivalent transfer function for a spiking neuron [19–22], adding noise in the model”