Accurate deep neural network inference using computational phase-change memory

By V. Joshi, M. L. Gallo, I. Boybat, Simon Haefeli, C. Piveteau, M. Dazzi, Bipin Rajendran, Abu Sebastian, Evangelos Eleftheriou. Nature Communications, 2019

474 citations recorded by the source corpus.

In-memory computing using resistive memory devices is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to phase-change memory (PCM) devices. We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on CIFAR-10 and a top-1 accuracy of 71.6% on ImageNet benchmarks after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one-day period, where each of the 361,722 synaptic weights is programmed on just two PCM devices organized in a differential configuration. Designing deep learning inference hardware based on in-memory computing remains a challenge. Here, the authors propose a strategy to train ResNet-type convolutional neural networks which results in reduced accuracy loss when transferring weights to in-memory computing hardware based on phase-change memory.

Subfields: Electrical and Electronic Engineering, Polymers and Plastics

View Paper: https://www.semanticscholar.org/paper/19c750a84f41de577efb7c30570658e66aa395d8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In Polymers and Plastics

Citations in the corpus

Hardware implementation of memristor-based artificial neural networks (2024, unknown): “To avoid this loss of accuracy, hardware-aware training methods, in which device non-idealities are incorporated during training have been proposed in the literature 189,190 .”
In-memory hyperdimensional computing (2019, methodology): “PCM has also been successfully employed in novel computing paradigms such as neuromorphic computing 36–40 and computational memory 20,22,41,42 , which makes it a good candidate for realizing the in-memory HDC system.”
A compute-in-memory chip based on resistive random-access memory (2022, unknown): “…results using a mixture of hardware characterization and software simulation, for example, emulate the array-level MVM process in software using measured device characteristics 3,5,21,24 , such an approach often fails to model the complete set of non-idealities existing in realistic hardware.”
Memristive technologies for data storage, computation, encryption, and radio-frequency communication (2022, background): “However, a critical aspect for IMC implementations is the custom offline training and/or on-chip retraining of the network needed to mitigate the effects of stuck-at devices, device noise, and circuit level non-ideality on network accuracy ( 61 ).”