Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications

By Guanpeng Li, S. Hari, Michael B. Sullivan, Timothy Tsai, K. Pattabiraman, Joel S. Emer, S. Keckler. 2017

599 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

The Hidden Vulnerability of Distributed Learning in Byzantium (2018, methodology): “stions such as: how much error will be forward propagated if units of the model are erroneous? Recently, NVIDIA investigated (a closely related statement of) the question through experimental lenses (Li et al., 2017). In the mean time, theoretical evaluations of this propagated error were derived wi”
MaxNVM: Maximizing DNN Storage Density and Inference Efficiency with Sparse Encoding and Error Mitigation (2019, background): “Abundant prior work exists related to the fault tolerance of DNNs [23, 25, 44, 44, 54, 57, 58, 65], and our presented fault tolerance studies (Section 4) confirm and expound upon previous results by additionally considering the resiliency of sparsely-encoded DNN weights.”
A Survey on Distributed Machine Learning (2019, background): “[91] investigated the resilience characteristics of deep neural networks with regard to hardware errors when running on accelerators, which are frequently deployed in major HPC systems.”
Ares: A framework for quantifying the resilience of deep neural networks (2018, background): “…in understanding can lead to problematic image misclassifications; in order to meet ISO standards, techniques to improve reliability are required [8] In [9], it is shown that multiple circuit faults can be circumvented with retraining in neural networks that have dozens of parameters Finally,…”