Ares: A framework for quantifying the resilience of deep neural networks

By Brandon Reagen, Udit Gupta, Lillian Pentecost, Paul N. Whatmough, Sae Kyu Lee, Niamh Mulholland, David M. Brooks, Gu-Yeon Wei. Design Automation Conference, 2018

352 citations recorded by the source corpus.

As the use of deep neural networks continues to grow, so does the fraction of compute cycles devoted to their execution. This has led the CAD and architecture communities to devote considerable attention to building DNN hardware. Despite these efforts, the fault tolerance of DNNs has generally been overlooked. This paper is the first to conduct a large-scale, empirical study of DNN resilience. Motivated by the inherent algorithmic resilience of DNNs, we are interested in understanding the relationship between fault rate and model accuracy. To do so, we present Ares: a light-weight, DNN-specific fault injection framework validated within 12% of real hardware. We find that DNN fault tolerance varies by orders of magnitude with respect to model, layer type, and structure.

Subfields: Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/52a337783d644a432dfc311a0d8eb018a9f4840f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

Citations in the corpus

CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery (2020, background): “Nodes in a large-scale training can fail for various reasons: hardware failures (Wang et al., 2017; Reagen et al., 2018; Birke et al., 2014; Narayanan et al., 2016; Dean & Barroso, 2013), system failures (e.g., out-of-memory), user errors (e.g., bug in the code), and maintenance failures (e.g.,…”
MnnFast: A Fast and Scalable System Architecture for Memory-Augmented Neural Networks (2019, neutral): “Fault-tolerance varies by NNs [57].”
MaxNVM: Maximizing DNN Storage Density and Inference Efficiency with Sparse Encoding and Error Mitigation (2019, methodology): “validated fault injection framework to quantify the impact of faults on DNN accuracy [57].”