Neural Acceleration for General-Purpose Approximate Programs

By Hadi Esmaeilzadeh, Adrian Sampson, Luis Ceze, Doug Burger. 2012

604 citations recorded by the source corpus.

This paper describes a learning-based approach to the acceleration of approximate programs. We describe the \emph{Parrot transformation}, a program transformation that selects and trains a neural network to mimic a region of imperative code. After the learning phase, the compiler replaces the original code with an invocation of a low-power accelerator called a \emph{neural processing unit} (NPU). The NPU is tightly coupled to the processor pipeline to accelerate small code regions. Since neural networks produce inherently approximate results, we define a programming model that allows programmers to identify approximable code regions -- code that can produce imprecise but acceptable results. Offloading approximable code regions to NPUs is faster and more energy efficient than executing the original code. For a set of diverse applications, NPU acceleration provides whole-application speedup of 2.3x and energy savings of 3.0x on average with quality loss of at most 9.6%.

Subfields: Electrical and Electronic Engineering, Hardware and Architecture, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In Hardware and Architecture

In Optimization & Training

Citations in the corpus

Neural Execution Engines: Learning to Execute Subroutines (2020, methodology): “The computer architecture community has also explored using neural networks to execute approximate portions of algorithms, as there could be execution speed and efﬁciency advantages [7].”
Post-compiler software optimization for reducing energy (2014, background): “Software engineers are studying methods that trade the guarantee of semantic preservation [11, 15, 22, 27, 65] for greater reliability and robustness [2, 8, 49, 58], availability and security [50], compile-time algorithmic ﬂexibility [5], or performance and quality of service [6, 39, 40].”
Approximate Computing: A Survey (2016, methodology): “With the emergence of the approximate computing paradigm, they are proposed to accelerate hot codes in errorresilient applications [14]–[16].”
Cambricon: An Instruction Set Architecture for Neural Networks (2016, background): “Hardware accelerators customized to NNs have been recently investigated as energyefficient alternatives [3], [5], [11], [29], [32].”
A Survey of Techniques for Approximate Computing (2016, methodology): “…et al. 2013, 2014] and variable accuracy [Ansel et al. 2011] (b) Applications or their code portions that are amenable to AC are called: Approximable [Esmaeilzadeh et al. 2012b], relaxable [Yazdanbakhsh et al. 2015a], soft slices/ computations [Esmaeilzadeh et al. 2012a], best-effort computations…”
Efficient embedded learning for IoT devices (2016, methodology): “To address these challenges, several approximation techniques at various layers of the computing stack, including software [17]–[20], architecture [12], [21]–[24] and circuits [25]–[29], have been proposed.”
ShiDianNao: Shifting vision processing closer to the sensor (2015, neutral): “In the past few years, accelerators have gained increasing attention as an energy and cost effective alternative to CPUs and GPUs [20, 57, 13, 14, 60, 61].”
Approximate storage in solid-state memories (2013, neutral): “Approximate storage in solid-state memories.”
AxNN: Energy-efficient neuromorphic systems using approximate computing (2014, methodology): “This ability to relax the accuracy of computations is translated into significant improvements in energy or performance by utilizing a variety of hardware [9–11,13,14] and software [8, 12] techniques.”
Cambricon-X: An accelerator for sparse neural networks (2016, background): “Hence, researchers have proposed a number of NN accelerators [3], [11]–[14].”
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, background): “improved by customizing memory hierarchy and assigning dedicated resources [63].”
MachSuite: Benchmarks for accelerator design and customized architectures (2014, background): “While this effect shows up in benchmarks for conventional platforms, accelerators are more Research Area Publications Acc-Centric [11], [12], [13], [14], [15], [16], [21] Architectures [22], [34], [37], [40], [41], [43], [46] HLS Optimizations [4], [6], [7], [17], [20], [23], [31], [32], [33], [35],”
PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory (2016, neutral): “Researchers have also utilized NNs to accelerate approximate computing [32], [33].”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators (2020, neutral): “[17], TrueNorth [18–20], MT-spike [21], PT-spike [22] DNN Co-processor NPU [23], DianNao-family [24–27], Cambricon [28], Cambricon-x [29], TPU [30,31], ScaleDeep [32], Stripes [33], Neural Cache [123,124], Diffy [125] FPGA FPGA-DCNN [34], Embedded-FPGA-CNN [35], FPGA-Exploration [36],…”