Hardware-Software Co-design Approach for Deep Learning Inference

By Debdeep Paul, Jawar Singh, Jimson Mathew. 2019

7 citations recorded by the source corpus.

Hardware implementation of machine learning algorithms is a promising solution for higher performance and improved throughput. However, the most challenging task lies in the design of power, energy, and area efficient architectures that can be deployed in tightly constrained embedded systems. In this paper, we investigate the concept of intermediate exit branches in a CNN architecture aiming to reduce latency and energy. We propose a heterogeneous hardware-software based architecture for performing inference in feed-forward neural networks which is capable of early exiting through intermediate exits according to the difficulty level of the input. Our proposed architecture can be reconfigured to support different kernel sizes and is fully scalable. Our experiments show an increase in classification accuracy by 0.27% for MNIST dataset (LeNet) with 2.2× decrease in average computations and 1.07% for Cifar-10 dataset (AlexNet) with 1.55× decrease in average computations along with 1.39× reduction in energy consumption for LeNet architecture.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Julian McAuley
5. Le Yang
6. Yizeng Han
7. Canwen Xu
8. Honghui Wang

In Computer Vision and Pattern Recognition

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Yizeng Han
5. Le Yang
6. Julian McAuley
7. Honghui Wang
8. Canwen Xu

In Electrical and Electronic Engineering

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Julian McAuley
5. Canwen Xu

In ML Systems & Hardware

1. Gao Huang
2. Julian McAuley

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, methodology): “tions, including pixel-level dynamic computation [114], [275], [276], adaptive channel pruning [277], [278] and early exiting [279], have also been tailored together with hard-”
• A Survey on Dynamic Neural Networks for Natural Language Processing (2022, background): “There have been works exploring optimized runtime for MoE (Shazeer et al., 2018; Jia et al., 2020; He et al., 2021; Rajbhandari et al., 2022) and early exit (Paul et al., 2019) while more to be done in the future.”