FPGA/DNN Co-Design: An Efficient Design Methodology for 1oT Intelligence on the Edge

By Cong Hao, Xiaofan Zhang, Yuhong Li, Sitao Huang, Jinjun Xiong, Kyle Rupnow, Wen-mei W. Hwu, Deming Chen. 2019

192 citations recorded by the source corpus.

While embedded FPGAs are attractive platforms for DNN acceleration on edge-devices due to their low latency and high energy efficiency, the scarcity of resources of edge-scale FPGA devices also makes it challenging for DNN deployment. In this paper, we propose a simultaneous FPGA/DNN co-design methodology with both bottom-up and top-down approaches: a bottom-up hardware-oriented DNN model search for high accuracy, and a top-down FPGA accelerator design considering DNN-specific characteristics. We also build an automatic co-design flow, including an Auto-DNN engine to perform hardware-oriented DNN model search, as well as an Auto-HLS engine to generate synthesizable C code of the FPGA accelerator for explored DNNs. We demonstrate our co-design approach on an object detection task using PYNQ-ZI FPGA. Results show that our proposed DNN model and accelerator outperform the state-of-the-art FPGA designs in all aspects including Intersection-over-Union (IoU) (6.2% higher), frames per second (FPS) (2.48\times higher), power consumption (40% lower), and energy efficiency (2.5\times higher). Compared to GPU-based solutions, our designs deliver similar accuracy but consume far less energy.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Song Han
2. Chuang Gan
3. Kurt Keutzer
4. Luc Van Gool
5. Dacheng Tao
6. Quoc V. Le
7. Jitendra Malik
8. Trevor Darrell

In Electrical and Electronic Engineering

1. Kurt Keutzer
2. Zhen Dong
3. Song Han
4. Bichen Wu
5. Tian Li
6. Han Cai

Citations in the corpus

• Once for All: Train One Network and Specialize it for Efficient Deployment (2019, methodology): “Hardware-DNN co-design techniques (Jiang et al., 2019b;a; Hao et al., 2019) jointly optimize neural network architectures and hardware architectures.”
• Hardware/Software Co-Exploration of Neural Architectures (2019, background): “also verified in [34], which also targets a single FPGA.”
• CoDeNet: Efficient Deployment of Input-Adaptive Object Detection on Embedded FPGAs (2021, methodology): “Many prior acceleration works [47] [26][27][12][41][36] [35] have demonstrated the effectiveness of the co-design methodology for the deployment of real-time object detection on FPGAs.”