Energy-efficient ConvNets through approximate computing
By Bert Moons, Bert De Brabandere, Luc Van Gool, Marian Verhelst. 2016
105 citations recorded by the source corpus.
Recently convolutional neural networks (ConvNets) have come up as state-of-the-art classification and detection algorithms, achieving near-human performance in visual detection. However, ConvNet algorithms are typically very computation and memory intensive. In order to be able to embed ConvNet-based classification into wearable platforms and embedded systems such as smartphones or ubiquitous electronics for the internet-of-things, their energy consumption should be reduced drastically. This paper proposes methods based on approximate computing to reduce energy consumption in state-of-the-art ConvNet accelerators. By combining techniques both at the system- and circuit level, we can gain energy in the systems arithmetic: up to 30× without losing classification accuracy and more than 100× at 99% classification accuracy, compared to the commonly used 16-bit fixed point number format.
Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luca Benini
- Xianbin Wang
- Shree Krishna Sharma
- Bert Moons
- Marian Verhelst
- Ala Al-Fuqaha
- Jonathan Shen
- Vishnu Naresh Boddeti
In Electrical and Electronic Engineering
- Ala Al-Fuqaha
- M. Mohammadi
- Luca Benini
- Xianbin Wang
- Shree Krishna Sharma
- Marian Verhelst
- Bert Moons
- Vishnu Naresh Boddeti
Citations in the corpus
- Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, methodology): “Approximate computing is another approach to both implement machine learning tools on IoT devices and contribute to the energy saving of their hosting devices [145], • Good for time-critical IoT apps • Energy-efficient • Provides more security and privacy for data • Special-purpose networks [146].”
- YodaNN: An Ultra-Low Power Convolutional Neural Network Accelerator Based on Binary Weights (2016, methodology): “Moons et al. [26] were able to reduce power by 30x (compared to 16-bit fixed-point) without accuracy loss and 225x with an accuracy of 99% on AlexNet, CifarQuick and LeNet-5.”
- A 0.3–2.6 TOPS/W precision-scalable processor for real-time large-scale ConvNets (2016, methodology): “Dynamic precision and voltage scaling can be performed without sacrificing significant accuracy [4].”
- YodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration (2016, unknown): “[34] analyzed the accuracy-energy tradeoff by”
- Toward Massive Machine Type Communications in Ultra-Dense Cellular IoT Networks: Current Issues and Machine Learning-Assisted Solutions (2018, methodology): “Another enabling approach to make DL models implementable for IoT devices is to integrate DL models with approximate computing [38].”
- In Teacher We Trust: Learning Compressed Models for Pedestrian Detection (2016, background): “L are the input logits to the softmax layer, and are also the outputs of the fully-connected layer before it.”