Big/little deep neural network for ultra low power inference

By Eunhyeok Park, Dongyoung Kim, Soobeom Kim, Yong-Deok Kim, Gunhee Kim, Sungroh Yoon, Sungjoo Yoo. 2015

151 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Dimitrios Stamoulis
3. Diana Marculescu
4. Yulin Wang
5. Gao Huang
6. Julian McAuley
7. Le Yang
8. Yizeng Han

In Computer Vision and Pattern Recognition

1. Shiji Song
2. Yulin Wang
3. Diana Marculescu
4. Gao Huang
5. Dimitrios Stamoulis
6. Xu Chen
7. Yizeng Han
8. Le Yang

In Electrical and Electronic Engineering

1. Shiji Song
2. Yulin Wang
3. Liekang Zeng
4. Gao Huang
5. Zhi Zhou
6. Diana Marculescu
7. Xu Chen
8. Warren J. Gross

In Efficiency & Compression

1. Shiji Song
2. Gao Huang
3. Yulin Wang
4. Le Yang
5. Julian McAuley
6. Yizeng Han
7. Diana Marculescu
8. Dimitrios Stamoulis

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, background): “Alternative criteria include the entropy [45], [58] and the score margin [49].”
• Designing Adaptive Neural Networks for Energy-Constrained Image Classification (2018, methodology): “hardware-constrained hyper-parameter optimization of adaptive CNNs based on hardware measurements on a real mobile platform, even though prior art has highlighted this suboptimality [20].”
• The cascading neural network: building the Internet of Smart Things (2017, background): “The most relevant of these approaches are the Big-Little neural networks [25] where a little, fast to execute network is used to try to classify an input sample.”
• Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “Two DL models of different sizes (big and little) can be used for classification during inference [210].”
• A Survey on Dynamic Neural Networks for Natural Language Processing (2022, background): “Early works for early exit in computer vision (Park et al., 2015; Teerapittayanon et al., 2016; Kaya et al., 2019) often fall into this category.”
• Surprisal-Triggered Conditional Computation with Neural Networks (2020, methodology): “In the score margin approach, a small model is used to compute scores for each class in a classification problem; if the margin between the largest score and the second largest score is below a certain threshold, then a bigger model is used [18, 19, 20, 21, 22].”
• Neural networks designing neural networks: Multi-objective hyper-parameter optimization (2016, background): “With the proliferation of machine learning on embedded and mobile devices, ANN application designers must now deal with stringent power and cost requirements [9, 16, 21].”
• Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, background): “[102] propose a big/little DNN model selection framework.”