Energy-efficient Amortized Inference with Cascaded Deep Classifiers

By Jiaqi Guan, Yang Liu, Qiang Liu, Jian Peng. 2017

41 citations recorded by the source corpus.

Deep neural networks have been remarkable successful in various AI tasks but often cast high computation and energy cost for energy-constrained applications such as mobile sensing. We address this problem by proposing a novel framework that optimizes the prediction accuracy and energy cost simultaneously, thus enabling effective cost-accuracy trade-off at test time. In our framework, each data instance is pushed into a cascade of deep neural networks with increasing sizes, and a selection module is used to sequentially determine when a sufficiently accurate classifier can be used for this data instance. The cascade of neural networks and the selection module are jointly trained in an end-to-end fashion by the REINFORCE algorithm to optimize a trade-off between the computational cost and the predictive accuracy. Our method is able to simultaneously improve the accuracy and efficiency by learning to assign easy instances to fast yet sufficiently accurate classifiers to save computation and energy cost, while assigning harder instances to deeper and more powerful classifiers to ensure satisfiable accuracy. Moreover, we demonstrate our method's effectiveness with extensive experiments on CIFAR-10/100, ImageNet32x32 and original ImageNet dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Why Should We Add Early Exits to Neural Networks? (2020, background): “To conclude this section, we also mention that some authors have considered reinforcement learning strategies to train the overall architecture [21].”
SkipNet: Learning Dynamic Routing in Convolutional Networks (2017, methodology): “Close to our proposed algorithm, [9] and [21] study reinforcement learning algorithms to solve the decision problem.”
ILFO: Adversarial Attack on Adaptive Neural Networks (2020, background): “[10] use reinforcement learning to choose between multiple classifiers.”
InferLine: latency-aware provisioning and scaling for prediction serving pipelines (2020, methodology): “This conditional evaluation pattern appears in bandit algorithms [3, 20] used for model personalization as well as more general cascaded prediction pipelines [2, 14, 24, 34].”