Adaptive Classification for Prediction Under a Budget
By Feng Nan, Venkatesh Saligrama. 2017
65 citations recorded by the source corpus.
We propose a novel adaptive approximation approach for test-time resource-constrained prediction motivated by Mobile, IoT, health, security and other applications, where constraints in the form of computation, communication, latency and feature acquisition costs arise. We learn an adaptive low-cost system by training a gating and prediction model that limits utilization of a high-cost model to hard input instances and gates easy-to-handle input instances to a low-cost model. Our method is based on adaptively approximating the high-cost model in regions where low-cost models suffice for making highly accurate predictions. We pose an empirical loss minimization problem with cost constraints to jointly train gating and prediction models. On a number of benchmark datasets our method outperforms state-of-the-art achieving higher accuracy for the same cost.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Tom Véniat
- Ludovic Denoyer
- Tong Wang
- Simone Scardapane
- Wei Yang
- Michele Scarpiniti
- Aurelio Uncini
- Enzo Baccarelli
In Computer Vision and Pattern Recognition
- Michele Scarpiniti
- Enzo Baccarelli
- Tom Véniat
- Ludovic Denoyer
- Simone Scardapane
- Aurelio Uncini
- Wei Yang
- Tong Wang
In Optimization & Training
- Wei Yang
- Tong Wang
- Michele Scarpiniti
- Aurelio Uncini
- Enzo Baccarelli
- Simone Scardapane
- Anki Chauhan
- Mirazul Haque
Citations in the corpus
- Why Should We Add Early Exits to Neural Networks? (2020, background): “In these works, it is common to design systems in which separate neural networks, of different complexities, can run on the different tiers or on different computational units [ 44, 50 ] .”
- ILFO: Adversarial Attack on Adaptive Neural Networks (2020, methodology): “[22] use gating techniques to choose a high-cost or low-cost model.”
- Interpretable Companions for Black-Box Models (2020, background): “Nan and Saligrama [18] designed a low-cost adaptive system by training a gating and prediction model that limits the utilization of a high-cost model to hard input instances and gates easy-to-handle input instances to a low-cost model.”
- Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks (2017, methodology): “Another technique proposed by [12], [24] and used in [31, 21] is the distillation of knowledge, which consists of training a smaller network to imitate the outputs of a larger network.”