Anytime Recognition with Routing Convolutional Networks

By Zequn Jie, Peng Sun, Xin Li, Jiashi Feng, Wei David Liu. 2019

29 citations recorded by the source corpus.

Achieving an automatic trade-off between accuracy and efficiency for a single deep neural network is highly desired in time-sensitive computer vision applications. To achieve anytime prediction, existing methods only embed fixed exits to neural networks and make the predictions with the fixed exits for all the samples (refer to the “latest-all” strategy). However, it is observed that the latest exit within a time budget does not always provide a more accurate prediction than the earlier exits for testing samples of various difficulties, making the “latest-all” strategy a sub-optimal solution. Motivated by this, we propose to improve the anytime prediction accuracy by allowing each sample to adaptively select its own optimal exit within a specific time budget. Specifically, we propose a new Routing Convolutional Network (RCN). For any given time budget, it adaptively selects the optimal layer as exit for a specific testing sample. To learn an optimal policy for sample routing, a Q-network is embedded into the RCN at each exit, considering both potential information gain and time-cost. To further boost the anytime prediction accuracy, the exits and the Q-networks are optimized alternately to mutually boost each other under the cost-sensitive environment. Apart from applying to whole image classification, RCN can also be adapted to dense prediction tasks, e.g., scene parsing, to achieve the pixel-level anytime prediction. Extensive experimental results on CIFAR-10, CIFAR-100, and ImageNet classification benchmarks, and Cityscapes scene parsing benchmark demonstrate the efficacy of the proposed RCN for anytime recognition.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Shiji Song
  2. Yulin Wang
  3. Gao Huang
  4. Le Yang
  5. Yizeng Han
  6. Honghui Wang
  7. Quoc V. Le
  8. Alan L. Yuille

In Computer Vision and Pattern Recognition

  1. Shiji Song
  2. Yulin Wang
  3. Gao Huang
  4. Yizeng Han
  5. Le Yang
  6. Honghui Wang
  7. Luc Van Gool
  8. Alan L. Yuille

In Efficiency & Compression

  1. Shiji Song
  2. Gao Huang
  3. Yulin Wang
  4. Le Yang
  5. Yizeng Han
  6. Honghui Wang

Citations in the corpus