AutoFocus: Efficient Multi-Scale Inference

By Mahyar Najibi, Bharat Singh, Larry S. Davis. 2018

145 citations recorded by the source corpus.

This paper describes AutoFocus, an efficient multi-scale inference algorithm for deep-learning based object detectors. Instead of processing an entire image pyramid, AutoFocus adopts a coarse to fine approach and only processes regions which are likely to contain small objects at finer scales. This is achieved by predicting category agnostic segmentation maps for small objects at coarser scales, called FocusPixels. FocusPixels can be predicted with high recall, and in many cases, they only cover a small fraction of the entire image. To make efficient use of FocusPixels, an algorithm is proposed which generates compact rectangular FocusChips which enclose FocusPixels. The detector is only applied inside FocusChips, which reduces computation while processing finer scales. Different types of error can arise when detections from FocusChips of multiple scales are combined, hence techniques to correct them are proposed. AutoFocus obtains an mAP of 47.9% (68.3% at 50% overlap) on the COCO test-dev set while processing 6.4 images per second on a Titan X (Pascal) GPU. This is 2.5X faster than our multi-scale baseline detector and matches its mAP. The number of pixels processed in the pyramid can be reduced by 5X with a 1% drop in mAP. AutoFocus obtains more than 10% mAP gain compared to RetinaNet but runs at the same speed with the same ResNet-101 backbone.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multiscale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and”
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition (2021, unknown): “3.3.”
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition (2020, unknown): “Reinforcement learning has been used to adaptively select different regions for fast object detection in large images [41,19].”
FoveaBox: Beyound Anchor-Based Object Detection (2019, background): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [21], [34], [35], context and attention mechanism [36], training strategy and loss function [37], [38], feature fusion and enhancement [39]–[41].”
QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection (2021, result): “One similar work to our QueryDet is AutoFocus [35].”
FoveaBox: Beyond Anchor-based Object Detector (2019, background): “Several attempts have been performed to boost the performance of the detector, including feature pyramid [27][24][21], multiscale [44][34], and object relations [17], etc.”
Dynamic Network Quantization for Efficient Video Inference (2021, unknown): “Dynamic computation to improve efﬁciency has been studied from multiple perspectives [1, 2, 50, 54, 15, 37, 13, 33].”
Towards Large-Scale Small Object Detection: Survey and Benchmarks (2022, background): “Najibi et al. [80] proposed a coarse-to-ﬁne pipeline for detecting small objects.”
Video Self-Stitching Graph Network for Temporal Action Localization (2020, background): “Multiple methods have been proposed to deal with small objects specifically [3, 28] or object scale variation in general [4, 21] in images.”
AdaFrame: Adaptive Frame Selection for Fast Video Recognition (2018, background): “Related are also [30, 26, 32, 15, 5] that learn to drop layers in residual networks or learn where to look in images conditioned on inputs.”
CornerNet-Lite: Efficient Keypoint based Object Detection (2019, methodology): “Prior work that employs saccade-like mechanisms either detects a single object per crop (e.g. Faster R-CNN [46]) or produces multiple detections per crop with a two-stage network involving additional sub-crops (e.g. AutoFocus [37]).”
LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition (2019, unknown): “Several recent approaches learn to choose which layers in a large network to use [35, 31, 37] or select regions to attend to in images [25, 7], conditioned on inputs, to achieve fast inference.”
AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition (2021, background): “Adaptive selection of different regions for fast object detection is presented in [36, 16].”
2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition (2020, methodology): “a, conditional computation) methods have been developed in the image domain, achieving reduced computation by dynamically selecting channels [2, 25], skipping layers [42, 11, 40, 37], performing early exiting with auxiliary structures [22, 18, 3, 46], adaptively switching input resolutions [27, 36, ”