Finding Tiny Faces

By Peiyun Hu, Deva Ramanan. 2016

780 citations recorded by the source corpus.

Though tremendous strides have been made in object recognition, one of the remaining open challenges is detecting small objects. We explore three aspects of the problem in the context of finding small faces: the role of scale invariance, image resolution, and contextual reasoning. While most recognition approaches aim to be scale-invariant, the cues for recognizing a 3px tall face are fundamentally different than those for recognizing a 300px tall face. We take a different approach and train separate detectors for different scales. To maintain efficiency, detectors are trained in a multi-task fashion: they make use of features extracted from multiple layers of single (deep) feature hierarchy. While training detectors for large objects is straightforward, the crucial challenge remains training detectors for small objects. We show that context is crucial, and define templates that make use of massively-large receptive fields (where 99% of the template extends beyond the object of interest). Finally, we explore the role of scale in pre-trained deep networks, providing ways to extrapolate networks tuned for limited scales to rather extreme ranges. We demonstrate state-of-the-art results on massively-benchmarked face datasets (FDDB and WIDER FACE). In particular, when compared to prior art on WIDER FACE, our results reduce error by a factor of 2 (our models produce an AP of 82% while prior art ranges from 29-64%).

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “…image datasets [22, 31, 13, 69, 37, 24, 41, 68, 15, 27, 46, 8, 39] have been proposed and fostered research and development for automated face detection [35, 21], alignment [66, 44], recognition [57, 49], generation [67, 5, 26, 58], modiﬁcation [3, 32, 19], and attribute classiﬁcation [31, 37].”
An Analysis of Scale Invariance in Object Detection - SNIP (2017, methodology): “Recently, a pyramidal approach was proposed for detecting faces [17] where the gradients of all objects were back-propagated after max-pooling the responses from each scale.”
Deep Learning for Face Recognition: Pride or Prejudiced? (2019, background): “Deep Learning based systems show extremely high accuracy in many computer vision application such as object detection and recognition, and image synthesis [20, 22, 29].”
Attentional Local Contrast Networks for Infrared Small Target Detection (2020, background): “Besides carefully designing the anchor matching strategy [23] or training the network in a scaleaware scheme [24], encoding context that offers more evidence beyond the object extent is also highly explored as a solution to mitigate the problems arising from small objects [25], [26], which is genera”
Object Detection in 20 Years: A Survey (2019, methodology): “CVPR2017 [322].”
Deep Facial Expression Recognition: A Survey (2018, methodology): “d the Dlib C++ library4 [38] have also been adopted (e.g., [39], [40]). Recently, deep learning methods such as multitask cascaded convolutional networks (MTCNN)5 [41], DenseReg6 [42] and Tiny Faces7 [43] have achieved superior performance in many face alignment benchmarks. These methods are robust ”
Deep learning in video multi-object tracking: A survey (2019, methodology): “[70] used the Tiny CNN detector [71] in their face tracking algorithm, obtaining a better performance when compared to the Deformable Parts Model detector (DPM) [25], that does not use deep learning techniques.”
RetinaFace: Single-stage Dense Face Localisation in the Wild (2019, methodology): “Multi-scale Cascade CNN [60], Two-stage CNN [60], ACF-WIDER [58], Faceness-WIDER [59], Multitask Cascade CNN [66], CMS-RCNN [72], LDCF+ [37], HR [23], Face R-CNN [54], ScaleFace [61], SSH [36], SFD [68], Face R-FCN [57], MSCNN [4], FAN [56], Zhu et al .”
A Survey of Deep Learning-Based Object Detection (2019, background): “Hu and Ramanan [98] adopt feature fusion to better detect small faces which is produced by image pyramid.”
Context-Aware Single-Shot Detector (2017, background): “We note that context has been shown crucial in recognizing tiny objects [13].”
Unsupervised incremental learning of deep descriptors from video streams (2017, methodology): “In this paper, we present unsupervised learning of subject identities from video streams that exploits the ResNet deep network [8] to detect faces in consecutive images and fc7 deep descriptor of VGGface [9] for face representation, together with a smart incremental learning mechanism that collects…”
Learning RoI Transformer for Oriented Object Detection in Aerial Images (2019, background): “As pointed out in [13, 30], appropriate enlargement of the RoI will promote the performance.”
Recent Advances in Deep Learning for Object Detection (2019, background): “Image Pyramid: An intuitive idea is to resize input images into a number of different scales (Image Pyramid) and to train multiple detectors, each of which is responsible for a certain range of scales [98, 99, 100, 101].”
Memory Based Online Learning of Deep Representations from Video Streams (2017, methodology): “erged across multiple shots into ﬁnal trajectories according to the Hierarchical Agglomerative Clustering. We reported two alternative versions using the (Deformable Part Model) DPM [67] and the Tiny [68] face detectors. They are indicated as MuFTiR-DPM and MuFTiRTINY respectively. For such comparis”
Detecting small faces in the wild based on generative adversarial network and contextual information (2019, methodology): “Hu and Ramanan [47] explores three problems in the context of detecting tiny faces, ( i.e. the role of scale in-variance, image resolution, and contextual reasoning), and shows context is crucial for ﬁnding tiny faces in the wild.”
SSH: Single Stage Headless Face Detector (2017, methodology): “Sur-prisingly, SSH also outperforms HR based on ResNet-101 on the whole dataset ( i.e .”
Gaze360: Physically Unconstrained Gaze Estimation in the Wild (2019, background): “In recent years, while methods for related human modeling problems such as 2D body pose and face tracking have achieved impressive success by leveraging the representational power of deep convolutional neural networks along with very large annotated datasets [2, 6, 9, 14, 26], methods for gaze estim”
Scale Match for Tiny Person Detection (2019, positive): “Hu et al. [12] showed that the context is crucial and deﬁnes the templates that make use of massively large receptive ﬁelds.”
Augmentation for small object detection (2019, background): “[6,33,8,19] add the context when cropping a small object proposal.”